Convergence Properties of a Computational Learning Model for Unknown Markov Chains

Andreas A. Malikopoulos

doi:10.1115/1.3117202

Journal of Dynamic Systems, Measurement, and Control | Volume 131 | Issue 4 | Research Paper

< Previous Article Next Article >

Research Papers

Convergence Properties of a Computational Learning Model for Unknown Markov Chains

Andreas A. Malikopoulos

[+] Author and Article Information

Andreas A. Malikopoulos

Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109amaliko@umich.edu

J. Dyn. Sys., Meas., Control 131(4), 041011 (May 20, 2009) (7 pages) doi:10.1115/1.3117202 History: Received March 18, 2008; Revised February 04, 2009; Published May 20, 2009

Article

References

Figures

Citing Articles

Abstract

The increasing complexity of engineering systems has motivated continuing research on computational learning methods toward making autonomous intelligent systems that can learn how to improve their performance over time while interacting with their environment. These systems need not only to sense their environment, but also to integrate information from the environment into all decision-makings. The evolution of such systems is modeled as an unknown controlled Markov chain. In a previous research, the predictive optimal decision-making (POD) model was developed, aiming to learn in real time the unknown transition probabilities and associated costs over a varying finite time horizon. In this paper, the convergence of the POD to the stationary distribution of a Markov chain is proven, thus establishing the POD as a robust model for making autonomous intelligent systems. This paper provides the conditions that the POD can be valid, and be an interpretation of its underlying structure.

FIGURES IN THIS ARTICLE

Purchase this Content

$25.00

Purchase

Learn about subscription and purchase options

Your Session has timed out. Please sign back in to continue.

References

Figures

View Large | Save Figure | Download Slide (.ppt)

Figure 1

Stochastic system model schematic

Tables

Errata

Discussions

Web of Science® Times Cited: 2

Some tools below are only available to our subscribers or users with an online account.

PDF
Email

You must be logged in as an individual user to share content.

Return to: Convergence Properties of a Computational Learning Model for Unknown Markov Chains

Copyright in the material you requested is held by the American Society of Mechanical Engineers (unless otherwise noted). This email ability is provided as a courtesy, and by using it you agree that you are requesting the material solely for personal, non-commercial use, and that it is subject to the American Society of Mechanical Engineers' Terms of Use. The information provided in order to email this topic will not be used to send unsolicited email, nor will it be furnished to third parties. Please refer to the American Society of Mechanical Engineers' Privacy Policy for further information.
Share
Get Citation

Malikopoulos AA. Convergence Properties of a Computational Learning Model for Unknown Markov Chains. ASME. J. Dyn. Sys., Meas., Control. 2009;131(4):041011-041011-7. doi:10.1115/1.3117202.

Download citation file:

RIS (Zotero)

EndNote

BibTex

Medlars

ProCite

RefWorks

Reference Manager

© Copyright
Get Alerts

Processing your request... Please Wait.

Convergence Properties of a Computational Learning Model for Unknown Markov Chains

Abstract

FIGURES IN THIS ARTICLE

References

Figures

Tables

Errata

Discussions

Related Content

Journals

Conference Proceedings

eBooks