A new family of optimal adaptive controllers for Markov chains Academic Article uri icon


  • We consider the problem of adaptively controlling a Markov chain with unknown transition probabilities. A new family of adaptive controllers is exhibited which achieves a performance precisely equalling the optimal performance achievable if the transition probabilities (i.e., the model or dynamics of the system) were known instead. Hence, the adaptive controllers presented here are truly optimal The performance of the system to be controlled is measured by the average of the costs incurred over an infinite operating time period. These adaptive controllers can, potentially, be implemented on digital computers and used in control of unknown systems. 1982 IEEE

published proceedings

  • IEEE Transactions on Automatic Control

altmetric score

  • 3

author list (cited authors)

  • Kumar, P., & Becker, A.

citation count

  • 57

complete list of authors

  • Kumar, P||Becker, A

publication date

  • February 1982