Optimal Output Regulation for Model-Free Quanser Helicopter With Multistep Q-Learning Academic Article uri icon


  • 1982-2012 IEEE. In this paper, the optimal output regulation problem is considered for the model-free 2-degree-of-freedom (2-DOF) helicopter. A multistep Q-learning (MsQL) method is developed with multistep policy evaluation. First, by introducing the Q-function, the optimal output regulation problem is converted to finding the optimal Q-function. Therefore, the MsQL algorithm is proposed and its convergence theory is established by showing that it generates a nonincreasing Q-function sequence that converges to the optimal Q-function. In the MsQL, the step-size of multistep policy evaluation can be different at each iteration and an adaptive tuning rule is proposed. The MsQL learns the optimal Q-function by using real system data rather than using a system model. Finally, the developed MsQL method is employed to solve the optimal output regulation problem of the model-free 2-DOF helicopter, and its effectiveness is verified.

published proceedings


author list (cited authors)

  • Luo, B., Wu, H., & Huang, T.

citation count

  • 57

complete list of authors

  • Luo, Biao||Wu, Huai-Ning||Huang, Tingwen

publication date

  • June 2018