Optimal Output Regulation for Model-Free Quanser Helicopter With Multistep Q-Learning Academic Article

abstract

• 1982-2012 IEEE. In this paper, the optimal output regulation problem is considered for the model-free 2-degree-of-freedom (2-DOF) helicopter. A multistep Q-learning (MsQL) method is developed with multistep policy evaluation. First, by introducing the Q-function, the optimal output regulation problem is converted to finding the optimal Q-function. Therefore, the MsQL algorithm is proposed and its convergence theory is established by showing that it generates a nonincreasing Q-function sequence that converges to the optimal Q-function. In the MsQL, the step-size of multistep policy evaluation can be different at each iteration and an adaptive tuning rule is proposed. The MsQL learns the optimal Q-function by using real system data rather than using a system model. Finally, the developed MsQL method is employed to solve the optimal output regulation problem of the model-free 2-DOF helicopter, and its effectiveness is verified.

authors

• Huang, Tingwen

published proceedings

• IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

author list (cited authors)

• Luo, B., Wu, H., & Huang, T.

citation count

• 62

complete list of authors

• Luo, Biao||Wu, Huai-Ning||Huang, Tingwen

publication date

• June 2018

publisher

• Institute of Electrical and Electronics Engineers (IEEE)  Publisher

published in

• IEEE Transactions on Industrial Electronics  Journal

keywords

• Helicopter
• Model-free
• Multistep Policy Evaluation
• Optimal Output Regulation
• Q-learning

Digital Object Identifier (DOI)

• 10.1109/TIE.2017.2772162

start page

• 4953

end page

• 4961

volume

• 65

issue

• 6

URL

• http://dx.doi.org/10.1109/tie.2017.2772162