Multiresolution State-Space Discretization Method for Q-Learning Conference Paper

abstract

• For large scale problems Q-Learning often suffers from the Curse of Dimensionality due to large numbers of possible state-action pairs. This paper develops a multiresolution state-space discretization method for the episodic unsupervised learning method of Q-Learning, in which a state-space is adaptively discretized by progressively finer grids around the areas of interest within the state or learning space. Optimality of the learning algorithm is addressed by a cost function. Applied to a morphing airfoil with two morphing parameters (two state variables), it is shown that by setting the multiresolution method to define the area of interest by the goal the agent seeks, this method can learn a specific goal within 0.002, while reducing the total number of state-action pairs need to achieve this level of specificity by almost 90%. 2009 AACC.

name of conference

• 2009 American Control Conference

authors

• Valasek, John

published proceedings

• 2009 AMERICAN CONTROL CONFERENCE, VOLS 1-9

author list (cited authors)

• Lampton, A., & Valasek, J.

citation count

• 7

complete list of authors

• Lampton, Amanda||Valasek, John

publication date

• January 2009

publisher

• Institute of Electrical and Electronics Engineers (IEEE)  Publisher

published in

• Proceedings of the ... American Control Conference. American Control Conference  Journal

keywords

• 46 Information And Computing Sciences
• 4611 Machine Learning

Digital Object Identifier (DOI)

• 10.1109/ACC.2009.5160474

International Standard Book Number (ISBN) 13

• 978-1-4244-4523-3

start page

• 1646

end page

• 1651

URL

• http://dx.doi.org/10.1109/acc.2009.5160474