Characterization and Control of Hysteretic Dynamics Using Online Reinforcement Learning Conference Paper uri icon

abstract

  • Hysteretic dynamical systems are challenging to control due to their hard nonlinearity and difficulty in modeling. One type of system with hysteretic dynamics that is gaining use in aerospace systems is the shape-memory alloy-based actuator. These actuators provide aircraft and spacecraft systems with the ability to achieve component-level or vehicle-level geometry or shape changes. Characterization of the material dynamics and properties of these actuators is usually accomplished with empirical testing of physical specimens, in which the hysteresis dynamics are often abstracted to very simplified models or ignored entirely. Machine learning techniques have the potential to learn hysteretic dynamics, but they routinely encounter difficulties that make them unsuitable. This paper proposes and develops a reinforcement learning-based approach that directly learns an input-output mapping characterization of hysteretic dynamics, which is then used as a control policy. A hyperbolic tangent-based model is used to develop a simulation of a shape-memory alloy, which is then validated experimentally using the Sarsa algorithm. The simulation model produces the temperature-versus-strain behavior and characterizes both the major and minor hysteresis loops. The learning results produce a near-optimal control policy for modulating a shape-memory alloy wire to a specified length. Results presented in the paper show that casting the shape-memory alloy control problem as a reinforcement learning problem shows promise for characterizing and controlling shape-memory alloy hysteresis behavior. Copyright 2012 by Jochem Berends MScAE.

published proceedings

  • JOURNAL OF AEROSPACE INFORMATION SYSTEMS

author list (cited authors)

  • Kirkpatrick, K., Valasek, J., & Haag, C.

citation count

  • 4

complete list of authors

  • Kirkpatrick, Kenton||Valasek, John||Haag, Chris

publication date

  • June 2013