Xie, Yuanchang (2007-12). Development and evaluation of an arterial adaptive traffic signal control system using reinforcement learning. Doctoral Dissertation. Thesis uri icon

abstract

  • This dissertation develops and evaluates a new adaptive traffic signal control system for arterials. This control system is based on reinforcement learning, which is an important research area in distributed artificial intelligence and has been extensively used in many applications including real-time control. In this dissertation, a systematic comparison between the reinforcement learning control methods and existing adaptive traffic control methods is first presented from the theoretical perspective. This comparison shows both the connections between them and the benefits of using reinforcement learning. A Neural-Fuzzy Actor-Critic Reinforcement Learning (NFACRL) method is then introduced for traffic signal control. NFACRL integrates fuzzy logic and neural networks into reinforcement learning and can better handle the curse of dimensionality and generalization problems associated with ordinary reinforcement learning methods. This NFACRL method is first applied to isolated intersection control. Two different implementation schemes are considered. The first scheme uses a fixed phase sequence and variable cycle length, while the second one optimizes phase sequence in real time and is not constrained to the concept of cycle. Both schemes are further extended for arterial control, with each intersection being controlled by one NFACRL controller. Different strategies used for coordinating reinforcement learning controllers are reviewed, and a simple but robust method is adopted for coordinating traffic signals along the arterial. The proposed NFACRL control system is tested at both isolated intersection and arterial levels based on VISSIM simulation. The testing is conducted under different traffic volume scenarios using real-world traffic data collected during morning, noon, and afternoon peak periods. The performance of the NFACRL control system is compared with that of the optimized pre-timed and actuated control. Testing results based on VISSIM simulation show that the proposed NFACRL control has very promising performance. It outperforms optimized pre-timed and actuated control in most cases for both isolated intersection and arterial control. At the end of this dissertation, issues on how to further improve the NFACRL method and implement it in real world are discussed.
  • This dissertation develops and evaluates a new adaptive traffic signal control
    system for arterials. This control system is based on reinforcement learning, which is an
    important research area in distributed artificial intelligence and has been extensively
    used in many applications including real-time control.
    In this dissertation, a systematic comparison between the reinforcement learning
    control methods and existing adaptive traffic control methods is first presented from the
    theoretical perspective. This comparison shows both the connections between them and
    the benefits of using reinforcement learning. A Neural-Fuzzy Actor-Critic
    Reinforcement Learning (NFACRL) method is then introduced for traffic signal control.
    NFACRL integrates fuzzy logic and neural networks into reinforcement learning and can
    better handle the curse of dimensionality and generalization problems associated with
    ordinary reinforcement learning methods.
    This NFACRL method is first applied to isolated intersection control. Two
    different implementation schemes are considered. The first scheme uses a fixed phase sequence and variable cycle length, while the second one optimizes phase sequence in
    real time and is not constrained to the concept of cycle. Both schemes are further
    extended for arterial control, with each intersection being controlled by one NFACRL
    controller. Different strategies used for coordinating reinforcement learning controllers
    are reviewed, and a simple but robust method is adopted for coordinating traffic signals
    along the arterial.
    The proposed NFACRL control system is tested at both isolated intersection and
    arterial levels based on VISSIM simulation. The testing is conducted under different
    traffic volume scenarios using real-world traffic data collected during morning, noon,
    and afternoon peak periods. The performance of the NFACRL control system is
    compared with that of the optimized pre-timed and actuated control.
    Testing results based on VISSIM simulation show that the proposed NFACRL
    control has very promising performance. It outperforms optimized pre-timed and
    actuated control in most cases for both isolated intersection and arterial control. At the
    end of this dissertation, issues on how to further improve the NFACRL method and
    implement it in real world are discussed.

publication date

  • December 2007