UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Adaptive signal control using approximate dynamic programming

Cai, C.; Heydecker, B.; (2009) Adaptive signal control using approximate dynamic programming. Presented at: 41st Annual Universities' Transport Study Group Conference, UCL (University College London), London, UK. Green open access


Download (344kB)


This paper presents a concise summary of a study on adaptive traffic signal controller for real time operation. The adaptive controller is designed to achieve three operational objectives: first, the controller adopts a dual control principle to achieve a balanced influence between immediate cost and long-term cost in operation; second, controller switches signals without referring to a preset plan and is acyclic; third, controller adjusts its parameters online to adapt new environment. Not all of these features are available in existing operational controllers. Although dynamic programming (DP) is the only exact solution for achieving the operational objectives, it is usually impractical for real time operation because of demand in computation and information. To circumvent the difficulties, we use approximate dynamic programming (ADP) in conjunction with online learning techniques. This approach can substantially reduce computational burden by replacing the exact value function of DP with a continuous linear approximation function, which is then updated progressively by online learning techniques. Two online learning techniques, which are reinforcement learning and monotonicity approximation respectively, are investigated. We find in computer simulation that the ADP controller leads to substantial savings in vehicle delays in comparison with optimised fixed-time plans. The implications of this study to traffic control are: the ADP controller meet all of the three operational objectives with competitive results, and can be readily implemented for operations at both isolated intersection and traffic networks; the ADP algorithm is computationally efficient, and the ADP controller is an evolving system that requires minimum human intervention; the ADP technique offers a flexible theoretical framework in which a range of functional forms and learning techniques can be further studied.

Type: Conference item (Presentation)
Title: Adaptive signal control using approximate dynamic programming
Event: 41st Annual Universities' Transport Study Group Conference
Location: UCL (University College London), London, UK
Dates: 5 - 7 January 2009
Open access status: An open access version is available from UCL Discovery
Publisher version: http://www.utsg.net/publication.php?Year=2009&abst...
Language: English
Additional information: Paper presented in Session 06A Modelling (II)
UCL classification: UCL > School of BEAMS > Faculty of Engineering Science > Civil, Environmental and Geomatic Engineering
URI: http://discovery.ucl.ac.uk/id/eprint/18970
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item