Traffic Signal Control with Deep Reinforcement Learning Toward Enhanced Resilience of Urban Road Networks

Qiannian Xiang; Zaoli Yang; Haibo Chen; Washington Ochieng; Daoping Wang; Chi Xie; Wen-Long Shang

doi:10.1016/j.eng.2026.05.016

Engineering ›› :202605016 DOI: 10.1016/j.eng.2026.05.016

research-article

Traffic Signal Control with Deep Reinforcement Learning Toward Enhanced Resilience of Urban Road Networks

Author information +

History +

PDF (3300KB)

Abstract

The frequency of weather events associated with climate change has increased, heightening the risk of disruption and damage to urban road networks (URNs). Because critical infrastructure underpins urban development and socioeconomic activity, URNs require enhanced resilience to maintain stable operations under adverse conditions. Therefore, strengthening the resilience of URNs has become a pressing priority for infrastructure management. Deep reinforcement learning (DRL) has demonstrated considerable potential for urban traffic management because of its ability to perceive, learn, and act in complex, dynamic environments. However, few studies have developed DRL-based strategies that explicitly target resilience while accounting for travelers’ behavioral responses. To address this gap, this study proposes a DRL-based signal control strategy to enhance the resilience of URNs by improving their robustness and rapidity of recovery under extreme conditions. The proposed approach integrates a within-day/day-to-day dynamic traffic assignment (DTD-DTA) model with a DRL framework to capture the joint evolution of traffic flow and traveler behavior. A relative area index (RAI) is introduced as a quantitative resilience metric and incorporated into the reward function, enabling the agent to dynamically adjust signal timings and influence traveler route choices during and after disruptions. Experimental results on the Sioux Falls network demonstrate that the proposed strategy significantly outperforms conventional signal control methods across a range of disturbance scenarios, achieving resilience improvements of 23.8%–69.2% under severe conditions. Additional validation on the Anaheim network confirms the adaptability and transferability of the approach. This study provides a practical pathway for disruption-ready urban traffic management and demonstrates the alignment of artificial intelligence with resilience objectives in critical infrastructure systems.

Keywords

Resilience enhancement / Deep reinforcement learning / Traffic signal control / Urban road networks

Cite this article

Download citation ▾

Qiannian Xiang, Zaoli Yang, Haibo Chen, Washington Ochieng, Daoping Wang, Chi Xie, Wen-Long Shang. Traffic Signal Control with Deep Reinforcement Learning Toward Enhanced Resilience of Urban Road Networks. Engineering 202605016 DOI:10.1016/j.eng.2026.05.016

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Bai X. Build networked resilience across cities. Science 2024;383(6687):eado5304.

[2]	Ahmed N, Lee J, Liu L, Kim J, Jang KM, Wang J . The cost of climate change: a generalized cost function approach for incorporating extreme weather exposure into public transit accessibility. Comput Environ Urban Syst 2024; 112:102145.

[3]	Chrysoulakis N, Ludlow D, Mitraka Z, Somarakis G, Khan Z, Lauwaet D, et al. Copernicus for urban resilience in Europe. Sci Rep 2023; 13(1):16251.

[4]	Shang WL, Ochieng W, Chen Y, Xie C . Resilience of transportation systems under uncertainty. Transp Res Part A Policy Pract 2025; 191:104306.

[5]	Feng K, Lin N . Reconstructing and analyzing the traffic flow during evacuation in Hurricane Irma (2017). Transp Res Part D Transp Environ 2021; 94:102788.

[6]	Zhao X, Li H, Cai Q, Pan Y, Qi Y . Managing extreme rainfall and flooding events: a case study of the 20 July 2021 Zhengzhou Flood in China. Climate 2023;11(11):228.

[7]	Ganin AA, Kitsak M, Marchese D, Keisler JM, Seager T, Linkov I . Resilience and efficiency in transportation networks. Sci Adv 2017; 3(12):e1701079.

[8]	Ma F, Ao Y, Wang X, He H, Liu Q, Yang D, et al. Assessing and enhancing urban road network resilience under rainstorm waterlogging disasters. Transp Res Part D Transp Environ 2023; 123:103928.

[9]	Yadav N, Chatterjee S, Ganguly AR . Resilience of urban transport network—of—networks under intense flood hazards exacerbated by targeted attacks. Sci Rep 2020; 10(1):10350.

[10]	Li C, Wang W, Solé—Ribalta A, Borge—Holthoefer J, Jia B, Liu Z, et al. Adaptive capacity for multimodal transport network resilience to extreme floods. Nat Sustain 2025; 8(7):741—52.

[11]	Gu Y, Fu X, Liu Z, Xu X, Chen A . Performance of transportation network under perturbations: reliability, vulnerability, and resilience. Transp Res, Part E Logist Trans Rev 2020; 133:101809.

[12]	Wang Y, Wang J . Measuring and maximizing resilience of transportation systems for emergency evacuation. IEEE Trans Eng Manage 2020; 67(3):603—13.

[13]	Fang X, Lu L, Li Y, Hong Y . A driver—pressure—state—impact—response study for urban transport resilience under extreme rainfall—flood conditions. Transp Res Part D Transp Environ 2023; 121:103819.

[14]	Bruneau M, Chang SE, Eguchi RT, Lee GC, O’Rourke TD, Reinhorn AM, et al. A framework to quantitatively assess and enhance the seismic resilience of communities. Earthq Spectra 2003; 19(4):733—52.

[15]	Gao Z, Jia B, Xie D, Wang W, Wu J . A discussion on the complexity and transit mechanisms of urban traffic systems. Engineering 2025; 44:24-9.

[16]	Cascetta E, Cantarella GE . A day—to—day and within—day dynamic stochastic assignment model. Transp Res Part A Gen 1991; 25(5):277-91.

[17]	Watling DP, Cantarella GE . Modelling sources of variation in transportation systems: theoretical foundations of day—to—day dynamic models. Transportmetrica B Transp Dyn 2013;1(1):3-32.

[18]	Djavadian S, Chow JYJ . An agent—based day—to—day adjustment process for modeling ‘Mobility as a Service’ with a two—sided flexible transport market. Transp Res Part B Methodol 2017; 104:36-57.

[19]	Kucharski R, Gentile G . Simulation of rerouting phenomena in dynamic traffic assignment with the information comply model. Transp Res Part B Methodol 2019; 126:414-41.

[20]	Xu X, Chen A, Xu G, Yang C, Lam WHK . Enhancing network resilience by adding redundancy to road networks. Transp Res Part E Logist Trans Rev 2021; 154:102448.

[21]	Zhang X, Mahadevan S, Goebel K . Network reconfiguration for increasing transportation system resilience under extreme events. Risk Anal 2019; 39(9):2054-75.

[22]	Haydari A, Yilmaz Y . Deep reinforcement learning for intelligent transportation systems: a survey. IEEE Trans Intell Transp Syst 2022; 23(1):11-32.

[23]	Cai C, Wei M . Adaptive urban traffic signal control based on enhanced deep reinforcement learning. Sci Rep 2024; 14(1):14116.

[24]	Shang WL, Chen Y, Ochieng WY . Resilience analysis of transport networks by combining variable message signs with agent—based day—to—day dynamic learning. IEEE Access 2020; 8:104458-68.

[25]	Zhao Z, Wang K, Wang Y, Liang X . Enhancing traffic signal control with composite deep intelligence. Expert Syst Appl 2024; 244:123020.

[26]	Holling CS . Foundations of socio—environmental research: legacy readings with commentaries. Cambridge: Cambridge University Press; 2022.

[27]	Movahednia M, Kargarian A . Community—informed resource allocation optimization for power grid resilience against flood hazards. Electr Power Syst Res 2025; 247:111695.

[28]	Duque Monsalve LF, Navarrete Valladares CP, Sandoval Díaz J . Relationship between political participation and community resilience in the disaster risk process: a systematic review. Int J Disaster Risk Reduct 2024; 111:104751.

[29]	Trippl M, Fastenrath S, Isaksen A . Rethinking regional economic resilience: preconditions and processes shaping transformative resilience. Eur Urban Reg Stud 2024; 31(2):101—15.

[30]	Cui X, Li B, Zhang S, Ji Z, Wang S, Luo R, et al. Resilience—based restoration sequence optimization of disrupted transportation networks: a novel matheuristic approach. Transp Res Part D Transp Environ 2025; 145:104834.

[31]	Wang L, Yang X, Wu J, Xu X, Qu Y, Lee DH . Dual—assessment of resilience and recovery strategies of road networks under rainstorm—induced waterlogging disasters. Transportmetrica A Transp Sci 2024 Nov:1-32.

[32]	Su W, Zhang L, Lu J, Zhu J . Crude oil maritime transportation network resilience assessment based on optimal strategies. Transp Res Part D Transp Environ 2025; 142:104659.

[33]	Dunn S, Wilkinson SM . Increasing the resilience of air traffic networks using a network graph theory approach. Transp Res, Part E Logist Trans Rev 2016; 90:39-50.

[34]	Chen J, Liu X, Hu X, Du B, Yin Y . Resilience assessment of an urban rail transit network under natural disasters. Transp Res Part D Transp Environ 2025; 139:104587.

[35]	Zhang W, Han Q, Shang WL, Xu C . Seismic resilience assessment of interdependent urban transportation—electric power system under uncertainty. Transp Res Part A Policy Pract 2024; 183:104078.

[36]	Farahmand H, Yin K, Hsu CW, Savadogo I, Espinet Alegre X, Mostafavi A . Integrating climate projections and probabilistic network analysis into regional transport resilience planning. Transp Res Part D Transp Environ 2024; 133:104229.

[37]	Zhao T, Zhang Y . Transportation infrastructure restoration optimization considering mobility and accessibility in resilience measures. Transp Res Part C Emerg Technol 2020; 117:102700.

[38]	Yin K, Wu J, Wang W, Lee DH, Wei Y . An integrated resilience assessment model of urban transportation network: a case study of 40 cities in China. Transp Res Part A Policy Pract 2023; 173:103687.

[39]	Aydin NY, Duzgun HS, Wenzel F, Heinimann HR . Integration of stress testing with graph theory to assess the resilience of urban road networks under seismic hazards. Nat Hazards 2018; 91(1):37-68.

[40]	Cui Z, Zang D, Zhu H, Tang K . Predictive and multigranularity resilience assessment of urban transportation based on neural controlled differential equation. IEEE Trans Reliab 2024; 74(3):4230-44.

[41]	Mattsson LG, Jenelius E . Vulnerability and resilience of transport systems — a discussion of recent research. Transp Res Part A Policy Pract 2015; 81:16-34.

[42]	Zhou Y, Wang J, Yang H . Resilience of transportation systems: concepts and comprehensive review. IEEE Trans Intell Transp Syst 2019; 20(12):4262—76.

[43]	Miller—Hooks E, Zhang X, Faturechi R . Measuring and maximizing resilience of freight transportation networks. Comput Oper Res 2012; 39(7):1633—43.

[44]	Wei D, Rose A, Koc E, Chen Z, Soibelman L . Socioeconomic impacts of resilience to seaport and highway transportation network disruption. Transp Res Part D Transp Environ 2022; 106:103236.

[45]	Ye Q, Ukkusuri SV . Resilience as an Objective in the Optimal Reconstruction Sequence for Transportation Networks. J Transp Saf Secur 2015; 7(1):91-105.

[46]	Shang WL, Chen Y, Li X, Ochieng WY . Resilience analysis of urban road networks based on adaptive signal controls: day—to—day traffic dynamics with deep reinforcement learning. Complexity 2020; 2020:1-19.

[47]	Asadabadi A, Miller—Hooks E . Optimal transportation and shoreline infrastructure investment planning under a stochastic climate future. Transp Res Part B Methodol 2017; 100:156-74.

[48]	Xu X, Chen A, Jansuwan S, Yang C, Ryu S . Transportation network redundancy: complementary measures and computational methods. Transp Res Part B Methodol 2018; 114:68-85.

[49]	Qu K, Xu X, Zhou W, Chen A . Retrofit or new construction? Strategic budget allocation to improve transportation network redundancy under uncertain disruptions. Transp Res Part E Logist Trans Rev 2025; 198:104131.

[50]	Serdar MZ, Koç M, Al—Ghamdi SG . Urban transportation networks resilience: indicators, disturbances, and assessment methods. Sustain Cities Soc 2022; 76:103452.

[51]	Thorisson H, Lambert JH . Multiscale identification of emergent and future conditions along corridors of transportation networks. Reliab Eng Syst Saf 2017; 167:255—63.

[52]	Zhu C, Wen G, Li N, Bian L, Wu J, Kouvelas A . Resilience enhancement of urban roadway network during disruption via perimeter control. IEEE Trans Netw Sci Eng 2024; 11(1):1227—37.

[53]	Bagloee SA, Johansson KH, Asadi M . A hybrid machine—learning and optimization method for contraflow design in post—disaster cases and traffic management scenarios. Expert Syst Appl 2019; 124:67-81.

[54]	Abudayyeh D, Nicholson A, Ngoduy D . Traffic signal optimisation in disrupted networks, to improve resilience and sustainability. Travel Behav Soc 2021; 22:117—28.

[55]	Tao W, Wang ZZ, Liu CJ, Lu YN, Zhang Y, Jiang ZH . Resilience assessment and enhancement of urban road networks subject to traffic accidents: a network—scale optimization strategy. J Intell Transp Syst 2024; 28(4):494-510.

[56]	Chiou SW . A traffic—responsive signal control to enhance road network resilience with hazmat transportation in multiple periods. Reliab Eng Syst Saf 2018; 175:105-18.

[57]	Noaeen M, Naik A, Goodman L, Crebo J, Abrar T, Abad ZSH, et al. Reinforcement learning in urban network traffic signal control: a systematic literature review. Expert Syst Appl 2022; 199:116830.

[58]	Sattarzadeh AR, Pathirana PN . Unification of probabilistic graph model and deep reinforcement learning (UPGMDRL) for multi—intersection traffic signal control. Knowl Base Syst 2024; 305:112663.

[59]	Khan M, Imran Kays H, Pittenger DM, Miller GA, Sadri AM . Identifying traffic control mechanism to improve the level of service and resilience in road networks based on topological credentials. Transp Res Interdiscip Perspect 2025; 34:101729.

[60]	Wei Y, Yang X, Xiao X, Ma Z, Zhu T, Dou F, et al. Understanding the resilience of urban rail transit: concepts, reviews, and trends. Engineering 2024; 41:7-18.

[61]	Yang Y, Huang H, Li G, Han B, Yuan Z, Ma H . A systematic review of resilience assessment and enhancement of urban integrated transportation networks. J Transp Geogr 2025; 129:104420.

[62]	Zhao H, Dong C, Cao J, Chen Q . A survey on deep reinforcement learning approaches for traffic signal control. Eng Appl Artif Intell 2024; 133:108100.

[63]	Wang T, Zhu Z, Zhang J, Tian J, Zhang W . A large—scale traffic signal control algorithm based on multi—layer graph deep reinforcement learning. Transp Res Part C Emerg Technol 2024; 162:104582.

[64]	Yu F, Mi X, Yu C, Jiang Y . Distributed traffic signal control model for accurate policy learning under dynamic traffic flow: a graph forecast—state vector driven deep reinforcement learning framework. IEEE Trans Intell Transp Syst 2025; 26(9):13573—84.

[65]	Zhang G, Chang F, Jin J, Yang F, Huang H . Multi—objective deep reinforcement learning approach for adaptive traffic signal control system with concurrent optimization of safety, efficiency, and decarbonization at intersections. Accid Anal Prev 2024; 199:107451.

[66]	Wang L, Zhang G, Yang Q, Han T . An adaptive traffic signal control scheme with proximal policy optimization based on deep reinforcement learning for a single intersection. Eng Appl Artif Intell 2025; 149:110440.

[67]	Stevanovic A, Stevanovic J, So J, Ostojic M . Multi—criteria optimization of traffic signals: mobility, safety, and environment. Transp Res Part C Emerg Technol 2015; 55:46-68.

[68]	Bouktif S, Cheniki A, Ouni A, El—Sayed H . Parameterized—action based deep reinforcement learning for intelligent traffic signal control. Eng Appl Artif Intell 2025; 159:111422.

[69]	Yan L, Zhu L, Song K, Yuan Z, Yan Y, Tang Y, et al. Graph cooperation deep reinforcement learning for ecological urban traffic signal control. Appl Intell 2023; 53(6):6248-65.

[70]	Shang WL, Song X, Xiang Q, Chen H, Elhajj M, Bi H, et al. The impact of deep reinforcement learning—based traffic signal control on emission reduction in urban Road networks empowered by cooperative vehicle—infrastructure systems. Appl Energy 2025; 390:125884.

[71]	Gong Y, Abdel—Aty M, Cai Q, Rahman MS . Decentralized network level adaptive signal control by multi—agent deep reinforcement learning. Transp Res Interdiscip Perspect 2019; 1:100020.

[72]	Li D, Zhu F, Wu J, Wong YD, Chen T . Managing mixed traffic at signalized intersections: an adaptive signal control and CAV coordination system based on deep reinforcement learning. Expert Syst Appl 2024; 238:121959.

[73]	Shang WL . Robustness and resilience analysis of urban road networks [dissertation]. London:Imperial College London; 2017.

[74]	Wang Y, Szeto WY, Han K, Friesz TL . Dynamic traffic assignment: a review of the methodological advances for environmentally sustainable road transportation applications. Transp Res Part B Methodol 2018; 111:370-94.

[75]	Shang W, Han K, Ochieng W, Angeloudis P . Agent—based day—to—day traffic network model with information percolation. Transportmetrica A Transp Sci 2017; 13(1):38-66.

[76]	Han K, Piccoli B, Szeto WY . Continuous—time link—based kinematic wave model: formulation, solution existence, and well—posedness. Transportmetrica B Transp Dyn 2016; 4(3):187-222.

[77]	Fu H, Akamatsu T, Satsukawa K, Wada K . Dynamic traffic assignment in a corridor network: optimum versus equilibrium. Transp Res Part B Methodol 2022; 161:218-46.

[78]	Wardrop JG . Road paper. Some theoretical aspects of road traffic research. Proc— Inst Civ Eng 1952; 1(3):325-62.

[79]	Hoang NH, Vu HL, Panda M, Lo HK . A linear framework for dynamic user equilibrium traffic assignment in a single origin—destination capacitated network. Transp Res Part B Methodol 2019; 126:329-52.

[80]	Yu Y, Han K, Ochieng W . Day—to—day dynamic traffic assignment with imperfect information, bounded rationality and information sharing. Transp Res Part C Emerg Technol 2020; 114:59-83.

[81]	Han K, Eve G, Friesz TL . Computing dynamic user equilibria on large—scale networks with software implementation. Netw Spat Econ 2019; 19(3):869-902.

[82]	Puterman ML . Markov decision processes: discrete stochastic dynamic programming. Hoboken: Wiley; 1994.

[83]	Watkins CJCH, Dayan P . Q—learning. Mach Learn 1992; 8(3—4):279-92.

[84]	Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, et al. Human—level control through deep reinforcement learning. Nature 2015; 518(7540):529—33.

[85]	Bellman RE . Dynamic programming. Princeton: Princeton University Press; 1957.

[86]	Rumelhart DE, Hinton GE, Williams RJ . Learning representations by back—propagating errors. Nature 1986; 323(6088):533-6.

[87]	Shang WL, Gao Z, Daina N, Zhang H, Long Y, Guo Z, et al. Benchmark analysis for robustness of multi—scale urban road networks under global disruptions. IEEE Trans Intell Transp Syst 2023; 24(12):15344—54.

[88]	Ouyang M, Cheng Z, Ma J, Wang H, Mitoulis SA . Coupled urban risks: a complex systems perspective with a people—centric focus. Engineering 2025;44:44-50.