Integrated Process Planning, Reconfiguration, and Scheduling Optimization for Smart Manufacturing Systems with Reconfigurable Machine Tools Based on Deep Reinforcement Learning

Ming Huang; Sihan Huang; Guangyu Mo; Shokraneh K. Moghaddam; Zhiheng Zhao; Mohammed Dahane; Guoxin Wang; Yan Yan

doi:10.1016/j.eng.2026.05.019

Engineering ›› :202605019 DOI: 10.1016/j.eng.2026.05.019

research-article

Integrated Process Planning, Reconfiguration, and Scheduling Optimization for Smart Manufacturing Systems with Reconfigurable Machine Tools Based on Deep Reinforcement Learning

Author information +

History +

PDF (3417KB)

Abstract

Under Industry 4.0, market demand is rapidly transitioning from a reliance on conventional mass customization to requiring mass personalized customization, calling for manufacturing systems to be equipped with reconfigurable machine tools (RMTs) that offer greater flexibility and efficiency and are driven by smart technologies. In general, the effect of demand fluctuations on the operation of smart manufacturing systems (SMSs) is subject to a prolonged propagation process, especially when the optimization of key production segments such as process planning, reconfiguration, and scheduling lacks organic integration, resulting in low efficiency and responsiveness. To tackle this issue, this article performs integrated optimization of process planning, reconfiguration, and scheduling for SMSs with RMTs, considering flexible process routes, adaptive RMT reconfiguration, and flexible scheduling. A deep reinforcement learning (DRL)-based approach is proposed to address this integrated optimization problem. A reward function based on penalty and prediction enhancement is devised to guide the learning progress, and a double deep Q-network (DQN)-based training framework is adopted to explore the best decision actions for each state. Finally, a real-world industry case study and extended numerical experiments are conducted to demonstrate the practical applicability and superiority of this integrated optimization approach. Moreover, to further explore the empowerment of industrial optimization through generative artificial intelligence (GAI), the potential of adopting large language models (LLMs) to address integrated optimization problems is discussed.

Keywords

Industry 4.0 / Smart manufacturing systems / Integrated optimization / Reconfigurable machine tools / Scheduling optimization / Deep reinforcement learning / Large language models

Cite this article

Download citation ▾

Ming Huang, Sihan Huang, Guangyu Mo, Shokraneh K. Moghaddam, Zhiheng Zhao, Mohammed Dahane, Guoxin Wang, Yan Yan. Integrated Process Planning, Reconfiguration, and Scheduling Optimization for Smart Manufacturing Systems with Reconfigurable Machine Tools Based on Deep Reinforcement Learning. Engineering 202605019 DOI:10.1016/j.eng.2026.05.019

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Li X, Gu X, Huang S, Wang B, Leng J, Wang L . Smart reconfigurable manufacturing towards Industry 5.0. J Manuf Syst 2026; 86:178-202.

[2]	Huang S, Li X, Gu X, Koren Y . Design and operation of smart reconfigurable manufacturing systems in Industry 4.0/5.0. Cham: Springer; 2026.

[3]	Li X, Zheng P, Bao J, Gao L, Xu X . Achieving cognitive mass personalization via the Self—X cognitive manufacturing network: an industrial knowledge graph— and graph embedding—enabled pathway. Engineering 2023; 22:14-9.

[4]	Xiao R. Massive personalized customization: new development of mass personalization. Computer Integr Manuf Syst 2023; 29(12):4215—26. Chinese.

[5]	Lin Z, Wei K, Wang Y, Chen C, Guo J, Cheng Q, et al. An adaptive hybrid edge—cloud collaborative offloading method for large—scale computational tasks of intelligent machine tool: low—latency, energy—efficient, and secure. Engineering 2026; 56:201—18.

[6]	Li X, Gao L, Wang W, Wang C, Wen L . Particle swarm optimization hybridized with genetic algorithm for uncertain integrated process planning and scheduling with interval processing time. Comput Ind Eng 2019; 135:1036-46.

[7]	Koren Y, Heisel U, Jovane F, Moriwaki T, Pritschow G, Ulsoy G, et al. Reconfigurable manufacturing systems. CIRP Ann 1999; 48(2):527—40.

[8]	Huang S, Mo G, Jing S, Leng J, Li X, Gu X, et al. Digital twin—driven self—adaptive reconfiguration planning method of smart manufacturing systems using game theory and deep Q—network for Industry 5.0. J Ind Inf Integr 2025; 47:100901.

[9]	Huang M, Huang S, Gao L, Dong W, Zhang Y, Gu X, et al. Digital twin—driven multi—agent collaborative online optimization of production regulation for smart reconfigurable manufacturing systems with human—robot collaboration. Robot Comput—Integr Manuf 2026; 101:103322.

[10]	Huang S, Xu Z, Wang G, Zeng C, Yan Y . Reconfigurable machine tools design for multi—part families. Int J Adv Manuf Technol 2019; 105(1—4):813-29.

[11]	Vahedi—Nouri B, Rohaninejad M, Hanzálek Z, Foumani M . A batch production scheduling problem in a reconfigurable hybrid manufacturing—remanufacturing system. Comput Ind Eng 2025; 204:111099.

[12]	Bortolini M, Galizia FG, Mora C . Reconfigurable manufacturing systems: literature review and research trend. J Manuf Syst 2018; 49:93-106.

[13]	Morgan J, Halton M, Qiao Y, Breslin JG . Industry 4.0 smart reconfigurable manufacturing machines. J Manuf Syst 2021; 59:481-506.

[14]	Wang Y, Wang S, Yang B, Wang S . Dynamic adaptive reconfiguration method for cloud manufacturing service composition in practical multi—constraint environment. J Mech Eng 2023; 59(14):339-51. Chinese.

[15]	Tang H, Li D, Wan J, Imran M, Shoaib M . A reconfigurable method for intelligent manufacturing based on industrial cloud and edge intelligence. IEEE Internet Things J 2020; 7(5):4248-59.

[16]	Wan J, Tang S, Li D, Imran M, Zhang C, Liu C, et al. Reconfigurable smart factory for drug packing in healthcare Industry 4.0. IEEE Trans Industr Inform 2019; 15(1):507—16.

[17]	Chryssolouris G, Chan S, Cobb W. Decision making on the factory floor: an integrated approach to process planning and scheduling. Robot Comput—Integr Manuf 1984;1(3—4):315—9.

[18]	Liu Q, Li X, Gao L . A novel MILP model based on the topology of a network graph for process planning in an intelligent manufacturing system. Engineering 2021; 7(6):807—17.

[19]	Liu Q, Wang C, Li X, Gao L . An improved genetic algorithm with modified critical path—based searching for integrated process planning and scheduling problem considering automated guided vehicle transportation task. J Manuf Syst 2023; 70:127-36.

[20]	Demir H, Erden C . Dynamic integrated process planning, scheduling and due—date assignment using ant colony optimization. Comput Ind Eng 2020; 149:106799.

[21]	Tao F, Li Y, Wei Y, Zhang C, Zuo Y . Data—model fusion methods and applications toward smart manufacturing and digital engineering. Engineering 2025; 55:36-50.

[22]	Leng J, Wang D, Shen W, Li X, Liu Q, Chen X . Digital twins—based smart manufacturing system design in Industry 4.0: a review. J Manuf Syst 2021; 60(9):119-37.

[23]	Yang C, Liao F, Lan S, Wang L, Shen W, Huang GQ . Flexible resource scheduling for software—defined cloud manufacturing with edge computing. Engineering 2023; 22:60-70.

[24]	Li Y, Li X, Gao L . Real—time scheduling for production—logistics collaborative environment using multi—agent deep reinforcement learning. Adv Eng Inform 2025; 65(Pt B):103216.

[25]	Yu Z, Gong Y . ChatGPT, AI—generated content, and engineering management. Front Eng Manag 2024; 11(1):159-66.

[26]	Huang M, Huang S, Du B, Guo J, Li Y . Fuzzy superposition operation and knowledge—driven coevolutionary algorithm for integrated production scheduling and vehicle routing problem with soft time windows and fuzzy travel times. IEEE Trans Fuzzy Syst 2025; 33(12):4152-66.

[27]	Huang S, Wang G, Yan Y . Delayed reconfigurable manufacturing system. Int J Prod Res 2019; 57(8):2372—91.

[28]	Huang S, Tan J, Lu Y, Moghaddam SK, Wang G, Yan Y . A multi—objective joint optimisation method for simultaneous part family formation and configuration design in delayed reconfigurable manufacturing system (D—RMS). Int J Prod Res 2024; 62(1—2):92-109.

[29]	Pang Z, Yang B, Chen R, Zhang Z, Mo F . A multi—phase scheduling method for reconfigurable flexible job—shops with multi—machine cooperation based on a Scout and Mutation—based Aquila Optimizer. CIRO J Manuf Sci Technol 2023; 46:116-34.

[30]	Zhu Q, Huang S, Wang G, Moghaddam SK, Lu Y, Yan Y . Dynamic reconfiguration optimization of intelligent manufacturing system with human—robot collaboration based on digital twin. J Manuf Syst 2022; 65:330—8.

[31]	Li Y, Li X, Gao L, Lu Z . Multi—agent deep reinforcement learning for dynamic reconfigurable shop scheduling considering batch processing and worker cooperation. Robot Comput—Integr Manuf 2025; 91:102834.

[32]	Vahedi—Nouri B, Tavakkoli—Moghaddam R, Hanzálek Z, Dolgui A . Production scheduling in a reconfigurable manufacturing system benefiting from human—robot collaboration. Int J Prod Res 2024; 62(3):767-83.

[33]	Guo H, Liu J, Wang Y, Zhuang C . An improved genetic programming hyper—heuristic for the dynamic flexible job shop scheduling problem with reconfigurable manufacturing cells. J Manuf Syst 2024; 74:252-63.

[34]	Xu W, Li Z, Liu J, Cui J, Hu Y . Virtual reconfiguration method of robotic mixed—model assembly line using bees algorithm based on digital twin. IEEE Trans Autom Sci Eng 2024; 21(3):2211-22.

[35]	Rohaninejad M, Vahedi—Nouri B, Hanzálek Z, Tavakkoli—Moghaddam R . An integrated lot—sizing and scheduling problem in a reconfigurable manufacturing system under workforce constraints. Int J Prod Res 2024; 62(11):3994-4013.

[36]	Huang J, Huang S, Moghaddam SK, Lu Y, Wang G, Yan Y, et al. Deep reinforcement learning—based dynamic reconfiguration planning for digital twin—driven smart manufacturing systems with reconfigurable machine tools. IEEE Trans Industr Inform 2024; 20(11):13135—46.

[37]	Yao A, Gao K, Suganthan PN, Sang H . Online reinforcement learning strategies driven artificial bee colony algorithm for bi—objective distributed reentrant flowshops scheduling problem with sequence—sependent setup time. Expert Syst Appl 2026; 298(Part B):129716.

[38]	Park IB, Park J . Scalable scheduling of semiconductor packaging facilities using deep reinforcement learning. IEEE Trans Cybern 2023; 53(6):3518—31.

[39]	Yang S, Xu Z . Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing. Int J Prod Res 2022; 60(16):4936-53.

[40]	Shen W, Wang L, Hao Q . Agent—based distributed manufacturing process planning and scheduling: a state—of—the—art survey. IEEE Trans Syst Man Cybern C 2006; 36(4):563-77.

[41]	Li X, Gao L, Pan Q, Wan L, Chao KM . An effective hybrid genetic algorithm and variable neighborhood search for integrated process planning and scheduling in a packaging machine workshop. IEEE Trans Syst Man Cybern Syst 2019; 49(10):1933—45.

[42]	Haro EH, Avalos O, Gálvez J, Camarena O . An integrated process planning and scheduling problem solved from an adaptive multi—objective perspective. J Manuf Syst 2024; 75:1-23.

[43]	Zhang W, Zheng Y, Ahmad R . The integrated process planning and scheduling of flexible job—shop—type remanufacturing systems using improved artificial bee colony algorithm. J Intell Manuf 2023; 34(7):2963-88.

[44]	Hosseinzadeh MR, Heydari M, Mahdavi Mazdeh M . Mathematical modeling and two metaheuristic algorithms for integrated process planning and group scheduling with sequence—dependent setup time. Oper Res Int J 2022; 22(5):5055-105.

[45]	Wen X, Qian Y, Lian X, Zhang Y, Wang H, Li H . Improved genetic algorithm based on multi—layer encoding approach for integrated process planning and scheduling problem. Robot Comput—Integr Manuf 2023; 84:102593.

[46]	Hu Y, Dong H, Liu J, Zhuang C, Zhang F . A learning—guided hybrid genetic algorithm and multi—neighborhood search for the integrated process planning and scheduling problem with reconfigurable manufacturing cells. Robot Comput—Integr Manuf 2025; 93:102919.

[47]	Wang J, Zheng H, Zhao S, Zhang Q . Energy—aware remanufacturing process planning and scheduling problem using reinforcement learning—based particle swarm optimization algorithm. J Clean Prod 2024; 476:143771.

[48]	Müller—Zhang Z, Kuhn T, Antonino PO . Towards live decision—making for service—based production: integrated process planning and scheduling with digital twins and deep—q—learning. Comput Ind 2023; 149:103933.

[49]	Liu Y, Ping Y, Zhang L, Wang L, Xu X . Scheduling of decentralized robot services in cloud manufacturing with deep reinforcement learning. Robot Comput—Integr Manuf 2023; 80:102454.

[50]	Liu Q, Li X, Gao L, Li Y . A modified genetic algorithm with new encoding and decoding methods for integrated process planning and scheduling problem. IEEE Trans Cybern 2021; 51(9):4429—38.

[51]	Han J, Liu Y, Luo L, Mao M . Integrated production planning and scheduling under uncertainty: a fuzzy bi—level decision—making approach. Knowl—Based Syst 2020; 201—202:106056.

[52]	Jin H, Li H, Bi J, Li M . Joint optimization of spare parts inventory and maintenance for wind turbine systems. Expert Syst Appl 2025; 292:128588.

[53]	Zhu J, Huang S, Wang P, Xu Z, Liu J, Wang B, et al. A review on large language models for industrial embodied intelligence. Adv Eng Inform 2026; 73:104602.

[54]	Huang S, Wang P, Zhu J, Guo WG, Gu X, Zhao QQ, et al. Human—like perception for embodied intelligent robots: a review. J Manuf Syst 2026; 86:887-912.

[55]	Chen J, Huang S, Wang X, Wang P, Zhu J, Xu Z, et al. Perception—decision—execution coordination mechanism driven dynamic autonomous collaboration method for human—like collaborative robot based on multimodal large language model. Robot Comput—Integr Manuf 2026; 98:103167.