Hedging Against Material Uncertainty via Chance-Constrained Recurrent Neural Networks: A Continuous Pharmaceutical Manufacturing Case Study

Qingbo Meng; I. David L. Bogle; Vassilis M. Charitopoulos

doi:10.1016/j.eng.2025.05.019

Engineering ›› 2025, Vol. 52 ›› Issue (9) :129 -141. DOI: 10.1016/j.eng.2025.05.019

Research

Article

Hedging Against Material Uncertainty via Chance-Constrained Recurrent Neural Networks: A Continuous Pharmaceutical Manufacturing Case Study

Author information +

History +

PDF (2866KB)

Abstract

In the pharmaceutical industry, model-based prediction is a crucial stage in process development that allows pharmaceutical companies to simulate different scenarios toward improving process efficiency, reducing costs, and enhancing product quality. Nevertheless, ensuring the quality of formulated pharmaceutical products through the management of raw material variations has always been a challenging task. In this work, data-driven chance-constrained recurrent neural networks (CCRNNs) are developed to address the issue arising from raw material uncertainty. Our goal is to explore how, by proactively incorporating uncertainty into the model training process, more accurate predictions and enhanced robustness can be realized. The proposed approach is tested on a fluid bed dryer (FBD) from a continuous pharmaceutical manufacturing pilot plant. The results demonstrate that CCRNN models offer more robust and accurate predictions for the critical quality attribute (CQA)—in this case, moisture content—when material variations occur, compared with conventional recurrent neural network-based models.

Keywords

Data-driven chance constraints / Recurrent neural networks / Managing material uncertainty / Continuous pharmaceutical manufacturing / Smart manufacturing

Cite this article

Download citation ▾

Qingbo Meng, I. David L. Bogle, Vassilis M. Charitopoulos. Hedging Against Material Uncertainty via Chance-Constrained Recurrent Neural Networks: A Continuous Pharmaceutical Manufacturing Case Study. Engineering, 2025, 52 (9) : 129-141 DOI:10.1016/j.eng.2025.05.019

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Ierapetritou M, Muzzio F, Reklaitis G. Perspectives on the continuous manufacturing of powder-based pharmaceutical processes. AIChE J 2016; 62 (6):1846-62.

[2]	Wang Z, Ierapetritou M. Applications of optimization in the pharmaceutical process development. In: Muzzio FJ, Oka S, editors. How to design and implement powder-to-tablet continuous manufacturing systems. London: Elsevier Ltd.; 2022. p. 271-99.

[3]	Bogle IDL. A perspective on smart process manufacturing research challenges for process systems engineers. Engineering 2017; 3(2):161-5.

[4]	Napoleone A, Pozzetti A, Macchi M, Andersen R. Time to be responsive in the process industry: a literature-based analysis of trends of change, solutions and challenges. Prod Plann Contr 2023; 34(6):572-86.

[5]	Litster J, Bogle IDL. Smart process manufacturing for formulated products. Engineering 2019; 5(6):1003-9.

[6]	Chen Y, Yang O, Sampat C, Bhalode P, Ramachandran R, Ierapetritou M. Digital twins in pharmaceutical and biopharmaceutical manufacturing: a literature review. Processes 2020; 8(9):1088.

[7]	Sharifian S, Sotudeh-Gharebagh R, Zarghami R, Tanguy P, Mostoufi N. Uncertainty in chemical process systems engineering: a critical review. Rev Chem Eng 2021; 37(6):687-714.

[8]	Department of Health and Human Services, Food and Drug Administration. Pharmaceutical CGMPS for the 21st century—a risk-based approach. Rockville: Food and Drug Administration; 2004.

[9]	Boukouvala F, Ierapetritou MG. Surrogate-based optimization of expensive flowsheet modeling for continuous pharmaceutical manufacturing. J Pharm Innov 2013; 8(2):131-45.

[10]	Boukouvala F, Muzzio FJ, Ierapetritou MG. Design space of pharmaceutical processes using data-driven-based methods. J Pharm Innov 2010; 5(3):119-37.

[11]	Bounitsis GL, Papageorgiou LG, Charitopoulos VM. Stable optimisation-based scenario generation via game theoretic approach. Comput Chem Eng 2024; 185:108646.

[12]	Gao J, Liu L, Dong Y, Zhang L, Zhuang Y, Du J. Stochastic programming-based mathematical model and solution strategy for chemical production scheduling with processing time uncertainty. Comput Chem Eng 2022; 168:108043.

[13]	Zeng Z, Cremaschi S. Multistage stochastic programming models for pharmaceutical clinical trial planning. Processes 2017; 5(4):71.

[14]	Nikzad E, Bashiri M, Oliveira F. Two-stage stochastic programming approach for the medical drug inventory routing problem under uncertainty. Comput Ind Eng 2019; 128:358-70.

[15]	Jang HA, Kim SY, Lim YY, Lim JL, Shin S. Robust optimization approaches for a natural pharmaceutical complex product of Atractylodes japonica Koidz and Schisandra chinensis. Appl Sci 2020; 10(19):7006.

[16]	Xie X, Schenkendorf R. Robust optimization of a pharmaceutical freeze-drying process under non-Gaussian parameter uncertainties. Chem Eng Sci 2019; 207:805-19.

[17]	Ning C, You F. Data-driven stochastic robust optimization: general computational framework and algorithm leveraging machine learning for optimization under uncertainty in the big data era. Comput Chem Eng 2018; 111:115-33.

[18]	Liu J, Su Q, Moreno M, Laird C, Nagy Z, Reklaitis G. Robust state estimation of feeding-blending systems in continuous pharmaceutical manufacturing. Chem Eng Res Des 2018; 134:140-53.

[19]	Häussling Löwgren B, Weigert J, Esche E, Repke JU. Uncertainty analysis for data-driven chance-constrained optimization. Sustainability 2020; 12(6):2450.

[20]	Calfa BA, Grossmann IE, Agarwal A, Bury SJ, Wassick JM. Data-driven individual and joint chance-constrained optimization via kernel smoothing. Comput Chem Eng 2015; 78:51-69.

[21]	Kumar Ananthu M, Kumar Chintamaneni P, Basha Shaik S, Thadipatri R, Mahammed N. Artificial neural networks in optimization of pharmaceutical formulations. Saudi J Med Pharm Sci 2021; 7(8):368-78.

[22]	Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996; 49 (11):1225-31.

[23]	Wesolowski M, Suchacz B. Artificial neural networks: theoretical background and pharmaceutical applications: a review. J AOAC Int 2012; 95(3):652-68.

[24]	Zhang Z, Zhou D, Zhang J, Xu Y, Lin G, Jin B, et al. Multilayer perceptronbased prediction of stroke mimics in prehospital triage. Sci Rep 2022; 12(1): 17994.

[25]	Khalid MH, Kazemi P, Perez-Gandarillas L, Michrafy A, Szlek J, Jachowicz R, et al. Computational intelligence models to predict porosity of tablets using minimum features. Drug Des Devel Ther 2017; 11:193-202.

[26]	Korteby Y, Mahdi Y, Azizou A, Daoud K, Regdon Jr G. Implementation of an artificial neural network as a PAT tool for the prediction of temperature distribution within a pharmaceutical fluidized bed granulator. Eur J Pharm Sci 2016; 88:219-32.

[27]	Han R, Yang Y, Li X, Ouyang D. Predicting oral disintegrating tablet formulations by neural network techniques. Asian J Pharm Sci 2018; 13 (4):336-42.

[28]	Galata DL, Farkas A, Könyves Z, Mészáros LA, Szabó E, Csontos I, et al. Fast, spectroscopy-based prediction of in vitro dissolution profile of extended release tablets using artificial neural networks. Pharmaceutics 2019; 11 (8):400.

[29]	Velásco-Mejía A, Vallejo-Becerra V, Chávez-Ramírez AU, Torres-González J, Reyes-Vidal Y, Castañeda-Zaldivar F. Modeling and optimization of a pharmaceutical crystallization process by using neural networks and genetic algorithms. Powder Technol 2016; 292:122-8.

[30]	Madzarevic M, Medarevic D, Vulovic A, Sustersic T, Djuris J, Filipovic N, et al. Optimization and prediction of ibuprofen release from 3D DLP printlets using artificial neural networks. Pharmaceutics 2019; 11(10):544.

[31]	Meng D, Liu Z. Machine learning aided pharmaceutical engineering: model development and validation for estimation of drug solubility in green solvent. J Mol Liq 2023; 392(Pt 1):123286.

[32]	Ragb HK, Gopal P. Multi-layered deep learning perceptron based model for predicting drug price changes. TechRxiv: 24417697.v1; 2023.

[33]	Metta N, Ramachandran R, Ierapetritou M. A novel adaptive sampling based methodology for feasible region identification of compute intensive models using artificial neural network. AIChE J 2021; 67(2):e17095.

[34]	Park YS, Lek S. Artificial neural networks:multilayer perceptron for ecological modeling. In: Jørgensen SE, editor. Developments in environmental modelling. Amsterdam: Elsevier B. 2016. p. 123-40.

[35]	Montesinos López OA, Montesinos López A, Crossa J. Fundamentals of artificial neural networks and deep learning. In: Multivariate statistical machine learning methods for genomic prediction. Cham: Springer; 2022. p. 379-425.

[36]	Sarker IH. Deep learning: a comprehensive overview on techniques, taxonomy, applications and research directions. SN Comput Sci 2021; 2:420.

[37]	Wong WC, Chee E, Li J, Wang X. Recurrent neural network-based model predictive control for continuous pharmaceutical manufacturing. Mathematics 2018; 6(11):242.

[38]	Baranilingesan I. Optimization algorithm-based Elman neural network controller for continuous stirred tank reactor process model. Curr Sci 2021; 120(8):1324-33.

[39]	Vega-Zambrano C, Diangelakis NA, Charitopoulos VM. Data-driven model predictive control for continuous pharmaceutical manufacturing. Int J Pharmaceutics 2025; 672:125322.

[40]	Vora LK, Gholap AD, Jetha K, Thakur RRS, Solanki HK, Chavda VP. Artificial intelligence in pharmaceutical technology and drug delivery design. Pharmaceutics 2023; 15(7):1916.

[41]	Wang S, Di J, Wang D, Dai X, Hua Y, Gao X, et al. State-of-the-art review of artificial neural networks to predict, characterize and optimize pharmaceutical formulation. Pharmaceutics 2022; 14(1):183.

[42]	Nagy B, Galata DL, Farkas A, Nagy ZK. Application of artificial neural networks in the process analytical technology of pharmaceutical manufacturing—a review. AAPS J 2022; 24:74.

[43]	Peng J, Li J, Shang X. A learning-based method for drug-target interaction prediction based on feature representation learning and deep neural network. BMC Bioinf 2020; 21(S13):394.

[44]	Bizmark N, Mostoufi N, Sotudeh-Gharebagh R, Ehsani H. Sequential modeling of fluidized bed paddy dryer. J Food Eng 2010; 101(3):303-8.

[45]	Izadifar M, Mowla D. Simulation of a cross-flow continuous fluidized bed dryer for paddy rice. J Food Eng 2003; 58(4):325-9.

[46]	Mohseni M, Kolomijtschuk A, Peters B, Demoulling M. Biomass drying in a vibrating fluidized bed dryer with a Lagrangian-Eulerian approach. Int J Therm Sci 2019; 138:219-34.

[47]	Handayani SU, Yohana E, Tauviqirrahman M, Rahman AG, Yulianto ME, Choi KH. Performance improvement of continuous horizontal fluidised bed dryer based on computational fluid dynamics. Results Eng 2023; 17:100972.

[48]	Burgschweiger J, Tsotsas E. Experimental investigation and modelling of continuous suidized bed drying under steady-state and dynamic conditions. Chem Eng Sci 2002; 57(24):5021-38.

[49]	Wang LG, Morrissey JP, Barrasso D, Slade D, Clifford S, Reynolds G, et al. Model driven design for twin screw granulation using mechanistic-based population balance model. Int J Pharm 2021; 607:120939.

[50]	Jiang SL, Papageorgiou LG, Bogle IDL, Charitopoulos VM. Investigating the trade-off between design and operational flexibility in continuous manufacturing of pharmaceutical tablets: a case study of the fluid bed dryer. Processes 2022; 10(3):454.

[51]	Charitopoulos VM, Aguirre AM, Papageorgiou LG, Dua V. Uncertainty aware integration of planning, scheduling and multi-parametric control. In: Eden MR, Ierapetritou MG, Towler GP, editors. Computer aided chemical engineering. Amsterdam: Elsevier B. 2018. p. 1171-6.

[52]	Jiang R, Guan Y. Data-driven chance constrained stochastic program. Math Program 2016; 158(1-2):291-327.

[53]	Li P, Arellano-Garcia H, Wozny G. Chance constrained programming approach to process optimization under uncertainty. Comput Chem Eng 2008; 32(1- 2):25-45.

[54]	Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. In: Dasgupta S, McAllester D, editors. Proceedings of the 30th International Conference on International Conference on Machine Learning; 2013 Jun 16-21; Atlanta, GA, USA. New York City: PMLR; 2012. p. 1310-8.

[55]	Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 1994; 5(2):157-66.

[56]	Zhang J, Zhu Y, Zhang X, Ye M, Yang J. Developing a long short-term memory (LSTM) based model for predicting water table depth in agricultural areas. J Hydrol 2018; 561:918-29.

[57]	Yang SB, Li Z. Recurrent neural network-based joint chance constrained stochastic model predictive control. IFAC-PapersOnLine 2022; 55(7):780-5.

[58]	Sagheer A, Kotb M. Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing 2019; 323:203-13.

[59]	Shah SRB, Chadha GS, Schwung A, Ding SX. A sequence-to-sequence approach for remaining useful lifetime estimation using attention-augmented bidirectional LSTM. Intell Syst Appl 2021;10-11:200049.

[60]	Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014; 15(1):1929-58.

[61]	Piotrowski AP, Napiorkowski JJ, Piotrowska AE. Impact of deep learning-based dropout on shallow neural networks applied to stream temperature modelling. Earth Sci Rev 2020; 201:103076.

[62]	Meng Q, Bogle IDL, Charitopoulos VM. Data-driven chance-constrained optimization for minimizing the influence of material uncertainty on product quality. In: Manenti F, Reklaitis GV, editors. Computer aided chemical engineering. Amsterdam: Elsevier B. 2024. p. 1579-84.

[63]	Charitopoulos VM, Papageorgiou LG, Dua V. Multi set-point explicit model predictive control for nonlinear process systems. Processes 2021; 9(7): 1156.

[64]	Charitopoulos VM, Dua V. Explicit model predictive control of hybrid systems and multiparametric mixed integer polynomial programming. AIChE J 2016; 62 (9):3441-60.