2023, Volume 22, Issue 3
Engineering >> 2023, Volume 22, Issue 3 doi: 10.1016/j.eng.2022.06.019
A Causal Model-Inspired Automatic Feature-Selection Method for Developing Data-Driven Soft Sensors in Complex Industrial Processes
a School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
b Institute of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai 200240, China
Next Previous
Abstract
The soft sensing of key performance indicators (KPIs) plays an essential role in the decision-making of complex industrial processes. Many researchers have developed data-driven soft sensors using cutting-edge machine learning (ML) or deep learning (DL) models. Moreover, feature selection is a crucial issue because a raw industrial dataset is usually high-dimensional, and not all features are conducive to the development of soft sensors. A perfect feature-selection method should not rely on hyperparameters and subsequent ML or DL models. Rather, it should be able to automatically select a subset of features for soft sensor modeling, in which each feature has a unique causal effect on industrial KPIs. Therefore, this study proposes a causal model-inspired automatic feature-selection method for the soft sensing of industrial KPIs. First, inspired by the post-nonlinear causal model, we integrate it with information theory to quantify the causal effect between each feature and the KPIs in the raw industrial dataset. After that, a novel feature-selection method is proposed to automatically select the feature with a non-zero causal effect to construct the subset of features. Finally, the constructed subset is used to develop soft sensors for the KPIs by means of an AdaBoost ensemble strategy. Experiments on two practical industrial applications confirm the effectiveness of the proposed method. In the future, this method can also be applied to other industrial processes to help develop more advanced data-driven soft sensors.
Keywords
Big data analytics ; Machine intelligence ; Quality prediction ; Soft sensors ; Intelligent manufacturing
Figures
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
References
[ 1 ] Gao L, Shen W, Li X. New trends in intelligent manufacturing. Engineering 2019;5(4):619–20. link1
[ 2 ] Wang J, Zheng P, Lv Y, Bao J, Zhang J. Fog-IBDIS: industrial big data integration and sharing with fog computing for manufacturing systems. Engineering 2019;5(4):662–70. link1
[ 3 ] Wang J, Ma Y, Zhang L, Gao RX, Wu D. Deep learning for smart manufacturing: methods and applications. J Manuf Syst 2018;48:144–56. link1
[ 4 ] Yuan X, Gu Y, Wang Y, Yang C, Gui W. A deep supervised learning framework for data-driven soft sensor modeling of industrial processes. IEEE Trans Neural Netw Learn Syst 2020;31(11):4737–46. link1
[ 5 ] Liu C, Wang K, Wang Y, Yuan X. Learning deep multimanifold structure feature representation for quality prediction with an industrial application. IEEE Trans Ind Inform 2022;18(9):5849–58. link1
[ 6 ] Ren L, Meng Z, Wang X, Zhang L, Yang LT. A data-driven approach of product quality prediction for complex production systems. IEEE Trans Ind Inform 2021;17(9):6457–65. link1
[ 7 ] Geng Z, Dong J, Chen J, Han Y. A new self-organizing extreme learning machine soft sensor model and its applications in complicated chemical processes. Eng Appl Artif Intell 2017;62:38–50. link1
[ 8 ] Shi J, Zhou S. Quality control and improvement for multistage systems: a survey. IIE Trans 2009;41(9):744–53. link1
[ 9 ] Schrangl P, Tkachenko P, del Re L. Iterative model identification of nonlinear systems of unknown structure: systematic data-based modeling utilizing design of experiments. IEEE Control Syst Mag 2020;40(3):26–48. link1
[10] Mao J, Chen D, Zhang L. Mechanical assembly quality prediction method based on state space model. Int J Adv Manuf Technol 2016;86:107–16. link1
[11] Zhou X, Zhang Y, Mao T, Zhou H. Monitoring and dynamic control of quality stability for injection molding process. J Mater Process Technol 2017;249:358–66. link1
[12] Sun Q, Ge Z. Deep learning for industrial KPI prediction: when ensemble learning meets semi-supervised data. IEEE Trans Ind Inform 2021;17 (1):260–9. link1
[13] Jiang Q, Yan X, Yi H, Gao F. Data-driven batch-end quality modeling and monitoring based on optimized sparse partial least squares. IEEE Trans Ind Electron 2020;67(5):4098–107. link1
[14] Zhou P, Guo D, Wang H, Chai T. Data-driven robust M-LS-SVR-based NARX modeling for estimation and control of molten iron quality indices in blast furnace ironmaking. IEEE Trans Neural Netw Learn Syst 2018;29(9):4007–21. link1
[15] Ren L, Meng Z, Wang X, Lu R, Yang LT. A wide–deep-sequence model-based quality prediction method in industrial process analysis. IEEE Trans Neural Netw Learn Syst 2020;31(9):3721–31. link1
[16] Yuan X, Jia Z, Li L, Wang K, Ye L, Wang Y, et al. A SIA-LSTM based virtual metrology for quality variables in irregular sampled time sequence of industrial processes. Chem Eng Sci 2022;249:117299. link1
[17] Ou C, Zhu H, Shardt YAW, Ye L, Yuan X, Wang Y, et al. Quality-driven regularization for deep learning networks and its application to industrial soft sensors. IEEE Trans Neural Netw Learn Syst. In press.
[18] Yuan X, Ou C, Wang Y, Yang C, Gui W. A layer-wise data augmentation strategy for deep learning networks and its soft sensor application in an industrial hydrocracking process. IEEE Trans Neural Netw Learn Syst 2021;32 (8):3296–305. link1
[19] Wang X, Hu T, Tang L. A multiobjective evolutionary nonlinear ensemble learning with evolutionary feature selection for silicon prediction in blast furnace. IEEE Trans Neural Netw Learn Syst 2022;33(5):2080–93. link1
[20] Chai Z, Zhao C, Huang B, Chen H. A deep probabilistic transfer learning framework for soft sensor modeling with missing data. IEEE Trans Neural Netw Learn Syst 2022;33(12):7598–609.
[21] Peng H, Fan Y. Feature selection by optimizing a lower bound of conditional mutual information. Inf Sci 2017;418–419:652–67. link1
[22] Wang J, Xu C, Zhang J, Zhong R. Big data analytics for intelligent manufacturing systems: a review. J Manuf Syst 2022;62:738–52. link1
[23] Lee DH, Yang JK, Lee CH, Kim KJ. A data-driven approach to selection of critical process steps in the semiconductor manufacturing process considering missing and imbalanced data. J Manuf Syst 2019;52:146–56. link1
[24] Sun S, Hu X, Liu Y. An imbalanced data learning method for tool breakage detection based on generative adversarial networks. J Intell Manuf 2021;2021:1–15. link1
[25] Perši N, Dušak V. Conceptual modelling of continuous discrete production systems. In: Proceedings of the 6th EUROSIM Conference on Modelling and Simulation; 2007 Sep 9–13; Ljubljana, Slovenia. EUROSIM; 2007. p. 1–7.
[26] Xu HW, Qin W, Lv YL, Zhang J. Data-driven adaptive virtual metrology for yield prediction in multi-batch wafers. IEEE Trans Ind Inform 2022;18(12):9008–16.
[27] Diaz CJL, Ocampo-Martinez C. Energy efficiency in discrete-manufacturing systems: insights, trends, and control strategies. J Manuf Syst 2019;52:131–45. link1
[28] Thiede S, Turetskyy A, Kwade A, Kara S, Herrmann C. Data mining in battery production chains towards multi-criterial quality prediction. CIRP Ann 2019;68(1):463–6. link1
[29] Finkeldey F, Volke J, Zarges JC, Heim HP, Wiederkehr P. Learning quality characteristics for plastic injection molding processes using a combination of simulated and measured data. J Manuf Process 2020;60:134–43. link1
[30] Keskin Z, Aste T. Information-theoretic measures for nonlinear causality detection: application to social media sentiment and cryptocurrency prices. R Soc Open Sci 2020;7(9):200863. link1
[31] Spirtes P, Zhang K. Causal discovery and inference: concepts and recent methodological advances. Appl Inform 2016;3:3. link1
[32] Janzing D, Mooij J, Zhang K, Lemeire J, Zscheischler J, Daniušis P, et al. Information-geometric approach to inferring causal directions. Artif Intell 2012;182–183:1–31. link1
[33] Xu L. Machine learning and causal analyses for modeling financial and economic data. Appl Inform 2018;5:11. link1
[34] Nowack P, Runge J, Eyring V, Haigh JD. Causal networks for climate model evaluation and constrained projections. Nat Commun 2020;11:1415. link1
[35] Sun Y, Qin W, Zhuang Z, Xu H. An adaptive fault detection and root-cause analysis scheme for complex industrial processes using moving window KPCA and information geometric causal inference. J Intell Manuf 2021;32 (7):2007–21. link1
[36] Sun Y, Qin W, Zhuang Z. Nonparametric-copula-entropy and network deconvolution method for causal discovery in complex manufacturing systems. J Intell Manuf 2022;33(6):1699–713. link1
[37] Sun Y, Qin W, Zhuang Z. Quality consistency analysis for complex assembly process based on Bayesian networks. Procedia Manuf 2020;51:577–83. link1
[38] Xu H, Zhang J, Lv Y, Zheng P. Hybrid feature selection for wafer acceptance test parameters in semiconductor manufacturing. IEEE Access 2020;8:17320–30. link1
[39] Qin W, Zhuang Z, Guo L, Sun Y. A hybrid multi-class imbalanced learning method for predicting the quality level of diesel engines. J Manuf Syst 2022;62:846–56. link1
[40] Cai J, Luo J, Wang S, Yang S. Feature selection in machine learning: a new perspective. Neurocomputing 2018;300:70–9. link1
[41] Han M, Ren W. Global mutual information-based feature selection approach using single-objective and multi-objective optimization. Neurocomputing 2015;168:47–54. link1
[42] Han Y, Yu L. A variance reduction framework for stable feature selection. Stat Anal Data Min: ASA Data Sci J 2012;5(5):428–45. link1
[43] Sun YN, Zhuang ZL, Xu HW, Qin W, Feng MJ. Data-driven modeling and analysis based on complex network for multimode recognition of industrial processes. J Manuf Syst 2022;62:915–24. link1
[44] Reshef DN, Reshef YA, Finucane HK, Grossman SR, McVean G, Turnbaugh PJ, et al. Detecting novel associations in large data sets. Science 2011;334 (6062):1518–24. link1
[45] Mokhtia M, Eftekhari M, Saberi-Movahed F. Feature selection based on regularization of sparsity based regression models by hesitant fuzzy correlation. Appl Soft Comput 2020;91:106255. link1
[46] Cai RC, Chen W, Zhang K, Hao ZF. A survey on non-temporal series observational data based causal discovery. Chin J Comput 2017;40 (6):1470–90. Chinese. link1
[47] Glymour C, Zhang K, Spirtes P. Review of causal discovery methods based on graphical models. Front Genet 2019;10:524. link1
[48] You D, Li R, Liang S, Sun M, Ou X, Yuan F, et al. Online causal feature selection for streaming features. IEEE Trans Neural Netw Learn Syst. In press.
[49] Shimizu S, Hoyer PO, Hyvärinen A, Kerminen A. A linear non-Gaussian acyclic model for causal discovery. J Mach Learn Res 2006;7:2003–30. link1
[50] Janzing D, Peters J, Mooij J, Schölkopf B. Identifying confounders using additive noise models. 2012. arXiv:1205.2640.
[51] Zhang K, Hyvärinen A. Nonlinear functional causal models for distinguishing cause from effect. In: Wiedermann W, von Eye A, editors. Statistics and causality: methods for applied empirical research. Wiley; 2016. p. 185–201.
[52] Drucker H. Improving regressors using boosting techniques. In: Proceedings of the 14th International Conference on Machine Learning (ICML); 1997 Jul 8–12; Nashville, TN, USA. San Francisco: Morgan Kaufmann Publishers Inc.; 1997. p. 107–15.
[53] Sun YN, Chen Y, Wang WY, Xu HW, Qin W. Modelling and prediction of injection molding process using copula entropy and multi-output SVR. In: Proceedings of 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE); 2021 Aug 23–27; Lyon, France. IEEE; 2021. p. 1677–82.
京公网安备 11010502051620号
