PDF(4193 KB)
因果模型启发的复杂工业过程软传感器自动化特征选择方法
Yan-Ning Sun, Wei Qin, Jin-Hua Hu, Hong-Wei Xu, Poly Z.H. Sun
工程（英文） ›› 2023, Vol. 22 ›› Issue (3) : 82-93.
PDF(4193 KB)
PDF(4193 KB)
因果模型启发的复杂工业过程软传感器自动化特征选择方法
A Causal Model-Inspired Automatic Feature-Selection Method for Developing Data-Driven Soft Sensors in Complex Industrial Processes
关键性能指标（KPI）的软测量在复杂工业过程决策中起着至关重要的作用。许多研究人员已经使用先进的机器学习（ML）或深度学习（DL）模型开发出了数据驱动的软传感器。其中，特征选择是一个关键的问题，因为一个原始的工业数据集通常是高维的，并不是所有的特征都有利于软传感器的建立。一种完善的特征选择方法不应该依赖于超参数和后续的ML或DL模型。换言之，这种特征选择方法应该能够自动地选择一个特征子集进行软传感器建模，选择的每个特征对工业KPI 都有独特的因果影响。因此，本研究提出了一种受因果模型启发的自动特征选择方法，用于工业KPI 的软测量。首先，受后非线性因果模型的启发，本研究将数据驱动的软传感器与信息论相结合，以量化原始工业数据集中每个特征和KPI之间的因果效应。然后，提出了一种新的特征选择方法，即自动选择具有非零因果效应的特征来构造特征子集。最后，利用所构造的子集，通过AdaBoost 集成策略开发KPI 的软传感器。两个实际工业应用中的实验证实了该方法的有效性。在未来，该方法也可以应用于其他工业过程，以帮助开发更先进的数据驱动的软传感器。
The soft sensing of key performance indicators (KPIs) plays an essential role in the decision-making of complex industrial processes. Many researchers have developed data-driven soft sensors using cutting-edge machine learning (ML) or deep learning (DL) models. Moreover, feature selection is a crucial issue because a raw industrial dataset is usually high-dimensional, and not all features are conducive to the development of soft sensors. A perfect feature-selection method should not rely on hyperparameters and subsequent ML or DL models. Rather, it should be able to automatically select a subset of features for soft sensor modeling, in which each feature has a unique causal effect on industrial KPIs. Therefore, this study proposes a causal model-inspired automatic feature-selection method for the soft sensing of industrial KPIs. First, inspired by the post-nonlinear causal model, we integrate it with information theory to quantify the causal effect between each feature and the KPIs in the raw industrial dataset. After that, a novel feature-selection method is proposed to automatically select the feature with a non-zero causal effect to construct the subset of features. Finally, the constructed subset is used to develop soft sensors for the KPIs by means of an AdaBoost ensemble strategy. Experiments on two practical industrial applications confirm the effectiveness of the proposed method. In the future, this method can also be applied to other industrial processes to help develop more advanced data-driven soft sensors.
大数据分析 / 机器智能 / 质量预测 / 软传感器 / 智能制造
Big data analytics / Machine intelligence / Quality prediction / Soft sensors / Intelligent manufacturing
| [1] |
Gao L, Shen W, Li X. New trends in intelligent manufacturing. Engineering 2019;5(4):619–20.
|
| [2] |
Wang J, Zheng P, Lv Y, Bao J, Zhang J. Fog-IBDIS: industrial big data integration and sharing with fog computing for manufacturing systems. Engineering 2019;5(4):662–70.
|
| [3] |
Wang J, Ma Y, Zhang L, Gao RX, Wu D. Deep learning for smart manufacturing: methods and applications. J Manuf Syst 2018;48:144–56.
|
| [4] |
Yuan X, Gu Y, Wang Y, Yang C, Gui W. A deep supervised learning framework for data-driven soft sensor modeling of industrial processes. IEEE Trans Neural Netw Learn Syst 2020;31(11):4737–46.
|
| [5] |
Liu C, Wang K, Wang Y, Yuan X. Learning deep multimanifold structure feature representation for quality prediction with an industrial application. IEEE Trans Ind Inform 2022;18(9):5849–58.
|
| [6] |
Ren L, Meng Z, Wang X, Zhang L, Yang LT. A data-driven approach of product quality prediction for complex production systems. IEEE Trans Ind Inform 2021;17(9):6457–65.
|
| [7] |
Geng Z, Dong J, Chen J, Han Y. A new self-organizing extreme learning machine soft sensor model and its applications in complicated chemical processes. Eng Appl Artif Intell 2017;62:38–50.
|
| [8] |
Shi J, Zhou S. Quality control and improvement for multistage systems: a survey. IIE Trans 2009;41(9):744–53.
|
| [9] |
Schrangl P, Tkachenko P, del Re L. Iterative model identification of nonlinear systems of unknown structure: systematic data-based modeling utilizing design of experiments. IEEE Control Syst Mag 2020;40(3):26–48.
|
| [10] |
Mao J, Chen D, Zhang L. Mechanical assembly quality prediction method based on state space model. Int J Adv Manuf Technol 2016;86:107–16.
|
| [11] |
Zhou X, Zhang Y, Mao T, Zhou H. Monitoring and dynamic control of quality stability for injection molding process. J Mater Process Technol 2017;249:358–66.
|
| [12] |
Sun Q, Ge Z. Deep learning for industrial KPI prediction: when ensemble learning meets semi-supervised data. IEEE Trans Ind Inform 2021;17 (1):260–9.
|
| [13] |
Jiang Q, Yan X, Yi H, Gao F. Data-driven batch-end quality modeling and monitoring based on optimized sparse partial least squares. IEEE Trans Ind Electron 2020;67(5):4098–107.
|
| [14] |
Zhou P, Guo D, Wang H, Chai T. Data-driven robust M-LS-SVR-based NARX modeling for estimation and control of molten iron quality indices in blast furnace ironmaking. IEEE Trans Neural Netw Learn Syst 2018;29(9):4007–21.
|
| [15] |
Ren L, Meng Z, Wang X, Lu R, Yang LT. A wide–deep-sequence model-based quality prediction method in industrial process analysis. IEEE Trans Neural Netw Learn Syst 2020;31(9):3721–31.
|
| [16] |
Yuan X, Jia Z, Li L, Wang K, Ye L, Wang Y, et al. A SIA-LSTM based virtual metrology for quality variables in irregular sampled time sequence of industrial processes. Chem Eng Sci 2022;249:117299.
|
| [17] |
Ou C, Zhu H, Shardt YAW, Ye L, Yuan X, Wang Y, et al. Quality-driven regularization for deep learning networks and its application to industrial soft sensors. IEEE Trans Neural Netw Learn Syst. In press.
|
| [18] |
Yuan X, Ou C, Wang Y, Yang C, Gui W. A layer-wise data augmentation strategy for deep learning networks and its soft sensor application in an industrial hydrocracking process. IEEE Trans Neural Netw Learn Syst 2021;32 (8):3296–305.
|
| [19] |
Wang X, Hu T, Tang L. A multiobjective evolutionary nonlinear ensemble learning with evolutionary feature selection for silicon prediction in blast furnace. IEEE Trans Neural Netw Learn Syst 2022;33(5):2080–93.
|
| [20] |
Chai Z, Zhao C, Huang B, Chen H. A deep probabilistic transfer learning framework for soft sensor modeling with missing data. IEEE Trans Neural Netw Learn Syst 2022;33(12):7598–609.
|
| [21] |
Peng H, Fan Y. Feature selection by optimizing a lower bound of conditional mutual information. Inf Sci 2017;418–419:652–67.
|
| [22] |
Wang J, Xu C, Zhang J, Zhong R. Big data analytics for intelligent manufacturing systems: a review. J Manuf Syst 2022;62:738–52.
|
| [23] |
Lee DH, Yang JK, Lee CH, Kim KJ. A data-driven approach to selection of critical process steps in the semiconductor manufacturing process considering missing and imbalanced data. J Manuf Syst 2019;52:146–56.
|
| [24] |
Sun S, Hu X, Liu Y. An imbalanced data learning method for tool breakage detection based on generative adversarial networks. J Intell Manuf 2021;2021:1–15.
|
| [25] |
Perši N, Dušak V. Conceptual modelling of continuous discrete production systems. In: Proceedings of the 6th EUROSIM Conference on Modelling and Simulation; 2007 Sep 9–13; Ljubljana, Slovenia. EUROSIM; 2007. p. 1–7.
|
| [26] |
Xu HW, Qin W, Lv YL, Zhang J. Data-driven adaptive virtual metrology for yield prediction in multi-batch wafers. IEEE Trans Ind Inform 2022;18(12):9008–16.
|
| [27] |
Diaz CJL, Ocampo-Martinez C. Energy efficiency in discrete-manufacturing systems: insights, trends, and control strategies. J Manuf Syst 2019;52:131–45.
|
| [28] |
Thiede S, Turetskyy A, Kwade A, Kara S, Herrmann C. Data mining in battery production chains towards multi-criterial quality prediction. CIRP Ann 2019;68(1):463–6.
|
| [29] |
Finkeldey F, Volke J, Zarges JC, Heim HP, Wiederkehr P. Learning quality characteristics for plastic injection molding processes using a combination of simulated and measured data. J Manuf Process 2020;60:134–43.
|
| [30] |
Keskin Z, Aste T. Information-theoretic measures for nonlinear causality detection: application to social media sentiment and cryptocurrency prices. R Soc Open Sci 2020;7(9):200863.
|
| [31] |
Spirtes P, Zhang K. Causal discovery and inference: concepts and recent methodological advances. Appl Inform 2016;3:3.
|
| [32] |
Janzing D, Mooij J, Zhang K, Lemeire J, Zscheischler J, Daniušis P, et al. Information-geometric approach to inferring causal directions. Artif Intell 2012;182–183:1–31.
|
| [33] |
Xu L. Machine learning and causal analyses for modeling financial and economic data. Appl Inform 2018;5:11.
|
| [34] |
Nowack P, Runge J, Eyring V, Haigh JD. Causal networks for climate model evaluation and constrained projections. Nat Commun 2020;11:1415.
|
| [35] |
Sun Y, Qin W, Zhuang Z, Xu H. An adaptive fault detection and root-cause analysis scheme for complex industrial processes using moving window KPCA and information geometric causal inference. J Intell Manuf 2021;32 (7):2007–21.
|
| [36] |
Sun Y, Qin W, Zhuang Z. Nonparametric-copula-entropy and network deconvolution method for causal discovery in complex manufacturing systems. J Intell Manuf 2022;33(6):1699–713.
|
| [37] |
Sun Y, Qin W, Zhuang Z. Quality consistency analysis for complex assembly process based on Bayesian networks. Procedia Manuf 2020;51:577–83.
|
| [38] |
Xu H, Zhang J, Lv Y, Zheng P. Hybrid feature selection for wafer acceptance test parameters in semiconductor manufacturing. IEEE Access 2020;8:17320–30.
|
| [39] |
Qin W, Zhuang Z, Guo L, Sun Y. A hybrid multi-class imbalanced learning method for predicting the quality level of diesel engines. J Manuf Syst 2022;62:846–56.
|
| [40] |
Cai J, Luo J, Wang S, Yang S. Feature selection in machine learning: a new perspective. Neurocomputing 2018;300:70–9.
|
| [41] |
Han M, Ren W. Global mutual information-based feature selection approach using single-objective and multi-objective optimization. Neurocomputing 2015;168:47–54.
|
| [42] |
Han Y, Yu L. A variance reduction framework for stable feature selection. Stat Anal Data Min: ASA Data Sci J 2012;5(5):428–45.
|
| [43] |
Sun YN, Zhuang ZL, Xu HW, Qin W, Feng MJ. Data-driven modeling and analysis based on complex network for multimode recognition of industrial processes. J Manuf Syst 2022;62:915–24.
|
| [44] |
Reshef DN, Reshef YA, Finucane HK, Grossman SR, McVean G, Turnbaugh PJ, et al. Detecting novel associations in large data sets. Science 2011;334 (6062):1518–24.
|
| [45] |
Mokhtia M, Eftekhari M, Saberi-Movahed F. Feature selection based on regularization of sparsity based regression models by hesitant fuzzy correlation. Appl Soft Comput 2020;91:106255.
|
| [46] |
Cai RC, Chen W, Zhang K, Hao ZF. A survey on non-temporal series observational data based causal discovery. Chin J Comput 2017;40 (6):1470–90. Chinese.
|
| [47] |
Glymour C, Zhang K, Spirtes P. Review of causal discovery methods based on graphical models. Front Genet 2019;10:524.
|
| [48] |
You D, Li R, Liang S, Sun M, Ou X, Yuan F, et al. Online causal feature selection for streaming features. IEEE Trans Neural Netw Learn Syst. In press.
|
| [49] |
Shimizu S, Hoyer PO, Hyvärinen A, Kerminen A. A linear non-Gaussian acyclic model for causal discovery. J Mach Learn Res 2006;7:2003–30.
|
| [50] |
Janzing D, Peters J, Mooij J, Schölkopf B. Identifying confounders using additive noise models. 2012. arXiv:1205.2640.
|
| [51] |
Zhang K, Hyvärinen A. Nonlinear functional causal models for distinguishing cause from effect. In: Wiedermann W, von Eye A, editors. Statistics and causality: methods for applied empirical research. Wiley; 2016. p. 185–201.
|
| [52] |
Drucker H. Improving regressors using boosting techniques. In: Proceedings of the 14th International Conference on Machine Learning (ICML); 1997 Jul 8–12; Nashville, TN, USA. San Francisco: Morgan Kaufmann Publishers Inc.; 1997. p. 107–15.
|
| [53] |
Sun YN, Chen Y, Wang WY, Xu HW, Qin W. Modelling and prediction of injection molding process using copula entropy and multi-output SVR. In: Proceedings of 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE); 2021 Aug 23–27; Lyon, France. IEEE; 2021. p. 1677–82.
|
/
| 〈 |
|
〉 |
