PDF(2323 KB)
一种局部二次嵌入学习算法及其在软测量中的应用
Yaoyao Bao, Yuanming Zhu, Feng Qian
工程（英文） ›› 2022, Vol. 18 ›› Issue (11) : 186-196.
PDF(2323 KB)
PDF(2323 KB)
一种局部二次嵌入学习算法及其在软测量中的应用
A Local Quadratic Embedding Learning Algorithm and Applications for Soft Sensing
鉴于元学习在众多领域取得的巨大成就，本文针对数据回归问题提出了融合度量学习和神经网络（NN）的局部二次嵌入学习（LQEL）算法。首先，通过优化输入输出空间里样本间度量的全局一致性来改进马氏度量（Mahalanobis metric）学习算法；同时，通过引入松弛约束进一步证明了改进的度量学习问题等价于一个凸规划问题。然后，基于局部二次插值假设原理，引入了两个轻量级的神经网络，其一用于学习局部二次模型中的系数矩阵，另一个则用于对从不同局部近邻获得的预测结果进行权重分配。最后，将两个子模型嵌入统一的回归框架中，并通过随机梯度下降（SGD）算法学习模型参数。所提出的算法优势在于可充分利用目标标签中隐含的信息找到更可靠的参考样本。并且，使用LQEL算法对变量进行差分建模，避免了因传感器漂移或不可测量变量导致的模型退化问题。多个基准数据集和两个实际工业应用数据集的计算结果表明，所提出的方法优于几种典型的回归方法。
Inspired by the tremendous achievements of meta-learning in various fields, this paper proposes the local quadratic embedding learning (LQEL) algorithm for regression problems based on metric learning and neural networks (NNs). First, Mahalanobis metric learning is improved by optimizing the global consistency of the metrics between instances in the input and output space. Then, we further prove that the improved metric learning problem is equivalent to a convex programming problem by relaxing the constraints. Based on the hypothesis of local quadratic interpolation, the algorithm introduces two lightweight NNs; one is used to learn the coefficient matrix in the local quadratic model, and the other is implemented for weight assignment for the prediction results obtained from different local neighbors. Finally, the two sub-models are embedded in a unified regression framework, and the parameters are learned by means of a stochastic gradient descent (SGD) algorithm. The proposed algorithm can make full use of the information implied in target labels to find more reliable reference instances. Moreover, it prevents the model degradation caused by sensor drift and unmeasurable variables by modeling variable differences with the LQEL algorithm. Simulation results on multiple benchmark datasets and two practical industrial applications show that the proposed method outperforms several popular regression methods.
Local quadratic embedding / Metric learning / Regression machine / Soft sensor
| [1] |
Bao YY, Zhu YM, Du WL, Zhong WM, Qian F. A distributed PCA-TSS based soft sensor for raw meal fineness in VRM system. Control Eng Pract 2019;90:38–49.
|
| [2] |
Zhong WM, Jiang C, Peng X, Li Z, Qian F. Online quality prediction of industrial terephthalic acid hydropurification process using modified regularized slowfeature analysis. Ind Eng Chem Res 2018;57(29):9604–14.
|
| [3] |
Lu B, Chiang L. Semi-supervised online soft sensor maintenance experiences in the chemical industry. J Process Contr 2018;67:23–34.
|
| [4] |
Li YG, Gui WH, Yang CH, Xie YF. Soft sensor and expert control for blending and digestion process in alumina metallurgical industry. J Process Contr 2013;23 (7):1012–21.
|
| [5] |
Song YC, Zhou H, Wang PF, Yang MJ. Prediction of clathrate hydrate phase equilibria using gradient boosted regression trees and deep neural networks. J Chem Thermodyn 2019;135:86–96.
|
| [6] |
Touzani S, Granderson J, Fernandes S. Gradient boosting machine for modeling the energy consumption of commercial buildings. Energ Build 2018;158:1533–43.
|
| [7] |
Fernández-Delgado M, Sirsat MS, Cernadas E, Alawadi S, Barro S, FebreroBande M. An extensive experimental survey of regression methods. Neural Netw 2019;111:11–34.
|
| [8] |
Yu J. Multiway Gaussian mixture model based adaptive kernel partial least squares regression method for soft sensor estimation and reliable quality prediction of nonlinear multiphase batch processes. Ind Eng Chem Res 2012;51 (40):13227–37.
|
| [9] |
Yuan XF, Ge ZQ, Song ZH. Locally weighted kernel principal component regression model for soft sensing of nonlinear time-variant processes. Ind Eng Chem Res 2014;53(35):13736–49.
|
| [10] |
Gou JP, Ma HX, Ou WH, Zeng SN, Rao YB, Yang HB. A generalized mean distance-based k-nearest neighbor classifier. Expert Syst Appl 2019;115: 356–72.
|
| [11] |
Juez-Gil M, Erdakov IN, Bustillo A, Pimenov DY. A regression-tree multilayerperceptron hybrid strategy for the prediction of ore crushing-plate lifetimes. J Adv Res 2019;18:173–84.
|
| [12] |
Suarez A, Lutsko JF. Globally optimal fuzzy decision trees for classification and regression. IEEE Trans Pattern Anal Mach Intell 1999;21(12):1297–311.
|
| [13] |
Huang GB, Zhou HM, Ding XJ, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 2012;42(2):513–29.
|
| [14] |
Vilela LFS, Leme RC, Pinheiro CAM, Carpinteiro OAS. Forecasting financial series using clustering methods and support vector regression. Artif Intell Rev 2019;52(2):743–73.
|
| [15] |
Paul A, Mukherjee DP. Reinforced quasi-random forest. Pattern Recognit 2019;94:13–24.
|
| [16] |
Rodriguez-Galiano VF, Ghimire B, Rogan J, Chica-Olmo M, Rigol-Sanchez JP. An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS J Photogramm Remote Sens 2012;67:93–104.
|
| [17] |
Freund Y. Boosting a weak learning algorithm by majority. Inf Comput 1995;121(2):256–85.
|
| [18] |
Chen TQ, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2016 Aug 13–17; San Francisco, CA, USA; 2016. p. 785–94.
|
| [19] |
Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002;35(5– 6):352–9.
|
| [20] |
Zhou ZH, Wu JX, Tang W. Ensembling neural networks: many could be better than all. Artif Intell 2002;137(1–2):239–63.
|
| [21] |
Liu WB, Wang ZD, Liu XH, Zeng NY, Liu YR, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing 2017;234:11–26.
|
| [22] |
Martos G,Muñoz A, González J. On the generalization of the Mahalanobis distance. In: Ruiz-Shulcloper J, Sanniti di Baja G, editors. Progress in pattern recognition, image analysis, computer vision, and applications. Berlin: Springer; 2013. p. 125–32.
|
| [23] |
Atkeson CG, Moore AW, Schaal S. Locally weighted learning. Artif Intell Rev 1997;11:11–73.
|
| [24] |
Zhang JJ. Identification of moving loads using a local linear embedding algorithm. J Vib Control 2019;25(11):1780–90.
|
| [25] |
Loia V, Tomasiello S, Vaccaro A, Gao JW. Using local learning with fuzzy transform: application to short term forecasting problems. Fuzzy Optim Decis Making 2020;19(1):13–32.
|
| [26] |
Weinberger KQ, Saul LK. Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 2009;10:207–44.
|
| [27] |
Nguyen B, Morell C, De Baets B. Large-scale distance metric learning for knearest neighbors regression. Neurocomputing 2016;214:805–14.
|
| [28] |
Xing EP, Ng AY, Jordan MI, Russell S. Distance metric learning, with application to clustering with side-information. In: Becker S, Thrun S, Obermayer K, editors. Advances in neural information processing systems 15: proceedings of the 2002 conference. Cambridge: A Bradford Book; 2003. p. 521–8.
|
| [29] |
Duan YQ, Lu JW, Zheng WZ, Zhou J. Deep adversarial metric learning. IEEE Trans Image Process 2020;29:2037–51.
|
| [30] |
Song HO, Xiang Y, Jegelka S, Savarese S. Deep metric learning via lifted structured feature embedding. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26–Jul 1; Las Vegas, NV, USA; 2016. p. 4004–12.
|
| [31] |
Cui Y, Zhou F, Lin YQ, Belongie S. Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26–Jul 1; Las Vegas, NV, USA; 2016. p. 1153–62.
|
| [32] |
ALzubi JA, Bharathikannan B, Tanwar S, Manikandan R, Khanna A, Thaventhiran C. Boosted neural network ensemble classification for lung cancer disease diagnosis. Appl Soft Comput 2019;80:579–91.
|
| [33] |
Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. 2015. arXiv:1502.03167.
|
| [34] |
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improving neural networks by preventing co-adaptation of feature detectors. 2012. arXiv:1207.0580.
|
| [35] |
Kay S. Can detectability be improved by adding noise? IEEE Signal Process Lett 2000;7(1):8–10.
|
| [36] |
Boyd V, Vandenberghe L, Faybusovich L. Convex optimization. IEEE Trans Automat Contr 2006;51(11):1859.
|
| [37] |
Cortez P, Morais A. A data mining approach to predict forest fires using meteorological data. In: Neves JM, Santos MF, Machado JM, editors. New trends in artificial intelligence: proceedings of the 13th Portuguese Conference on Artificial Intelligence; 2007 Dec 3–7; Guimarães, Portugal; 2007. p. 512–23. French.
|
| [38] |
Cortez P, Cerdeira A, Almeida F, Matos T, Reis J. Modeling wine preferences by data mining from physicochemical properties. Decis Support Syst 2009;47 (4):547–53.
|
| [39] |
De Vito S, Massera E, Piga M, Martinotto L, Di Francia G. On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B Chem 2008;129(2):750–7.
|
| [40] |
Ke GL, Meng Q, Finley T, Wang TF, Chen W, Ma WD, et al. LightGBM: a highly efficient gradient boosting decision tree. In: Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Proceedings of the 31st Annual Conference on Neural Information Processing Systems; 2017 Dec 4–9; Long Beach, CA, USA; 2017.
|
| [41] |
Guo HF, Tang RM, Ye YM, Li ZG, He XQ. DeepFM: a factorization-machine based neural network for CTR prediction. In: Sierra C, editor. Proceedings of the 26th International Joint Conference on Artificial Intelligence; 2017 Aug 19–25; Melbourne, VIC, Australia; 2017. p. 1725–31.
|
| [42] |
Zhang L, Shen WC, Huang JH, Li SJ, Pan G. Field-aware neural factorization machine for click-through rate prediction. IEEE Access 2019;7:75032–40.
|
| [43] |
Huang JY, Zhang X, Fang BX. CoStock: a DeepFM model for stock market prediction with attentional embeddings. In: Proceedings of 2019 IEEE International Conference on Big Data; 2019 Dec 9–12; Los Angeles, CA, USA. New York City: IEEE; 2019. p. 5522–31.
|
/
| 〈 |
|
〉 |
