期刊首页 优先出版 当期阅读 过刊浏览 作者中心 关于期刊 English

《工程(英文)》 >> 2022年 第18卷 第11期 doi: 10.1016/j.eng.2022.04.025

一种局部二次嵌入学习算法及其在软测量中的应用

Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China

收稿日期: 2019-12-30 修回日期: 2021-01-25 录用日期: 2022-04-29 发布日期: 2022-09-28

下一篇 上一篇

摘要

鉴于元学习在众多领域取得的巨大成就,本文针对数据回归问题提出了融合度量学习和神经网络(NN)的局部二次嵌入学习(LQEL)算法。首先,通过优化输入输出空间里样本间度量的全局一致性来改进马氏度量(Mahalanobis metric)学习算法;同时,通过引入松弛约束进一步证明了改进的度量学习问题等价于一个凸规划问题。然后,基于局部二次插值假设原理,引入了两个轻量级的神经网络,其一用于学习局部二次模型中的系数矩阵,另一个则用于对从不同局部近邻获得的预测结果进行权重分配。最后,将两个子模型嵌入统一的回归框架中,并通过随机梯度下降(SGD)算法学习模型参数。所提出的算法优势在于可充分利用目标标签中隐含的信息找到更可靠的参考样本。并且,使用LQEL算法对变量进行差分建模,避免了因传感器漂移或不可测量变量导致的模型退化问题。多个基准数据集和两个实际工业应用数据集的计算结果表明,所提出的方法优于几种典型的回归方法。

图片

图1

图2

图3

图4

图5

图6

图7

图8

参考文献

[ 1 ] Bao YY, Zhu YM, Du WL, Zhong WM, Qian F. A distributed PCA-TSS based soft sensor for raw meal fineness in VRM system. Control Eng Pract 2019;90:38‒49. 链接1

[ 2 ] Zhong WM, Jiang C, Peng X, Li Z, Qian F. Online quality prediction of industrial terephthalic acid hydropurification process using modified regularized slow-feature analysis. Ind Eng Chem Res 2018;57(29):9604‒14. 链接1

[ 3 ] Lu B, Chiang L. Semi-supervised online soft sensor maintenance experiences in the chemical industry. J Process Contr 2018;67:23‒34. 链接1

[ 4 ] Li YG, Gui WH, Yang CH, Xie YF. Soft sensor and expert control for blending and digestion process in alumina metallurgical industry. J Process Contr 2013;23(7):1012‒21. 链接1

[ 5 ] Song YC, Zhou H, Wang PF, Yang MJ. Prediction of clathrate hydrate phase equilibria using gradient boosted regression trees and deep neural networks. J Chem Thermodyn 2019;135:86‒96. 链接1

[ 6 ] Touzani S, Granderson J, Fernandes S. Gradient boosting machine for modeling the energy consumption of commercial buildings. Energ Build 2018;158:1533‒43. 链接1

[ 7 ] Fernández-Delgado M, Sirsat MS, Cernadas E, Alawadi S, Barro S, Febrero-Bande M. An extensive experimental survey of regression methods. Neural Netw 2019;111:11‒34. 链接1

[ 8 ] Yu J. Multiway Gaussian mixture model based adaptive kernel partial least squares regression method for soft sensor estimation and reliable quality prediction of nonlinear multiphase batch processes. Ind Eng Chem Res 2012;51(40):13227‒37. 链接1

[ 9 ] Yuan XF, Ge ZQ, Song ZH. Locally weighted kernel principal component regression model for soft sensing of nonlinear time-variant processes. Ind Eng Chem Res 2014;53(35):13736‒49. 链接1

[10] Gou JP, Ma HX, Ou WH, Zeng SN, Rao YB, Yang HB. A generalized mean distance-based k-nearest neighbor classifier. Expert Syst Appl 2019;115:356‒72. 链接1

[11] Juez-Gil M, Erdakov IN, Bustillo A, Pimenov DY. A regression-tree multilayer-perceptron hybrid strategy for the prediction of ore crushing-plate lifetimes. J Adv Res 2019;18:173‒84. 链接1

[12] Suarez A, Lutsko JF. Globally optimal fuzzy decision trees for classification and regression. IEEE Trans Pattern Anal Mach Intell 1999;21(12):1297‒311. 链接1

[13] Huang GB, Zhou HM, Ding XJ, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 2012;42(2):513‒29. 链接1

[14] Vilela LFS, Leme RC, Pinheiro CAM, Carpinteiro OAS. Forecasting financial series using clustering methods and support vector regression. Artif Intell Rev 2019;52(2):743‒73.. 链接1

[15] Paul A, Mukherjee DP. Reinforced quasi-random forest. Pattern Recognit 2019;94:13‒24. 链接1

[16] Rodriguez-Galiano VF, Ghimire B, Rogan J, Chica-Olmo M, Rigol-Sanchez JP. An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS J Photogramm Remote Sens 2012;67:93‒104. 链接1

[17] Freund Y. Boosting a weak learning algorithm by majority. Inf Comput 1995;121(2):256‒85. 链接1

[18] Chen TQ, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2016 Aug 13‒17; San Francisco, CA, USA; 2016. p. 785‒94. 链接1

[19] Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002;35(5‒6):352‒9.

[20] Zhou ZH, Wu JX, Tang W. Ensembling neural networks: many could be better than all. Artif Intell 2002;137(1‒2):239‒63.

[21] Liu WB, Wang ZD, Liu XH, Zeng NY, Liu YR, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing 2017;234:11‒26. 链接1

[22] Martos G, Muñoz A, González J. On the generalization of the Mahalanobis distance. In: Ruiz-Shulcloper J, Sanniti di Baja G, editors. Progress in pattern recognition, image analysis, computer vision, and applications. Berlin: Springer; 2013. p. 125‒32. 链接1

[23] Atkeson CG, Moore AW, Schaal S. Locally weighted learning. Artif Intell Rev 1997;11:11‒73. 链接1

[24] Zhang JJ. Identification of moving loads using a local linear embedding algorithm. J Vib Control 2019;25(11):1780‒90. 链接1

[25] Loia V, Tomasiello S, Vaccaro A, Gao JW. Using local learning with fuzzy transform: application to short term forecasting problems. Fuzzy Optim Decis Making 2020;19(1):13‒32. 链接1

[26] Weinberger KQ, Saul LK. Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 2009;10:207‒44. 链接1

[27] Nguyen B, Morell C, De Baets B. Large-scale distance metric learning for k-nearest neighbors regression. Neurocomputing 2016;214:805‒14. 链接1

[28] Xing EP, Ng AY, Jordan MI, Russell S. Distance metric learning, with application to clustering with side-information. In: Becker S, Thrun S, Obermayer K, editors. Advances in neural information processing systems 15: proceedings of the 2002 conference. Cambridge: A Bradford Book; 2003. p. 521‒8.

[29] Duan YQ, Lu JW, Zheng WZ, Zhou J. Deep adversarial metric learning. IEEE Trans Image Process 2020;29:2037‒51. 链接1

[30] Song HO, Xiang Y, Jegelka S, Savarese S. Deep metric learning via lifted structured feature embedding. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26‒Jul 1; Las Vegas, NV, USA; 2016. p. 4004‒12. 链接1

[31] Cui Y, Zhou F, Lin YQ, Belongie S. Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26‒Jul 1; Las Vegas, NV, USA; 2016. p. 1153‒62. 链接1

[32] ALzubi JA, Bharathikannan B, Tanwar S, Manikandan R, Khanna A, Thaventhiran C. Boosted neural network ensemble classification for lung cancer disease diagnosis. Appl Soft Comput 2019;80:579‒91. 链接1

[33] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. 2015. arXiv:1502.03167.

[34] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improving neural networks by preventing co-adaptation of feature detectors. 2012. arXiv:1207.0580.

[35] Kay S. Can detectability be improved by adding noise? IEEE Signal Process Lett 2000;7(1):8‒10. 链接1

[36] Boyd V, Vandenberghe L, Faybusovich L. Convex optimization. IEEE Trans Automat Contr 2006;51(11):1859. 链接1

[37] Cortez P, Morais A. A data mining approach to predict forest fires using meteorological data. In: NevesJM, SantosMF, MachadoJM, editors. New trends in artificial intelligence: proceedings of the 13th Portuguese Conference on Artificial Intelligence; 2007 Dec 3‒7; Guimarães, Portugal; 2007. p. 512‒23. French. 链接1

[38] Cortez P, Cerdeira A, Almeida F, Matos T, Reis J. Modeling wine preferences by data mining from physicochemical properties. Decis Support Syst 2009;47(4):547‒53. 链接1

[39] De Vito S, Massera E, Piga M, Martinotto L, Di Francia G. On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B Chem 2008;129(2):750‒7. 链接1

[40] Ke GL, Meng Q, Finley T, Wang TF, Chen W, Ma WD, et al. LightGBM: a highly efficient gradient boosting decision tree. In: GuyonI, Von LuxburgU, BengioS, WallachH, FergusR, VishwanathanS, et al., editors. Proceedings of the 31st Annual Conference on Neural Information Processing Systems; 2017 Dec 4‒9; Long Beach, CA, USA; 2017.

[41] Guo HF, Tang RM, Ye YM, Li ZG, He XQ. DeepFM: a factorization-machine based neural network for CTR prediction. In: SierraC, editor. Proceedings of the 26th International Joint Conference on Artificial Intelligence; 2017 Aug 19‒25; Melbourne, VIC, Australia; 2017. p. 1725‒31. 链接1

[42] Zhang L, Shen WC, Huang JH, Li SJ, Pan G. Field-aware neural factorization machine for click-through rate prediction. IEEE Access 2019;7:75032‒40. 链接1

[43] Huang JY, Zhang X, Fang BX. CoStock: a DeepFM model for stock market prediction with attentional embeddings. In: Proceedings of 2019 IEEE International Conference on Big Data; 2019 Dec 9‒12; AngelesLos, CA, USA. New York City: IEEE; 2019. p. 5522‒31. 链接1

相关研究