Journal Home Online First Current Issue Archive For Authors Journal Information 中文版

Engineering >> 2021, Volume 7, Issue 6 doi: 10.1016/j.eng.2020.07.030

The State-of-the-Art Review on Applications of Intrusive Sensing, Image Processing Techniques, and Machine Learning Methods in Pavement Monitoring and Analysis

a Beijing Key Laboratory of Traffic Engineering, Beijing University of Technology, Beijing 100124, China
b Key Laboratory of Transport Industry of Big Data Application Technologies for Comprehensive Transport, Beijing Jiaotong University, Beijing 100044, China
c Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
d National Center for Materials Service Safety, University of Science and Technology Beijing, Beijing 100083, China
e School of Transportation, Southeast University, Nanjing 211189, China
f Department of Civil and Environmental Engineering, Virginia Tech, Blacksburg, VA 24061, USA

Received: 2020-05-10 Revised: 2020-07-05 Accepted: 2020-07-24 Available online: 2021-12-29

Next Previous

Abstract

In modern transportation, pavement is one of the most important civil infrastructures for the movement of vehicles and pedestrians. Pavement service quality and service life are of great importance for civil engineers as they directly affect the regular service for the users. Therefore, monitoring the health status of pavement before irreversible damage occurs is essential for timely maintenance, which in turn ensures public transportation safety. Many pavement damages can be detected and analyzed by monitoring the structure dynamic responses and evaluating road surface conditions. Advanced technologies can be employed for the collection and analysis of such data, including various intrusive sensing techniques, image processing techniques, and machine learning methods. This review summarizes the state-of-the-art of these three technologies in pavement engineering in recent years and suggests possible developments for future pavement monitoring and analysis based on these approaches.

Figures

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

References

[ 1 ] Rollings RS, Pittman DW. Field instrumentation and performance monitoring of rigid pavements. J Transp Eng 1992;118(3):361–70. link1

[ 2 ] Sebaaly P, Tabatabaee N, Scullion T. Instrumentation for flexible pavementsfield performance of selected sensors. Final report. Washington, DC: Federal Highway Administration; 1992 Jun. Report No.: FHWA-RD-91-094. Contract No.: DTFH61-88-R-00052.

[ 3 ] Xue W, Weaver E. Pavement shear strain response to dual and wide-base tires. Transp Res Rec J Transport Res Board 2011;2225(1):155–64. link1

[ 4 ] Al-Qadi IL, Loulizi A, Elseifi M, Lahouar S. The virginia smart road: the impact of pavement instrumentation on understanding pavement performance. Asphalt Paving Technol 2004;73:427–65. link1

[ 5 ] Gonçalves P, Ceratti JAP, Bica AD. The use of embedded stress cells for monitoring pavement performance. Geotech Test J 2003;26(4):363–72. link1

[ 6 ] Timm DH, Priest AL. Dynamic pavement response data collection and processing at the NCAT test track. Final report. Auburn: National Center for Asphalt Technology; 2004. Report No.: NCAT Report 04–03. link1

[ 7 ] Scholz T. Instrumentation for mechanistic design implementation. Final report. Portland: Transportation Research and Education Center; 2010. Report No.: OTREC-RR-10-02.

[ 8 ] Hornyak NJ, Crovetti JA, Newman DE, Schabelski JP. Perpetual pavement instrumentation for the marquette interchange project-phase 1. Final report. Milwaukee: Transportation Research Center, Marquette University, Wisconsin Highway Research Program; 2007 Aug. Report No.:WHRP 07–11.

[ 9 ] Xue W, Wang L, Wang D, Druta C. Pavement health monitoring system based on an embedded sensing network. J Mater Civ Eng 2014;26(10):04014072. link1

[10] Xue W, Wang D, Wang L. Monitoring the speed, configurations, and weight of vehicles using an in-situ wireless sensing network. IEEE Trans Intell Transport Syst 2015;16(4):1667–75. link1

[11] Cho I, Lee JH, Park J, Yi DH, Cho D, Kim SW. A new method for accurately estimating the weight of moving vehicles using piezoelectric sensors and adaptive-footprint tire model. Veh Syst Dyn 2003;39(2):135–48. link1

[12] Zhang Z, Huang Y, Bridgelall R, Palek L, Strommen R. Sampling optimization for high-speed weigh-in-motion measurements using in-pavement strainbased sensors. Meas Sci Technol 2015;26(6):065003. link1

[13] Zhang W, Wang Q, Suo C. A novel vehicle classification using embedded strain gauge sensors. Sensors 2008;8(11):6952–71.

[14] Xue W, Wang L, Wang D. A prototype integrated monitoring system for pavement and traffic based on an embedded sensing network. IEEE Trans Intell Transp Syst 2015;16(3):1380–90. link1

[15] Yang H. Development of a piezoelectric energy harvesting system for applications in collecting pavement deformation energy [dissertation]. Beijing Chinese: University of Science and Technology Beijing; 2018. Chinese. link1

[16] Mazurek B, Janiczek T, Chmielowiec J. Assessment of vehicle weight measurement method using PVDF transducers. J Electrostat 2001;51– 52:76–81. link1

[17] Zhang J, Lu Y, Lu Z, Liu C, Sun G, Li Z. A new smart traffic monitoring method using embedded cement-based piezoelectric sensors. Smart Mater Struct 2015;24(2):025023. link1

[18] Wang L, Hu X, Huang Y, Xu H. Based on fiber-optic sensor and the light intensity changes vehicle dynamic weighing system. In: Proceedings of the 5th International Conference on Computational Intelligence and Communication Networks; 2013 Sep 27–29; Washington, DC, USA; 2013.

[19] Malla RB, Sen A, Garrick NW. A special fiber optic sensor for measuring wheel loads of vehicles on highways. Sensors 2008;8(4):2551–68.

[20] Yuan S, Ansari F, Liu X, Zhao Y. Optic fiber-based dynamic pressure sensor for WIM system. Sens Actuators A 2005;120(1):53–8. link1

[21] Batenko A, Grakovski A, Kabashkin I, Petersons E, Sikerzhicki Y. Weight-inmotion (WIM) measurements by fiber optic sensor: problems and solutions. Transp Telecommun 2011;12(4):27–33. link1

[22] Zhang H, Wei Z, Fan L, Yang S, Wang P, Cui H. A high speed, portable, multifunction, weigh-in-motion (WIM) sensing system and a high performance optical fiber Bragg grating (FBG) demodulator. In: Proceedings of the SPIE 7647, Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems; 2010 Mar 9; San Diego, CA, USA; 2010.

[23] Zhao H, Wu D, Zeng M, Zhong S. A vibration-based vehicle classification system using distributed optical sensing technology. Transp Res Rec J Transp Res Board 2018;2672(43):12–23. link1

[24] Dong Z, Ma X, Shao X. Airport pavement responses obtained from wireless sensing network upon digital signal processing. Int J Pavement Eng 2018;19 (5):381–90. link1

[25] Gubbi J, Buyya R, Marusic S, Palaniswami M. Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener Comput Syst 2013;29(7):1645–60. link1

[26] Ho CM, Tai YC. Micro-electro-mechanical-systems (MEMS) and fluid flows. Annu Rev Fluid Mech 1998;30(1):579–612. link1

[27] Alavi AH, Hasni H, Lajnef N, Chatti K. Continuous health monitoring of pavement systems using smart sensing technology. Constr Build Mater 2016;114:719–36. link1

[28] Ong JB, You Z, Mills-Beale J, Tan EL, Pereles BD, Ong KG. A wireless, passive embedded sensor for real-time monitoring of water content in civil engineering materials. IEEE Sens J 2008;8(12):2053–8. link1

[29] Lian K. Developing embedded wireless strain/stress/temperature sensor platform for highway applications. Final report. Baton Rouge: Louisiana State University, Transportation Research Board of the National Academies; 2010 Mar. Report No.: NCHRP IDEA Project 129.

[30] Krüger M, Grosse CU, Marrón PJ. Wireless structural health monitoring using MEMS. Key Eng Mater 2005;293–294:625–34. link1

[31] Bennett R, Hayes-Gill B, Crowe JA, Armitage R, Rodgers D, Hendroff A. Wireless monitoring of highways. In: Proceedings of the SPIE 3671, Smart Structures and Materials 1999: Smart Systems for Bridges, Structures, and Highways; 1999 May 18; Newport Beach, CA, USA; 1999.

[32] Haoui A, Kavaler R, Varaiya P. Wireless magnetic sensors for traffic surveillance. Transp Res Part C Emerging Technol 2008;16(3):294–306. link1

[33] Pei JS, Ivey RA, Lin H, Landrum AR, Sandburg CJ, Ferzli NA, et al. An experimental investigation of applying Mica2 Motes in pavement condition monitoring. J Intell Mater Syst Struct 2009;20(1):63–85. link1

[34] Chua KM, Xu L. Simple procedure for identifying pavement damages from video images. J Transp Eng 1994;120(3):412–31. link1

[35] Zakeri H, Nejad FM, Fahimifar A. Image based techniques for crack detection, classification and quantification in asphalt pavement: a review. Arch Comput Methods Eng 2017;24(4):935–77. link1

[36] Hoang ND, Nguyen QL. Fast local Laplacian-based steerable and Sobel filters integrated with adaptive boosting classification tree for automatic recognition of asphalt pavement cracks. Adv Civ Eng 2018;2018:5989246. link1

[37] Sun B, Qiu Y, Liang S. Cracking recognition of pavement surface based on wavelet technology. J Chongqing Jiaotong University 2010;29(1):69–72. Chinese.

[38] Balbin JR, Hortinela CC, Garcia RG, Baylon S, Ignacio AJ, Rivera MA, et al. Pattern recognition of concrete surface cracks and defects using integrated image processing algorithms. In: Proceedings of the 2nd International Workshop on Pattern Recognition; 2017 May 1–3; Singapore, Singapore; 2017.

[39] Wang B. A study on cement pavement crack detection based on Matlab image processing. J Shangluo Univ 2014;28(4):42–5. Chinese.

[40] Li M. Design and implementation of pavement inspection system based on image analysis [dissertation]. Shengyang: Liaoning University; 2018. Chinese. link1

[41] Gonzalez RC, Woods RE. Digital image processing. 3rd ed. Upper Saddle River: Prentice Hall; 2007. link1

[42] Ma RG, Xu K, Liu FF. Highway surface crack image identifying algorithm. J Transp Inf Saf 2014;32(2):90–3. Chinese.

[43] Wang PP. Research on pavement crack recognition based on improved gray scale segmentation algorithm [dissertation]. Xi’an: Chang’an University; 2014. Chinese.

[44] Liu N, Song W, Zhao Q. Morphology and maximum entropy image segmentation based urban pavement cracks detection. J Liaoning Tech Univ 2015;34(1):57–61. Chinese.

[45] Han K, Han HF. Pavement crack detection method based on region-level and pixel-level features. J Railw Sci Eng 2018;15(5):1178–86. Chinese.

[46] Luo YZ. Asphalt pavement crack detection based on image local mean standard deviation algorithm. Geomatics Spat Inf Technol 2017;40(12):167– 70. Chinese.

[47] Li YY, Huang QY, Hou ZX. Improve of OSTU based on hough transformation and applied in pavement crack detection. Electron Des Eng 2016;24(5):43–6. Chinese.

[48] Talab AMA, Huang Z, Xi F, Haiming L. Detection crack in image using Otsu method and multiple filtering in image processing techniques. Optik 2016;127(3):1030–3. link1

[49] Gao SB, Xie Z, Pan ZG, Qiu FZ, Li R. Novel automatic pavement crack detection algorithm. J Syst Simul 2017;29(9):2009–15. Chinese.

[50] Qiu LY. An algorithm of pavement crack detection based on edge detection. J Yancheng Inst Technol 2015;28(3):37–43. Chinese.

[51] Zhu Q. Pavement crack detection algorithm based on image processing analysis. In: Proceedings of the 8th International Conference on Intelligent Human-Machine Systems and Cybernetics; 2016 Sep 11–12; Hangzhou, China; 2016.

[52] Di Y. Research on recognition algorithm of pavement crack [dissertation]. Zhenzhou: Zhengzhou University; 2018. Chinese.

[53] Liu S, Wang W, Cao T, Yang N, Yang Y. Road crack extraction based on differential box dimension and maximum entropy threshold. J Chang’an Univ 2015;35(5):13–21. Chinese.

[54] Zhang H. Research on pavement crack detection system based on image processing [dissertation]. Shenyang: Shenyang Aerospace University; 2018. Chinese.

[55] Wen L. Improved gray correction algorithm for image preprocessing pavement cracks. Software Tech Algorithm 2015;24(2):220–3. Chinese.

[56] Gang W, Xu X, Liang X, He A. Algorithm based on the finite ridgelet transform for enhancing faint pavement cracks. Opt Eng 2008;47(1):017004. link1

[57] Li S, Hou DH, Gao J, Tong Z. Research on preprocessing method of pavement crack image via mathematical morphology. Highw Eng 2018;43(2):270–4. Chinese.

[58] Jia L. The extraction method of pavement crack edge based on automatic threshold detection and edge connection. Shanxi Sci Technol Commun 2015;5:21–4. Chinese.

[59] Qi HC, Xiao F, Bao XW. Improvement of edge detection discontinuity in the pavement crack recognition. Electron Des Eng 2014;22(1):32–4. Chinese.

[60] Zhang H, Meng H, Liu T. Analysis on the image edge detection algorithm of asphalt pavement crack. North Commun 2015;3:56–9. Chinese.

[61] Xu AH, Gao J. Crack identification method for cement pavement based on image enhancement and mathematical morphology. Highway 2015;25 (10):55–8. Chinese.

[62] Liu S. Method of shadow pavement crack extraction based on improved local threshold segmentation. Wireless Internet Technol 2018;15(20):112–3. Chinese.

[63] Xu H, Li Z, Jiang Y, Huang J. Pavement crack detection based on OpenCv and improved Canny operator. Comput Eng Des 2014;35(12):4254–8. Chinese.

[64] Wang X, Feng D, Li W. Research and implementation of pavement crack detection algorithm. J North China Inst Aerosp Eng 2017;27(5):9–13. Chinese.

[65] Koza JR, Bennett FH, Andre D, Keane MA. Automated design of both the topology and sizing of analog electrical circuits using genetic programming. In: Artificial Intelligence in Design. Boston: Kluwer Academic Publishers; 1996. p. 151–70. link1

[66] Cortes C, Vapnik VN. Support vector networks. Mach Learn 1995;20 (3):273–97. link1

[67] Bishop CM. Pattern recognition and machine learning (information science and statistics). New York: Springer; 2006. link1

[68] Hoang ND, Nguyen QL. Automatic recognition of asphalt pavement cracks based on image processing and machine learning approaches: a comparative study on classifier performance. Math Probl Eng 2018;2018:6290498. link1

[69] Hoang ND, Nguyen QL, Tien BD. Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J Comput Civ Eng 2018;32(5):04018037. link1

[70] Schlotjes MR, Burrow MPN, Evdorides HT, Henning TFP. Using support vector machines to predict the probability of pavement failure. Proc Inst Civ Eng Transp 2015;168(3):212–22. link1

[71] Pan YF, Zhang XF, Cervone G, Yang LP. Detection of asphalt pavement potholes and cracks based on the unmanned aerial vehicle multispectral imagery. IEEE J Sel Top Appl Earth Obs Remote Sens 2018;11(10):3701–12. link1

[72] Fujita Y, Shimada K, Ichihara M, Hamamoto Y. A method based on machine learning using hand-crafted features for crack detection from asphalt pavement surface images. In: Proceedings of the International Conference on Quality Control by Artificial Vision; 2017 May 14; Tokyo, Japan; 2017.

[73] Sapatinas T. The elements of statistical learning. J R Stat Soc Ser A 2004;167 (1):183–98. link1

[74] Wang H, Xie P, Ji R, Gagnon J. Prediction of airfield pavement responses from surface deflections: comparison between the traditional backcalculation approach and the ANN model. Road Mater Pavement Des. Forthcoming 2020.

[75] Nair V, Hinton GE. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning; 2010 Jun 21–25; Haifa, Israel; 2010.

[76] Maas AL, Hannun AY, Ng AY. Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the 30th International Conference on Machine Learning; 2013 Jun 16–21; Atlanta, GA, USA; 2013.

[77] Schmidhuber J. Deep learning in neural networks: an overview. Neural Networks 2015;61:85–117. link1

[78] Banharnsakun A. Hybrid ABC-ANN for pavement surface distress detection and classification. Int J Mach Learn Cybern 2017;8(2):699–710. link1

[79] Elbagalati O, Elseifi MA, Gaspard K, Zhang Z. Development of an enhanced decision-making tool for pavement management using a neural network pattern-recognition algorithm. J Transp Eng Part B Pavements 2018;144 (2):04018018. link1

[80] Ng A. Deep learning [Internet]. deeplearning.ai; c2020. [cited 2019 Mar 1]. Available from: www.deeplearning.ai/ deep-learning-specialization/. link1

[81] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv:1409.1556.

[82] Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A. Deep convolutional neural networks with transfer learning for computer vision-based datadriven pavement distress detection. Constr Build Mater 2017;157:322–30. link1

[83] Zhang A, Wang KCP, Li BX, Yang EH, Dai X, Peng Y, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput-Aided Civ Infrastruct Eng 2017;32(10):805–19. link1

[84] Cha YJ, Choi W, Suh G, Mahmoudkhani S, Buyukozturk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput-Aided Civ Infrastruct Eng 2018;33(9):731–47. link1

[85] Wu W, Fan Q, Zurada JM, Wang J, Yang D, Liu Y. Batch gradient method with smoothing L1/2 regularization for training of feedforward neural networks. Neural Networks 2014;50:72–8. link1

[86] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R. Improving neural networks by preventing co-adaptation of feature detectors. 2012. arXiv:1207.0580v1.

[87] Fei Y, Wang KCP, Zhang A, Chen C, Li B, Liu Y, et al. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V. IEEE Trans Intell Transp Syst 2020;21(1):273–84. link1

[88] Cha YJ, Choi W, Buyukozturk O. Deep learning-based crack damage detection using convolutional neural networks. Comput-Aided Civ Infrastruct Eng 2017;32(5):361–78. link1

[89] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning; 2015 Jul 6–11; Lille, France; 2015.

[90] Wu Y, He K. Group normalization. Int J Comput Vision 2020;128:742–55. link1

[91] Cotter A, Shamir O, Srebro N, Sridharan K. Better mini-batch algorithms via accelerated gradient methods. In: Shawe-Taylor J, Zemel R, Bartlett P, Pereira F, Weinberger KQ, editors. Advances in Neural Information Processing Systems 24. Red Hook: Curran Associates Inc.; 2011. p. 1647–55.

[92] Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning; 2013 Jun 16–21; Atlanta, GA, USA; 2013.

[93] Kingma D, Ba J. Adam: a method for stochastic optimization. 2014. arXiv:1412.6980.

[94] Dorafshan S, Thomas RJ, Maguire M. Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete. Constr Build Mater 2018;186:1031–45. link1

[95] Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks. Commun ACM 2012;60(6):1–9. link1

[96] Sobel I. Neighborhood coding of binary images for fast contour following and general binary array processing. Comput Graphics Image Process 1978;8 (1):127–35. link1

[97] Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986;8(6):679–98. link1

[98] Wang X, Hu Z. Grid-based pavement crack analysis using deep learning. In: Proceedings of the 4th International Conference on Transportation Information and Safety; 2017 Aug 8–10; Banff, Canada; 2017.

[99] Zhang A, Wang KCP, Fei Y, Liu Y, Tao SY, Chen C, et al. Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. J Comput Civ Eng 2018;32(5):04018041. link1

[100] Sha AM, Tong Z, Gao J. Recognition and measurement of pavement disasters based on convolutional neural networks. China J Highway Transp 2018;31 (1):1–10. Chinese.

[101] Tong Z, Gao J, Sha A, Hu L, Li S. Convolutional neural network for asphalt pavement surface texture analysis. Comput-Aided Civ Infrastruct Eng 2018;33(12):1056–72. link1

[102] Yang G, Li Q, Zhan Y, Fei Y, Zhang A. Convolutional neural network–based friction model using pavement texture data. J Comput Civ Eng 2018;32 (6):04018052. link1

[103] Tong Z, Gao J, Zhang HT. Innovative method for recognizing subgrade defects based on a convolutional neural network. Constr Build Mater 2018;169:69–82. link1

[104] Cao X, Wang P, Meng C, Bai X, Gong G, Liu M, et al. Region based CNN for foreign object debris detection on airfield pavement. Sensors 2018;18 (3):737. link1

[105] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 2017;39(6):1137–49. link1

[106] Majidifard H, Jahangiri B, Buttlar W, Alavi A. New machine learning-based prediction models for fracture energy of asphalt mixtures. Measurement 2019;135:438–51. link1

[107] Gong H, Sun Y, Mei Z, Huang B. Improving accuracy of rutting prediction for mechanistic-empirical pavement design guide with deep neural networks. Constr Build Mater 2018;190:710–8. link1

[108] Majidifard H, Adu-Gyamfi Y, Buttlar WG. Deep machine learning approach to develop a new asphalt pavement condition index. Constr Build Mater 2020;247:118513. link1

Related Research