2021, Volume 7, Issue 12
Engineering >> 2021, Volume 7, Issue 12 doi: 10.1016/j.eng.2020.07.026
Real-Time Detection of Cracks on Concrete Bridge Decks Using Deep Learning in the Frequency Domain
a Department of Civil and Environmental Engineering, University of Pittsburgh, Pittsburgh, PA 15261, USA
b Wiss, Janney, Elstner Associates (WJE) Inc., Northbrook, IL 60062, USA
c Department of Computer Science and Information Engineering, Asia University, Taichung, Taiwan 41354, China
Next Previous
Abstract
This paper presents a vision-based crack detection approach for concrete bridge decks using an integrated one-dimensional convolutional neural network (1D-CNN) and long short-term memory (LSTM) method in the image frequency domain. The so-called 1D-CNN-LSTM algorithm is trained using thousands of images of cracked and non-cracked concrete bridge decks. In order to improve the training efficiency, images are first transformed into the frequency domain during a preprocessing phase. The algorithm is then calibrated using the flattened frequency data. LSTM is used to improve the performance of the developed network for long sequence data. The accuracy of the developed model is 99.05%, 98.9%, and 99.25%, respectively, for training, validation, and testing data. An implementation framework is further developed for future application of the trained model for large-scale images. The proposed 1D-CNN-LSTM method exhibits superior performance in comparison with existing deep learning methods in terms of accuracy and computation time. The fast implementation of the 1D-CNN-LSTM algorithm makes it a promising tool for real-time crack detection.
Keywords
Crack detection ; Concrete bridge deck ; Deep learning ; Real-time
Figures
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
References
[ 1 ] ASCE’s 2017 infrastructure report card: bridges [Internet]. Reston: American Society of Civil Engineers; 2017 [cited 2017 Oct 11]. Available from: https:// www.infrastructurereportcard.org/cat-item/bridges/.
[ 2 ] Spencer BF Jr, Hoskere V, Narazaki Y. Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering 2019;5(2):199–222. link1
[ 3 ] Bao Y, Chen Z, Wei S, Xu Y, Tang Z, Li H. The state of the art of data science and engineering in structural health monitoring. Engineering 2019;5(2):234–42. link1
[ 4 ] Fujino Y, Siringoringo DM, Ikeda Y, Nagayama T, Mizutani T, Fujino ZY, et al. Research and implementations of structural monitoring for bridges and buildings in Japan. Engineering 2019;5(6):1093–119. link1
[ 5 ] Ansari F. Sensing issues in civil structural health monitoring. New York: Springer; 2005. link1
[ 6 ] Babanajad SK, Zhan Y, Taylor T, Ansari F. Virtual reference approach for dynamic distributed sensing of damage in large structures. J Aerosp Eng 2017;30(2):1–12. link1
[ 7 ] Iyer S, Sinha SK. A robust approach for automatic detection and segmentation of cracks in underground pipeline images. Image Vis Comput 2005;23 (10):921–33. link1
[ 8 ] Salman M, Mathavan S, Kamal K, Rahman M. Pavement crack detection using the Gabor filter. In: Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems; 2013 Oct 6–9; Hague, the Netherlands; 2014. link1
[ 9 ] Talab AMA, Huang Z, Xi F, HaiMing L. Detection crack in image using Otsu method and multiple filtering in image processing techniques. Optik 2016;127 (3):1030–3. link1
[10] Shan B, Zheng S, Ou J. A stereovision-based crack width detection approach for concrete surface assessment. KSCE J Civ Eng 2016;20(2):803–12. link1
[11] Sinha SK, Fieguth PW. Automated detection of cracks in buried concrete pipe images. Autom Construct 2006;15(1):58–72. link1
[12] Zou Q, Cao Y, Li Q, Mao Q, Wang S. CrackTree: automatic crack detection from pavement images. Pattern Recognit Lett 2012;33(3):227–38. link1
[13] Fujita Y, Hamamoto Y. A robust automatic crack detection method from noisy concrete surfaces. Mach Vis Appl 2011;22(2):245–54. link1
[14] Plehiers PP, Symoens SH, Amghizar I, Marin GB, Stevens CV, Van Geem KM. Artificial intelligence in steam cracking modeling: a deep learning algorithm for detailed effluent prediction. Engineering 2019;5(6):1027–40. link1
[15] Shang C, You F. Data analytics and machine learning for smart process manufacturing: recent advances and perspectives in the big data era. Engineering 2019;5(6):1010–6. link1
[16] Liu SW, Huang JH, Sung JC, Lee CC. Detection of cracks using neural networks and computational mechanics. Comput Methods Appl Mech Eng 2002;191 (25–26):2831–45. link1
[17] Moon H, Kim J. Intelligent crack detecting algorithm on the concrete crack image using neural network. In: Proceedings of the 28th International Symposium on Automation and Robotics in Construction; 2011 Jun 29–Jul 2; Seoul, Republic of Korea; 2011.
[18] O’Byrne M, Ghosh B, Schoefs F, Pakrashi V. Regionally enhanced multiphase segmentation technique for damaged surfaces. Comput Aided Civ Infrastruct Eng 2014;29(9):644–58. link1
[19] Prasanna P, Dana KJ, Gucunski N, Basily BB, La HM, Lim RS, et al. Automated crack detection on concrete bridges. IEEE Trans Autom Sci Eng 2016;13 (2):591–9. link1
[20] Ciresan DC, Meier U, Masci J, Gambardella LM, Schmidhuber J. Flexible, high performance convolutional neural networks for image classification. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence; 2011 Jul 16–22; Barcelona, Spain; 2011. link1
[21] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012;1:1097–105. link1
[22] Abdeljaber O, Avci O, Kiranyaz MS, Boashash B, Sodano H, Inman DJ. 1-D CNNs for structural damage detection: verification on a structural health monitoring benchmark data. Neurocomputing 2018;275:1308–17. link1
[23] De Oliveira MA, Monteiro AV, Filho JV. A new structural health monitoring strategy based on PZT sensors and convolutional neural network. Sensors 2018;18(9):1–21. link1
[24] Khodabandehlou H, Pekcan G, Fadali MS, Pekcan G, Fadali MS. Vibration-based structural condition assessment using convolution neural networks. Struct Contr Health Monit 2018;26(2):e2308. link1
[25] Tabian I, Fu H, Khodaei ZS. A convolutional neural network for impact detection and characterization of complex composite structures. Sensors 2019;19(22):4933. link1
[26] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436–44. link1
[27] Azimi M, Pekcan G. Structural health monitoring using extremely compressed data through deep learning. Comput-Aided Civ Infrastruct Eng 2020;35:597–614. link1
[28] Soukup D, Huber-Mörk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Bebis G, Boyle R, Parvin B, Koracin D, McMahan R, Jerald J, editors. Advances in visual computing. Cham: Springer; 2014. p. 668–77. link1
[29] Cha YJ, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks: deep learning-based crack damage detection using CNNs. Comput Aided Civ Infrastruct Eng 2017;32(5): 361–78. link1
[30] Da Silva WRL, de Lucena DS. Concrete cracks detection based on deep learning image classification. Proceedings 2018;2(8):489. link1
[31] Matsugu M, Mori K, Mitari Y, Kaneda Y. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Netw 2003;16(5–6):555–9. link1
[32] Fukushima K. Neocognitron. Scholarpedia 2007;2(1):1717. link1
[33] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44. link1
[34] Scherer D, Muller A, Behnke S. Evaluation of pooling operations in convolutional architectures for object recognition. In: Proceedings of the 20th International Conference on Artificial Neural Networks; 2010 Sep 15–18; Thessaloniki, Greece; 2010.
[35] Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015;61:85–117. link1
[36] Donahue J, Jia Y, Vinyals O, Hoffman J. DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st International Conference on Machine Learning; 2014 Jun 21–26; Beijing, China; 2014. p. 647–55.
[37] Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. Overfeat: integrated recognition, localization and detection using convolutional networks. 2013. ArXiv:1312.6229.
[38] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9 (8):1735–80. link1
[39] Gers FA, Schmidhuber E. LSTM recurrent networks learn simple context-free and context-sensitive languages. IEEE Trans Neural Netw 2001;12 (6):1333–40. link1
[40] Swapna G, Kp S, Vinayakumar R. Automated detection of diabetes using CNN and CNN-LSTM network and heart rate signals. Procedia Comput Sci 2018;132:1253–62. link1
[41] Zhao J, Mao X, Chen L. Speech emotion recognition using deep 1D & 2D CNN LSTM networks. Biomed Signal Process Control 2019;47:312–23. link1
[42] Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J. LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst 2017;28 (10):2222–32. link1
[43] Wang J, Zhang J, Wang X. Bilateral LSTM: a two-dimensional long short-term memory model with multiply memory units for short-term cycle time forecasting in re-entrant manufacturing systems. IEEE Trans Ind Inf 2018;14 (2):748–58. link1
[44] Tsironi E, Barros P, Weber C, Wermter S. An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 2017;268:76–86. link1
[45] Oehmcke S, Zielinski O, Kramer O. Input quality aware convolutional LSTM networks for virtual marine sensors. Neurocomputing 2018;275:2603–15. link1
[46] Zhou X, Hu B, Chen Q, Wang X. Recurrent convolutional neural network for answer selection in community question answering. Neurocomputing 2018;274:8–18. link1
[47] Sainath TN, Vinyals O, Senior A, Sak H. Convolutional longshort-term memory fully connected deep neural networks. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing; 2015 Apr 19–24; South Brisbance, QLD, Australia; 2015.
[48] Liu T, Bao J, Wang J, Zhang Y. A hybrid CNN–LSTM algorithm for online defect recognition of CO2 welding. Sensors 2018;18(12):4369. link1
[49] Das A. Interpretation and processing of image in frequency domain. In: Guide to signals and patterns in image processing. Berlin: Springer; 2015. p. 93–147. link1
[50] Chang SG, Yu B, Vetterli M. Adaptive wavelet thresholding for image denoising and compression. IEEE Trans Image Process 2000;9(9):1532–46. link1
[51] Zhang M, Gunturk BK. Multiresolution bilateral filtering for image denoising. IEEE Trans Image Process 2008;17(12):2324–33. link1
[52] Portilla J, Strela V, Wainwright MJ, Simoncelli EP. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Process 2003;12(11):1338–51. link1
[53] Gonzalez RC, Woods RE, Masters BR. Digital image processing. 3rd edition. Upper Saddle River: Prentice Hall; 2008. link1
[54] Maguire M, Dorafshan S, Thomas R. SDNET2018: a concrete crack image dataset for machine learning applications. Logan: Utah State University; 2018. link1
[55] Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. 2016. arXiv:1603.04467.
[56] Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986;8(6):679–98. link1
[57] Sobel I, Feldman G. A 3 3 isotropic gradient operator for image processing. A talk at the stanford Artificial Project. 1968:271–2.
京公网安备 11010502051620号
