An algorithm based on deep semantic segmentation called LC-DeepLab is proposed for detecting the trends and geometries of cracks on tunnel linings at the pixel level. The proposed method addresses the low accuracy of tunnel crack segmentation and the slow detection speed of conventional models in complex backgrounds. The novel algorithm is based on the DeepLabv3+ network framework. A lighter backbone network was used for feature extraction. Next, an efficient shallow feature fusion module that extracts crack features across pixels is designed to improve the edges of crack segmentation. Finally, an efficient attention module that significantly improves the anti-interference ability of the model in complex backgrounds is validated. Four classic semantic segmentation algorithms (fully convolutional network, pyramid scene parsing network, U-Net, and DeepLabv3+) are selected for comparative analysis to verify the effectiveness of the proposed algorithm. The experimental results show that LC-DeepLab can accurately segment and highlight cracks from tunnel linings in complex backgrounds, and the accuracy (mean intersection over union) is 78.26%. The LC-DeepLab can achieve a real-time segmentation of 416 × 416 × 3 defect images with 46.98 f/s and 21.85 Mb parameters.