A primary research area in structure health monitoring is damage localization and assessment. Reliably locating damage and forecasting its tendency are important for protecting the safety of the structure. There have been many studies on the structural damage assessment. This problem was explored using physicsbased[
51–
53] and data-driven-based methods [
54]. The natural frequency, mode, curvature, and vibratory characteristics play a vital role in the former type of methods. The analytical models are used with simulations for calibration to obtain the physical characteristics and structural current states. However, the rapidly growing data volume poses a challenge for physical-based methods. Additionally, determining the physical model could be rather difficult, as the quality of data could vary and strongly influencing. Differently, the data-driven methods discover the hidden correlations and sensitive features to assess the structural conditions. Numerous statistical approaches were employed, such as the state-space model[
55,
56], the auto-regressive model[
57–
59], and the ANN[
60,
61]. The Mahalanobis distance was employed to detect the outliers [
62]. A fuzzy-logic model was established to build correlations between maximum acceleration amplitudes with nominal train speeds [
63]. Clustering is also a popular unsupervised learning method, which can discover abnormal data according to the outliers[
64,
65]. Liu and Ni [
66] assumed a Gaussian distribution for the normalized rail bending strain. Recently, some deep learning methods, such as one-dimentional (1D) CNN and DNN, were introduced to deal with the insufficiency of statistical methods’ ability to capture the non-linear correlation[
67,
68]. However, these solutions do not improve the problem formulation or capture the high-level feature correlations of the monitoring data and the damage. The correlations between multiple kinds of monitoring data are still unknown. With a feature combination and selection, the hidden pattern and information could be shared and transferred properly in this domain. In this way, the deep multi-task learning framework explores the information sharing mechanism in the inputs and outputs[
69,
70], which provides a new perspective to solve the real-world problem.