From Signal to Knowledge: The Diagnostic Value of Raw Data in the Artificial Intelligence Prediction of Human Data for the First Time

Bingxi He, Yu Guo, Yongbei Zhu, Lixia Tong, Boyu Kong, Kun Wang, Caixia Sun, Hailin Li, Feng Huang, Liwei Wu, Meng Wang, Fanyang Meng, Le Dou, Kai Sun, Tong Tong, Zhenyu Liu, Ziqi Wei, Wei Mu, Shuo Wang, Zhenchao Tang, Shuaitong Zhang, Jingwei Wei, Lizhi Shao, Mengjie Fang, Juntao Li, Shouping Zhu, Lili Zhou, Shuo Wang, Di Dong, Huimao Zhang, Jie Tian

Engineering ›› 2024, Vol. 34 ›› Issue (3) : 60-69.

PDF(2888 KB)
PDF(2888 KB)
Engineering ›› 2024, Vol. 34 ›› Issue (3) : 60-69. DOI: 10.1016/j.eng.2023.02.013
Research
Article

From Signal to Knowledge: The Diagnostic Value of Raw Data in the Artificial Intelligence Prediction of Human Data for the First Time

Author information +
History +

Highlights

・The flow of modern medical services is from signal to image to knowledge

・Information distortion and noise are introduced in the process from signal to image.

・It is the first rawdata experiment using human clinical data for a clinical problem.

・The fusion of CT rawdata can stably improve the diagnostic performance of the model.

Abstract

Encouraging and astonishing developments have recently been achieved in image-based diagnostic technology. Modern medical care and imaging technology are becoming increasingly inseparable. However, the current diagnosis pattern of signal to image to knowledge inevitably leads to information distortion and noise introduction in the procedure of image reconstruction (from signal to image). Artificial intelligence (AI) technologies that can mine knowledge from vast amounts of data offer opportunities to disrupt established workflows. In this prospective study, for the first time, we develop an AI-based signal-to-knowledge diagnostic scheme for lung nodule classification directly from the computed tomography (CT) raw data (the signal). We find that the raw data achieves almost comparable performance with CT, indicating that it is possible to diagnose diseases without reconstructing images. Moreover, the incorporation of raw data through three common convolutional network structures greatly improves the performance of the CT models in all cohorts (with a gain ranging from 0.01 to 0.12), demonstrating that raw data contains diagnostic information that CT does not possess. Our results break new ground and demonstrate the potential for direct signal-to-knowledge domain analysis.

Graphical abstract

Keywords

Computed tomography / Diagnosis / Deep learning / Lung cancer / Raw data

Cite this article

EndNote

Ris (Procite)

Bibtex

Download citation ▾
Bingxi He, Yu Guo, Yongbei Zhu, Lixia Tong, Boyu Kong, Kun Wang, Caixia Sun, Hailin Li, Feng Huang, Liwei Wu, Meng Wang, Fanyang Meng, Le Dou, Kai Sun, Tong Tong, Zhenyu Liu, Ziqi Wei, Wei Mu, Shuo Wang, Zhenchao Tang, Shuaitong Zhang, Jingwei Wei, Lizhi Shao, Mengjie Fang, Juntao Li, Shouping Zhu, Lili Zhou, Shuo Wang, Di Dong, Huimao Zhang, Jie Tian. From Signal to Knowledge: The Diagnostic Value of Raw Data in the Artificial Intelligence Prediction of Human Data for the First Time. Engineering, 2024, 34(3): 60‒69 https://doi.org/10.1016/j.eng.2023.02.013

References

Publishing order | Descend order by publishing year | Descend order by cited within
[1]
E.C. Ciccarelli, A.J. Jacobs, P. Berman. Looking back on the millennium in medicine. N Engl J Med, 342 (2000), pp. 42-49.
[2]
L.J. Lauwerends, P.B.A.A. van Driel, R.J. Baatenburg de Jong, J.A.U. Hardillo, S. Koljenovic, G. Puppels, et al. Real-time fluorescence imaging in intraoperative decision making for cancer surgery. Lancet Oncol, 22 (5) (2021), pp. e186-e195.
[3]
C.D. Lehman, R.D. Wellman, D.S.M. Buist, K. Kerlikowske, A.N. Tosteson, D.L. Miglioretti, the Breast Cancer Surveillance Consortium. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med, 175 (11) (2015), pp. 1828-1837.
[4]
W.L. Bi, A. Hosny, M.B. Schabath, M.L. Giger, N.J. Birkbak, A. Mehrtash, et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin, 69 (2) (2019), pp. 127-157.
[5]
A. Hosny, C. Parmar, J. Quackenbush, L.H. Schwartz, H.J.W.L. Aerts. Artificial intelligence in radiology. Nat Rev Cancer, 18 (8) (2018), pp. 500-510.
[6]
G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, et al. A survey on deep learning in medical image analysis. Med Image Anal, 42 (2017), pp. 60-88.
[7]
P. Lambin, R.T.H. Leijenaar, T.M. Deist, J. Peerlings, E.E.C. de Jong, J. van Timmeren, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol, 14 (12) (2017), pp. 749-762.
[8]
X. Liu, L. Faes, A.U. Kale, S.K. Wagner, D.J. Fu, A. Bruynseels, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health, 1 (6) (2019), pp. e271-e297.
[9]
D. Killock. AI outperforms radiologists in mammographic screening. Nat Rev Clin Oncol, 17 (3) (2020), p. 134.
[10]
S.C. Rivera, X. Liu, A.W. Chan, A.K. Denniston, M.J. Calvert, H. Ashrafian, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digit Health, 2 (10) (2020), pp. e549-e560.
[11]
D. Dong, M.J. Fang, L. Tang, X.H. Shan, J.B. Gao, F. Giganti, et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol, 31 (7) (2020), pp. 912-920.
[12]
Y.Q. Huang, C.H. Liang, L. He, J. Tian, C.S. Liang, X. Chen, et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol, 34 (18) (2016), pp. 2157-2164.
[13]
W. Mu, M.B. Schabath, R.J. Gillies. Images are data: challenges and opportunities in the clinical translation of radiomics. Cancer Res, 82 (11) (2022), pp. 2066-2068.
[14]
R.J. Gillies, P.E. Kinahan, H. Hricak. Radiomics: images are more than pictures, they are data. Radiology, 278 (2) (2016), pp. 563-577.
[15]
B. Zhu, J.Z. Liu, S.F. Cauley, B.R. Rosen, M.S. Rosen. Image reconstruction by domain-transform manifold learning. Nature, 555 (7697) (2018), pp. 487-492.
[16]
G. Wang, J.C. Ye. B. De Man. Deep learning for tomographic image reconstruction. Nat Mach Intell, 2 (12) (2020), pp. 737-748.
[17]
C. Chung, J. Kalpathy-Cramer, M.V. Knopp, D.A. Jaffray. In the era of deep learning, why reconstruct an image at all?. J Am Coll Radiol, 18 (1) (2021), pp. 170-173.
[18]
G. Wang. A perspective on deep imaging. IEEE Access, 4 (2016), pp. 8914-8924.
[19]
M. Kalra, G. Wang, C.G. Orton. Radiomics in lung cancer: its time is here. Med Phys, 45 (3) (2018), pp. 997-1000.
[20]
G. Wang, J.C. Ye, K. Mueller, J.A. Fessler. Image reconstruction is a new frontier of machine learning. IEEE Trans Med Imaging, 37 (6) (2018), pp. 1289-1296.
[21]
Q. De Man, E. Haneda, B. Claus, P. Fitzgerald, B. De Man, G. Qian, et al. A two-dimensional feasibility study of deep learning-based feature detection and characterization directly from CT sinograms. Med Phys, 46 (12) (2019), pp. e790-e800.
[22]
Wu D, Kim K, Dong B, Li Q. End-to-end abnormality detection in medical imaging. In:Proceedings of the 6th International Conference on Learning Representations (ICLR 2018) 2018 Apr 30-May 3; Vancouve, BC, Canada. San Francisco: OpenReview; 2018..
[23]
Y. Gao, J. Tan, Z. Liang, L. Li, Y. Huo. Improved computer-aided detection of pulmonary nodules via deep learning in the sinogram domain. Vis Comput Ind Biomed Art, 2 (1) (2019), p. 15.
[24]
D. Dong, B. He, B. Kong, L. Zhang, L. Tong, F. Huang, et al. Abstract CT274: diagnosis based on signal: the first time break the routinely used circle of signal-to-image-to-diagnose. Cancer Res, 80 (Suppl 16) (2020), p. CT274.
[25]
X. Xu, C. Wang, J. Guo, Y. Gan, J. Wang, H. Bai, et al. MSCS-DeepLN: evaluating lung nodule malignancy using multi-scale cost-sensitive neural networks. Med Image Anal, 65 (2020), 101772.
[26]
D. Ardila, A.P. Kiraly, S. Bharadwaj, B. Choi, J.J. Reicher, L. Peng, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med, 25 (6) (2019), pp. 954-961.
[27]
L. Liu, Q. Dou, H. Chen, J. Qin, P.A. Heng. Multi-task deep model with margin ranking loss for lung nodule analysis. IEEE Trans Med Imaging, 39 (3) (2020), pp. 718-728.
[28]
Y. Xie, Y. Xia, J. Zhang, Y. Song, D. Feng, M. Fulham, et al. Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans Med Imaging, 38 (4) (2019), pp. 991-1004.
[29]
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017); 2017 Jul 21-26; Honolulu, HI, USA. New York City: IEEE; 4700-8.
[30]
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision Pattern Recognition; 2016 Jun 26-Jul 1; Las Vegas, NV, USA. New York City: IEEE; 2016. p. 770-8..
[31]
Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017); 2017 Jul 21-26; Honolulu, HI, USA. New York City: IEEE; 1492-500.
[32]
W. Shen, M. Zhou, F. Yang, D. Yu, D. Dong, C. Yang, et al. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit, 61 (2017), pp. 663-673.
[33]
P. Mukherjee, M. Zhou, E. Lee, A. Schicht, Y. Balagurunathan, S. Napel, et al. A shallow convolutional neural network predicts prognosis of lung cancer patients in multi-institutional computed tomography image datasets. Nat Mach Intell, 2 (5) (2020), pp. 274-282.
[34]
Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM++:generalized gradient-based visual explanations for deep convolutional networks. In: Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV 2018); 2018 Mar 12-15; Lake Tahoe, NV, USA. New York City: IEEE; 839-47.
[35]
Wang H, Wang Z, Du M, Yang F, Zhang Z, Ding S, et al. Score-CAM:score-weighted visual explanations for convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020); 2020 Jun 14-19; online. New York City: IEEE; 24-5.
AI Summary AI Mindmap
PDF(2888 KB)

Accesses

Citations

Detail
Sections
Recommended

/

〈 〉