2023, Volume 23, Issue 4
Engineering >> 2023, Volume 23, Issue 4 doi: 10.1016/j.eng.2022.10.015
Bayesian Inference and Dynamic Neural Feedback Promote the Clinical Application of Intelligent Congenital Heart Disease Diagnosis
a Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai 200433, China
b Cardiovascular Center & Clinical Laboratory Center, Children’s Hospital of Fudan University, National Children’s Medical Center, Shanghai 201102, China
# These authors contributed equally to this work.
Next Previous
Abstract
Congenital heart disease (CHD) is the leading cause of infant death. An artificial intelligence (AI)-based CHD diagnosis network (CHDNet) is an echocardiogram video-based binary classification model that judges whether echocardiogram videos contain heart defects. Existing CHDNets have shown performances comparable to or even better than medical experts, but their unreliability on cases outside of the training set has become the main bottleneck for their deployment. This is a common problem for most AI-based diagnostic approaches. Here, to overcome this challenge, we present two essential mechanisms— Bayesian inference and dynamic neural feedback—to respectively measure and improve the diagnostic reliability of AI. The former easily makes the neural network output its reliability instead of a single prediction result, while the latter is a computational neural feedback cell that allows the neural network to feed knowledge from the output layer back to the shallow layers and enables the neural network to selectively activate relevant neurons. To evaluate the effectiveness of these two mechanisms, we trained CHDNets on 4151 echocardiogram videos containing three common CHD defects and tested them on an internal test set of 1037 echocardiogram videos and an external set of 692 videos that were newly collected from other cardiovascular imaging devices. Each echocardiogram video corresponds to a unique patient and a unique visit. We demonstrate on various neural network architectures how the reliability obtained by Bayesian inference interprets and quantifies the significant performance difference between internal and external test sets of neural networks, and how the devised feedback cell helps the neural networks to maintain high accuracy and reliability, despite the input being corrupted by noise or when using an external test set.
SupplementaryMaterials
Figures
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
References
[ 1 ] Erikssen G, Liestøl K, Seem E, Birkeland S, Saatvedt KJ, Hoel TN, et al. Achievements in congenital heart defect surgery: a prospective, 40-year study of 7038 patients. Circulation 2015;131(4):337‒46, Discussion 346. link1
[ 2 ] Luo H, Qin G, Wang L, Ye Z, Pan Y, Huang L, et al. Outcomes of infant cardiac surgery for congenital heart disease concomitant with persistent pneumonia: a retrospective cohort study. J Cardiothorac Vasc Anesth 2019;33(2):428‒32. link1
[ 3 ] Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, et al. Deep learning in medical ultrasound analysis: a review. Engineering. 2019;5(2):261‒75. link1
[ 4 ] Rong G, Mendez A, Bou Assi E, Zhao B, Sawan M. Artificial intelligence in healthcare: review and prediction case studies. Engineering. 2020;6(3):291‒301. link1
[ 5 ] Sedghi S, Huang B. Real-time sssessment and diagnosis of process operating performance. Engineering 2017;3(2):214‒9. link1
[ 6 ] O’Neill S. Handheld ultrasound advances diagnosis. Engineering 2021;7(11):1505‒7. link1
[ 7 ] Cui Z, Yang B, Li RK. Application of biomaterials in cardiac repair and regeneration. Engineering 2016;2(1):141‒8. link1
[ 8 ] Li C, Pisignano D, Zhao Y, Xue J. Advances in medical applications of additive manufacturing. Engineering 2020;6(11):1222‒31. link1
[ 9 ] Ouyang D, He B, Ghorbani A, Yuan N, Ebinger J, Langlotz CP, et al. Video-based AI for beat-to-beat assessment of cardiac function. Nature 2020;580(7802):252‒6. link1
[10] Arnaout R, Curran L, Zhao Y, Levine JC, Chinn E, Moon-Grady AJ. An ensemble of neural networks provides expert-level prenatal detection of complex congenital heart disease. Nat Med 2021;27(5):882‒91. link1
[11] Bates S, Angelopoulos A, Lei L, Malik J, Jordan M. Distribution-free, risk-controlling prediction sets. J Assoc Comput Mach 2021;68(6):1‒34. link1
[12] Sadinle M, Lei J, Wasserman L. Least ambiguous set-valued classifiers with bounded error levels. J Am Stat Assoc 2019;114(525):223‒34. link1
[13] Affenit RN, Barns ER, Furst JD, Rasin A, Raicu DS. Building confidence and credibility into CAD with belief decision trees. In: Proceedings of the Medical Imaging 2017: Computer-Aided Diagnosis; 2017 Mar 3, Orlando, FL, USA. link1
[14] Scheffe H, Tukey JW. Non-parametric estimation. I. validation of order statistics. Ann Math Stat 1945;16(2):187‒92. link1
[15] McClure P, Kriegeskorte N. Robustly representing uncertainty in deep neural networks through sampling. In: Proceedings of the Second Workshop on Bayesian Deep Learning (NIPS 2017); LongBeach, CA, USA. link1
[16] Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on Machine Learning; 2016 Jun 16‒24; New York, NY, USA. JMLR: W&CP; 2016. p.1050‒9.
[17] Vishnevskiy V, Walheim J, Kozerke S. Deep variational network for rapid 4D flow MRI reconstruction. Nat Mach Intell 2020;2(4):228‒35. link1
[18] Piray P, Daw ND. A model for learning based on the joint estimation of stochasticity and volatility. Nat Commun 2021;12(1):6587. link1
[19] Lazar A, Lewis C, Fries P, Singer W, Nikolic D. Visual exposure enhances stimulus encoding and persistence in primary cortex. Proc Natl Acad Sci USA 2021;118(43):e2105276118. link1
[20] Chariker L, Shapley R, Hawken M, Young LS. A theory of direction selectivity for macaque primary visual cortex. Proc Natl Acad Sci USA 2021;118(32):e2105062118. link1
[21] Ferro D, van Kempen J, Boyd M, Panzeri S, Thiele A. Directed information exchange between cortical layers in macaque V1 and V4 and its modulation by selective attention. Proc Natl Acad Sci USA 2021;118(12):e2022097118. link1
[22] Gilbert CD, Sigman M. Brain states: top-down influences in sensory processing. Neuron 2007;54(5):677‒96. link1
[23] Hupé JM, James AC, Payne BR, Lomber SG, Girard P, Bullier J. Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 1998;394(6695):784‒7. link1
[24] Stollenga MF, Masci J, Gomez F, Schmidhuber J. Deep networks with internal selective attention through feedback connections. In: Proceedings of the 27th International Conference on Neural Information Processing Systems; 2014 Dec 8‒13; Montreal, Canada. Cambridge, MA: MIT Press; 2014. p. 3545‒53. link1
[25] Ozawa T, Ycu EA, Kumar A, Yeh LF, Ahmed T, Koivumaa J, et al. A feedback neural circuit for calibrating aversive memory strength. Nat Neurosci 2017;20(1):90‒7. link1
[26] Williams MA, Baker CI, Op de Beeck HP, Shim WM, Dang S, Triantafyllou C, et al. Feedback of visual object information to foveal retinotopic cortex. Nat Neurosci 2008;11(12):1439‒45. link1
[27] Cao C, Huang Y, Yang Y, Wang L, Wang Z, Tan T. Feedback convolutional neural network for visual localization and segmentation. IEEE Trans Pattern Anal Mach Intell 2019;41(7):1627‒40. link1
[28] Cao C, Liu X, Yang Y, Yu Y, Wang J, Wang Z, et al. Look and think twice: capturing top-down visual attention with feedback convolutional neural networks. In: Proceedings of the 2015 IEEE International Conference on Computer Vision; 2015 Dec 7‒13; Santiago, Chile. Washington, DC: IEEE Computer Society; 2015. p. 2956‒64. link1
[29] Haris M, Shakhnarovich G, Ukita N. Deep back-projection networks for super-resolution. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2018 Jun 18‒23; Salt Lake City, UT, USA. Washington, DC: IEEE; 2018. p. 1664‒73. link1
[30] Gal Y, Islam R, Ghahramani Z. Deep Bayesian active learning with image data. In: Proceedings of the 34th International Conference on Machine Learning; 2017 Aug 6‒11; Sydney, NSW, Australia. JMLR.org; 2017. p. 1183‒92.
[31] Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542(7639):115‒8. link1
[32] Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018;24(10):1559‒67. link1
[33] Ardila D, Kiraly AP, Bharadwaj S, Choi B, Reicher JJ, Peng L, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med 2019;25(6):954‒61. link1
[34] Arnaout R. Toward a clearer picture of health. Nat Med 2019;25(1):12. link1
[35] Howard A, Sandler M, Chen B, Wang W, Chen L, Tan M, et al. Searching for mobileNetV3. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision; 2019 Oct 27‒Nov 2; Seoul, Republic of Korea. Washington, DC: IEEE; 2019. link1
[36] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 2016 Jun 27‒30; Las Vegas, NV, USA. Washington, DC: IEEE; 2016. p. 770‒8. link1
[37] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 2017;39(6):1137‒49. link1
[38] Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition; 2017 Jul 21‒26; Honolulu, HI, USA. Washington, DC: IEEE; 2017. p.: 936‒44. link1
[39] Liu Z, Hu J, Weng L, Yang Y. Rotated region based CNN for ship detection. In: Proceedings of the 2017 IEEE International Conference on Image Processing; 2017 Sep 17‒20; Beijing, China. Washington, DC: IEEE; 2017.p. 900‒4. link1
[40] Frazer J, Notin P, Dias M, Gomez A, Min JK, Brock K, et al. Disease variant prediction with deep generative models of evolutionary data. Nature 2021;599(7883):91‒5. link1
[41] Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39(12):2481‒95. link1
[42] Krygier MC, LaBonte T, Martinez C, Norris C, Sharma K, Collins LN, et al. Quantifying the unknown impact of segmentation uncertainty on image-based simulations. Nat Commun 2021;12(1):5414. link1
[43] Lemhadri I, Ruan F, Abraham L, Tibshirani R. LassoNet: neural networks with feature sparsity. J Mach Learn Res 2021;22:1‒29.
[44] Suway SB, Schwartz AB. Activity in primary motor cortex related to visual feedback. Cell Rep 2019;29(12):3872‒84.e4. link1
[45] Marques T, Nguyen J, Fioreze G, Petreanu L. The functional organization of cortical feedback inputs to primary visual cortex. Nat Neurosci 2018;21(5):757‒64. link1
京公网安备 11010502051620号
