期刊首页 优先出版 当期阅读 过刊浏览 作者中心 关于期刊 English

《工程(英文)》 >> 2021年 第7卷 第6期 doi: 10.1016/j.eng.2020.03.016

基于迁移学习与多视角感兴趣点的膝关节运动追踪网络

a Shanghai Key Laboratory of Orthopaedic Implants & Clinical Translational R&D Center of 3D Printing Technology, Department of Orthopaedic Surgery, Shanghai Ninth People’s
Hospital, Shanghai Jiao Tong University School of Medicine; School of Biomedical Engineering & Med-X Research Institute, Shanghai Jiao Tong University, Shanghai 200030, China
b SenseTime Research, Shanghai 200233, China
c Engineering Research Center of Digital Medicine and Clinical Translation, Ministry of Education, Shanghai 200030, China
d Department of Orthopaedics, New Jersey Medical School, Rutgers University, Newark, NJ 07103, USA
e Department of Orthopaedics, Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, Shanghai 200233, China

收稿日期: 2020-02-17 修回日期: 2020-03-22 录用日期: 2020-03-30 发布日期: 2020-07-22

下一篇 上一篇

摘要

近年来,深度学习为一种基于二维(2D)—三维(3D)配准技术以测量人体膝关节运动的方法,该方法提供了快速完成配准并增加捕捉范围的可能性。但这类方法受限于大量的数据需求,因此,我们提出了一种基于特征的迁移学习法,用于提取荧光透视影像的特征。通过三个受试者以及不到100对荧光透视影像,我们获得了40%的平均配准成功率。本研究提出的基于学习的配准方法,可在荧光透视影像数量有限时使用。

图片

图1

图2

图3

图4

图5

图6

图7

图8

参考文献

[ 1 ] KernkampWA,Wang C, Li C, Hu H, Van Arkel ERA, Nelissen RGHH, et al. Themedial patellofemoral ligament is a dynamic and anisometric structure: an in vivo study on length changes and isometry. Am J Sports Med 2019;47:1645–53. 链接1

[ 2 ] Li JS, Tsai TY, Clancy MM, Li G, Lewis CL, Felson DT. Weight loss changed gait kinematics in individuals with obesity and knee pain. Gait Posture 2019;68:461–5. 链接1

[ 3 ] Li G, Li JS, Torriani M, Hosseini A. Short-term contact kinematic changes and longer-term biochemical changes in the cartilage after ACL reconstruction: a pilot study. Ann Biomed Eng 2018;46:1797–805. 链接1

[ 4 ] Tsai TY, Liow MHL, Li G, Arauz P, Peng Y, Klemt C, et al. Bi-cruciate retaining total knee arthroplasty does not restore native tibiofemoral articular contact kinematics during gait. J Orthop Res 2019;37(9):1929–37. 链接1

[ 5 ] Perriman A, Leahy E, Semciw AI. The effect of open- versus closed-kineticchain exercises on anterior tibial laxity, strength, and function following anterior cruciate ligament reconstruction: a systematic review and metaanalysis. J Orthop Sports Phys Ther 2018;48(7):552–66. 链接1

[ 6 ] Tsai TY, Lu TW, Kuo MY, Lin CC. Effects of soft tissue artifacts on the calculated kinematics and kinetics of the knee during stair-ascent. J Biomech 2011;44 (6):1182–8. 链接1

[ 7 ] Lu TW, O’Connor JJ. Bone position estimation from skin marker co-ordinates using global optimisation with joint constraints. J Biomech 1999;32 (2):129–34. 链接1

[ 8 ] Alexander EJ, Andriacchi TP. Correcting for deformation in skin-based marker systems. J Biomech 2001;34(3):355–61. 链接1

[ 9 ] Lucchetti L, Cappozzo A, Cappello A, Della CU. Skin movement artefact assessment and compensation in the estimation of knee-joint kinematics. J Biomech 1998;31(11):977–84. 链接1

[10] Leardini A, Chiari L, Della Croce U, Cappozzo A. Human movement analysis using stereophotogrammetry: part 3. Soft tissue artifact assessment and compensation. Gait Posture 2005;21(2):212–25. 链接1

[11] Borotikar B, Lempereur M, Lelievre M, Burdin V, Ben Salem D, Brochard S. Dynamic MRI to quantify musculoskeletal motion: a systematic review of concurrent validity and reliability, and perspectives for evaluation of musculoskeletal disorders. PLoS ONE 2017;12(12):e0189587. 链接1

[12] Tanaka MJ, Elias JJ, Williams AA, Demehri S, Cosgarea AJ. Characterization of patellar maltracking using dynamic kinematic CT imaging in patients with patellar instability. Knee Surg Sports Traumatol Arthrosc 2016;24:3634–41. 链接1

[13] Patel VV, Hall K, Ries M, Lotz J, Ozhinsky E, Lindsey C, et al. A threedimensional MRI analysis of knee kinematics. J Orthop Res 2004;22 (2):283–92. 链接1

[14] Muhle C, Brossmann J, Heller M. Kinematic CT and MR imaging of the patellofemoral joint. Eur Radiol 1999;9:508–18. 链接1

[15] Li G, Wuerz TH, DeFrate LE. Feasibility of using orthogonal fluoroscopic images to measure in vivo joint kinematics. J Biomech Eng 2004;126(2):314–8. 链接1

[16] Acker S, Li R, Murray H, St JP, Banks S, Mu S, et al. Accuracy of single-plane fluoroscopy in determining relative position and orientation of total knee replacement components. J Biomech 2011;44(4):784–7. 链接1

[17] Kim J, Li S, Pradhan D, Hammoud R, Chen Q, Yin FF, et al. Comparison of similarity measures for rigid-body CT/dual X-ray image registrations. Technol Cancer Res Treat 2007;6(4):337–46. 链接1

[18] Van der Bom IMJ, Klein S, Staring M, Homan R, Bartels LW, Pluim JPW. Evaluation of optimization methods for intensity-based 2D–3D registration in X-ray guided interventions. In: Proceedings of SPIE Medical Imaging; 2011 Feb 12–17; Orlando, FL, USA; 2011.

[19] Alam F, Rahman SU, Khusro S, Ullah S, Khalil A. Evaluation of medical image registration techniques based on nature and domain of the transformation. J Med Imaging Radiat Sci 2016;47(2):178–93. 链接1

[20] Ghafurian S, Hacihaliloglu I, Metaxas DN, Tan V, Li K. A computationally efficient 3D/2D registration method based on image gradient direction probability density function. Neurocomputing 2017;229:100–8. 链接1

[21] Zheng J, Miao S, Wang ZJ, Liao R. Pairwise domain adaptation module for CNNbased 2-D/3-D registration. J Med Imaging 2018;5(2):21204. 链接1

[22] Zheng J, Miao S, Liao R. Learning CNNs with pairwise domain adaption for realtime 6DoF ultrasound transducer detection and tracking from X-ray images. In: Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention; 2017 Sep 11–13; Quebec City, QC, Canada; 2017.

[23] Markelj P, Tomazˇevicˇ D, Likar B, Pernuš F. A review of 3D/2D registration methods for image guided interventions. Med Image Anal 2012;16(3):642–61. 链接1

[24] Haskins G, Kruger U, Yan P. Deep learning in medical image registration: a survey. Mach Vis Appl 2020;31(1):8. 链接1

[25] Duan Q, Wang G, Wang R, Fu C, Li X, Gong M, et al. SenseCare: a research platform for medical image informatics and interactive 3D visualization 2020. arXiv:2004.07031v1.

[26] Toth D, Miao S, Kurzendorfer T, Rinaldi CA, Liao R, Mansi T, et al. 3D/2D modelto-image registration by imitation learning for cardiac procedures. Int J Comput Assisted Radiol Surg 2018;13:1141–9. 链接1

[27] Liao H, Lin WA, Zhang J, Zhang J, Luo J, Zhou SK. Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the 2019 IEEE Conference on Computer Vision Pattern Recognition; 2019 Jun 15–20; Long Beach, CA, USA; 2019. p. 12638–47.

[28] Miao S, Piat S, Fischer P, Tuysuzoglu A, Mewes P, Mansi T, et al. Dilated FCN for multi-agent 2D/3D medical image registration. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence; 2018 Feb 2–7; New Orleans, LA, USA; 2018.

[29] Wallace WA, Johnson F. Detection and correction of geometrical distortion in X-ray fluoroscopic images. J Biomech 1981;14(2):123–5. 链接1

[30] Cerciello T, Bifulco P, Cesarelli M, Fratini A. A comparison of denoising methods for X-ray fluoroscopic images. Biomed Signal Process Control 2012;7 (6):550–9. 链接1

[31] Moore CS, Liney GP, Beavis AW, Saunderson JR. A method to produce and validate a digitally reconstructed radiograph-based computer simulation for optimisation of chest radiographs acquired with a computed radiography imaging system. Br J Radiol 2011;84(1006):890–902. 链接1

[32] Moore CS, Wood TJ, Saunderson JR, Beavis AW. A method to incorporate the effect of beam quality on image noise in a digitally reconstructed radiograph (DRR) based computer simulation for optimisation of digital radiography. Phys Med Biol 2017;62(18):7379–93. 链接1

[33] Miao S, Wang ZJ, Zheng Y, Liao R. Real-time 2D/3D registration via CNN regression. In: Proceedings of the 2016 IEEE 13th International Symposium Biomedical Imaging; 2016 Apr 13–16; Prague, Czech Republic; 2016. p. 1430–4.

[34] Schaffert R, Wang J, Fischer P, Borsdorf A, Maier A. Metric-driven learning of correspondence weighting for 2-D/3-D image registration. In: Proceedings of the German Conference Pattern Recognition; 2018 Oct 9–12; Stuttgart, Germany; 2018. p. 140–52.

[35] Pan J, Min Z, Zhang A, Ma H, Meng MQH. Multiview global 2D–3D registration based on branch and bound algorithm. In: Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics; 2019 Dec 6–8; Dali, China; 2019. p. 3082–7.

[36] Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. J Big Data 2016;3:9. 链接1

[37] Csurka G. A comprehensive survey on domain adaptation for visual applications. In: Csurka G, editor. Domain adaptation in computer vision applications. Cham: Springer International Publishing AG; 2017. p. 1–35. 链接1

[38] Zhang Y, Miao S, Mansi T, Liao R. Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation. In: Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention; 2018 Sep 16–20; Granada, Spain; 2018. p. 599–607.

[39] Hu J, Sun S, Yang X, Zhou S, Wang X, Fu Y, et al. Towards accurate and robust multi-modal medical image registration using contrastive metric learning. IEEE Access 2019;7:132816–27. 链接1

[40] Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, et al. Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPUacceleration. IEEE Trans Med Imaging 2012;31(4):948–62. 链接1

[41] Li T, Zhu H. Research on digitally reconstructed radiograph algorithm improvement based on computed tomography image. In: Proceedings of the 2018 11th International Congress on Image Signal Processing, Biomedical Engineering Informatics; 2018 Oct 13–15; Beijing, China; 2018. p. 1–5.

[42] Fischer B, Modersitzki J. Ill-posed medicine—an introduction to image registration. Inverse Probl 2008;24:034008. 链接1

[43] Hartley R, Zisserman A. Multiple view geometry in computer vision. Cambridge: Cambridge University Press; 2003. 链接1

[44] Seber GAF. Multivariate observations. New York: John Wiley & Sons; 1983. 链接1

[45] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv:1409.1556.

[46] Baltzopoulos V. A videofluoroscopy method for optical distortion correction and measurement of knee-joint kinematics. Clin Biomech 1995;10(2):85–92. 链接1

[47] Tsai TY, Dimitriou D, Li G, Kwon YM. Does total hip arthroplasty restore native hip anatomy? Three-dimensional reconstruction analysis. Int Orthop 2014;38:1577–83. 链接1

[48] Van de Kraats EB, Penney GP, Tomazevic D, Van Walsum T, Niessen WJ. Standardized evaluation methodology for 2-D–3-D registration. IEEE Trans Med Imaging 2005;24(9):1177–89. 链接1

相关研究