期刊首页 优先出版 当期阅读 过刊浏览 作者中心 关于期刊 English

《工程(英文)》 >> 2020年 第6卷 第1期 doi: 10.1016/j.eng.2019.11.003

户外空中双机械手抓取设计和视觉伺服

Robotics, Vision, and Control Group, School of Engineering, University of Seville, Seville 41092, Spain

收稿日期 :2018-10-30 修回日期 :2019-03-10 录用日期 : 2019-03-26 发布日期 :2019-11-13

下一篇 上一篇

摘要

本文介绍了一种配备有RGB-D摄像机的使用带有双机械手的无人飞行器(unmanned aerial vehicle, UAV)抓取已知物体的系统。空中操纵仍然是一项极具挑战性的任务。本文主要从三个方面对这一任务进行了评价:目标检测与姿态估计、抓取设计、飞行中的抓取动作。人工神经网络(artificial neural network, ANN)首先被用来获得有关物体位置的线索。接下来,使用对齐算法获取对象的六维(six-dimensional, 6D)姿态,并使用扩展的卡尔曼滤波器进行滤波。然后,使用物体的三维(three-dimensional, 3D)模型来估计空中机械手可实现良好抓取的排列清单。检测算法的结果(即对象的姿态)用于更新手臂朝向对象的轨迹。如果由于无人机的振荡而无法达到目标姿态,则算法将切换到下一个可行的抓取。本文介绍了总体方法,给出了每个模块的仿真实验结果和实际实验结果,并提供了视频演示结果。

图片

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

图16

参考文献

[1]  Pouliot N, Richard PL, Montambault S. Linescout technology opens the way to robotic inspection and maintenance of high-voltage power lines. IEEE Power Energy Technol Syst J 2015;2(1):1–11. 链接1

[2]  Matikainen L, Lehtomäki M, Ahokas E, Hyyppä J, Karjalainen M, Jaakkola A, et al. Remote sensing methods for power line corridor surveys. ISPRS J Photogramm Remote Sens 2016;119:10–31. 链接1

[3]  Wang L, Zhang Z. Automatic detection of wind turbine blade surface cracks based on UAV-taken images. IEEE Trans Ind Electron 2017;64(9):7293–303. 链接1

[4]  Almadhoun R, Taha T, Seneviratne L, Dias J, Cai G. A survey on inspecting structures using robotic systems. Int J Adv Robot Syst 2016;13(6): 1729881416663664. 链接1

[5]  Reinoso JF, Gonçalves JE, Pereira C, Bleninger T. Cartography for civil engineering projects: photogrammetry supported by unmanned aerial vehicles. Iran J Sci Technol Trans Civ Eng 2018;42(1):91–6. 链接1

[6]  Ruggiero F, Lippiello V, Ollero A. Aerial manipulation: a literature review. IEEE Robot Automat Lett 2018;3(3):1957–64. 链接1

[7]  Gevers T, Smeulders AWM. Color-based object recognition. Pattern Recognit 1999;32(3):453–64. 链接1

[8]  Bruce J, Balch T, Veloso M. Fast and inexpensive color image segmentation for interactive robots. In: Proceedings of 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2000 Oct 31–Nov 5; Takamatsu, Japan. New York: IEEE; 2000. p. 2061–6. 链接1

[9]  Lowe DG. Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer Vision; 1999 Sep 20–25; Corfu, Greece. New York: IEEE; 1999. p. 1150–7. 链接1

[10]  Zhou H, Yuan Y, Shi C. Object tracking using SIFT features and mean shift. Comput Vis Image Underst 2009;113(3):345–52. 链接1

[11]  Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, et al. SSD: single shot multibox detector. 2015. arXiv:1512.02325.

[12]  Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition; 2014 Jun 23–28; Washington, DC, USA. New York: IEEE; 2014. p. 580–7. 链接1

[13]  Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. 2015. arXiv:1506.01497.

[14]  He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. 2017, arXiv:1703.068702017. 链接1

[15]  Redmon J, Farhadi A. YOLOv3: an incremental improvement. 2018. arXiv:1804.02767.

[16]  Xiang Y, Schmidt T, Narayanan V, Fox D. PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. 2018. arXiv:1711.00199.

[17]  Rai A, Patchaikani PK, Agarwal M, Gupta R, Behera L. Grasping region identification in novel objects using Microsoft kinect. In: Proceedings of 19th International Conference on Neural Information Processing; 2012 Nov 12–15; Doha, Qatar. Berlin: Springer; 2012. p. 172–9. 链接1

[18]  Lysenkov I, Eruhimov V, Bradski G. Recognition and pose estimation of rigid transparent objects with a Kinect sensor. In: Roy N, Newman P, Srinivasa S, editors. Robotics: science and systems VIII. Cambridge: MIT Press; 2013. p. 273–80. 链接1

[19]  Gordon I, Lowe DG. What and where: 3D object recognition with accurate pose. In: Ponce J, Hebert M, Schmid C, Zisserman A, editors. Toward categorylevel object recognition. Berlin: Springer; 2006. p. 67–82. 链接1

[20]  Collet A, Berenson D, Srinivasa SS, Ferguson D. Object recognition and full pose registration from a single image for robotic manipulation. In: Proceedings of the 2009 IEEE International Conference on Robotics and Automation; 2009 May 12–17; Kobe, Japan. New York: IEEE; 2009. p. 48–55. 链接1

[21]  Ramon Soria P, Arrue BC, Ollero A. Detection, location and grasping objects using a stereo sensor on UAV in outdoor environments. Sensors 2017;17 (1):103. 链接1

[22]  Marchand E, Bouthemy P, Chaumette F, Moreau V. Robust real-time visual tracking using a 2D-3D model-based approach. In: Proceedings of the 7th IEEE International Conference on Computer Vision; 1999 Sep 20–27; Kerkyra, Greece. New York: IEEE; 1999. p. 262–8. 链接1

[23]  Miller AT, Knoop S, Christensen HI, Allen PK. Automatic grasp planning using shape primitives. In: Proceedings of 2003 IEEE International Conference on Robotics and Automation; 2003 Sep 14–19; Taipei, China. New York: IEEE; 2003. p. 1824–9. 链接1

[24]  Xue Z, Xia S, Dillmann R. An efficient grasp planning algorithm based on decomposition of grasp regions. In: Proceedings of 12th IEEE-RAS International Conference on Humanoid Robots; 2012 Nov 29–Dec 1; Osaka, Japan. New York: IEEE; 2012. p. 686–91. 链接1

[25]  Song D, Huebner K, Kyrki V, Kragic D. Learning task constraints for robot grasping using graphical models. In: Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2010 Oct 18– 22; Taipei, China. New York: IEEE; 2010. p. 1579–85. 链接1

[26]  Saxena A, Driemeyer J, Ng AY. Robotic grasping of novel objects using vision. Int J Robot Res 2008;27(2):157–73. 链接1

[27]  Morrison D, Corke P, Leitner J. Closing the loop for robotic grasping: a realtime, generative grasp synthesis approach. 2018. arXiv:1804.05172.

[28]  Baier-Lowenstein T, Zhang J. Learning to grasp everyday objects using reinforcement-learning with automatic value cut-off. In: Proceedings of 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2007 Oct 29–Nov 2; San Diego, CA, USA. New York: IEEE; 2007. p. 1551–6. 链接1

[29]  Dragiev S, Toussaint M, Gienger M. Gaussian process implicit surfaces for shape estimation and grasping. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China. New York: IEEE; 2011. p. 2845–50. 链接1

[30]  Björkman M, Bekiroglu Y, Högman V, Kragic D. Enhancing visual perception of shape through tactile glances. In: Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2013 Nov 3–7; Tokyo, Japan. New York: IEEE; 2013. p. 3180–6. 链接1

[31]  Martens W, Poffet Y, Ramón Soria P, Fitch R, Sukkarieh S. Geometric priors for Gaussian process implicit surfaces. IEEE Robot Autom Lett 2017;2(2):373–80. 链接1

[32]  Roa MA, Suárez R. Grasp quality measures: review and performance. Autono Robots 2015;38(1):65–88. 链接1

[33]  Marturi N, Kopicki M, Rastegarpanah A, Rajasekaran V, Adjigble M, Stolkin R, et al. Dynamic grasp and trajectory planning for moving objects. Autono Robots 2019;43(5):1241–56. 链接1

[34]  Ruggiero F, Trujillo MA, Cano R, Ascorbe H, Viguria A, Peréz C, et al. A multilayer control for multirotor UAVs equipped with a servo robot arm. In: Proceedings of 2015 IEEE International Conference on Robotics and Automation; 2015 May 26–30; Seattle, WA, USA. New York: IEEE; 2015. p. 4014–20. 链接1

[35]  Suarez A, Jimenez-Cano AE, Vega VM, Heredia G, Rodríguez-Castaño A, Ollero A. Lightweight and human-size dual arm aerial manipulator. In: Proceedings of 2017 International Conference on Unmanned Aircraft Systems; 2017 Jun 13– 16; Miami, FL, USA. New York: IEEE; 2017. p. 1778–84. 链接1

[36]  Hutchinson S, Hager GD, Corke PI. A tutorial on visual servo control. IEEE Trans Robot Autom 1996;12(5):651–70. 链接1

[37]  Chaumette F, Hutchinson S. Visual servo control. I. basic approaches. IEEE Robot Autom Mag 2006;13(4):82–90. 链接1

[38]  Kim S, Seo H, Choi S, Kim HJ. Vision-guided aerial manipulation using a multirotor with a robotic arm. IEEE/ASME Trans Mechatron 2016;21 (4):1912–23. 链接1

[39]  Prada Delgado J, Ramon Soria P, Arrue BC, Ollero A. Bridge mapping for inspection using an UAV assisted by a total station. In: Ollero A, Sanfeliu A, Montano L, Lau N, Cardeira C, editors. Robot 2017: third Iberian robotics conference. Cham: Springer; 2018. p. 309–19. 链接1

[40]  Laiacker M, Huber F, Kondak K. High accuracy visual servoing for aerial manipulation using a 7 degrees of freedom industrial manipulator. In: Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2016 Oct 9–14; Daejeon, South Korea. New York: IEEE; 2016. p. 1631–6. 链接1

[41]  Korpela C, Orsag M, Oh P. Towards valve turning using a dual-arm aerial manipulator. In: Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2014 Sep 14–18; Chicago, IL, USA. New York: IEEE; 2014. p. 3411–6. 链接1

[42]  Suarez A, Heredia G, Ollero A. Design of an anthropomorphic, compliant, and lightweight dual arm for aerial manipulation. IEEE Access 2018;6:29173–89. 链接1

[43]  Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. 2016. arXiv:1603.04467.

[44]  Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, et al. Caffe: convolutional architecture for fast feature embedding. 2014. arXiv:1408.5093.

[45]  Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. In: Proceedings of Third International Conference on 3-D Digital Imaging and Modeling; 2001 May 28–Jun 1; Quebec City, QC, Canada. New York: IEEE; 2001. p. 145–52. 链接1

[46]  Men H, Gebre B, Pochiraju K. Color point cloud registration with 4D ICP algorithm. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China. New York: IEEE; 2011. p. 1511–6. 链接1

[47]  Liu W, Hu J, Fang Y, Shao Q, Zheng K, Zhu GN. Real time pose estimation based on extended Kalman filter for binocular camera. In: Proceedings of 2016 AsiaPacific Conference on Intelligent Robot Systems; 2016 Jul 20–22; Tokyo, Japan. New York: IEEE; 2016. p. 142–6. 链接1

[48]  Hajimolahoseini H, Amirfattahi R, Khorshidi S. Real-time pose estimation and tracking of rigid objects in 3D space using extended Kalman filter. In: Proceedings of 2014 22nd Iranian Conference on Electrical Engineering; 2014 May 20–22; Tehran, Iran. New York: IEEE; 2014. p. 1545–9. 链接1

[49]  Diankov R. Automated construction of robotic manipulation programs [dessertation]. Pittsburgh: Carnegie Mellon University; 2010. 链接1

[50]  Barrientos A, Peñín LF, Balaguer C, Aracil R. Fundaments in Robotics. New York: McGraw Hill; 1997. 链接1

[51]  Goldenberg A, Benhabib B, Fenton R. A complete generalized solution to the inverse kinematics of robots. IEEE J Robot Autom 1985;1(1):14–20. 链接1

[52]  Courty N, Arnaud E. Inverse kinematics using sequential monte carlo methods. In: Perales FJ, Fisher RB, editors. Articulated motion and deformable objects. Berlin: Springer; 2008. p. 1–10. 链接1

[53]  Wampler CW, Leifer LJ. Applications of damped least-squares methods to resolved-rate and resolved-acceleration control of manipulators. J Dyn Sys Meas Control 1988;110(1):31–8. 链接1

[54]  Buss SR. Introduction to inverse kinematics with Jacobian transpose, pseudoinverse and damped least squares methods. IEEE J Robot Autom 2004;17(1–19):16. 链接1

[55]  Ghosh M, Amato NM, Lu Y, Lien JM. Fast approximate convex decomposition using relative concavity. Comput Aided Des 2013;45(2):494–504. 链接1

[56]  Mamou K, Ghorbel F. A simple and efficient approach for 3D mesh approximate convex decomposition. In: Proceedings of 2009 16th IEEE International Conference on Image Processing; 2009 Nov 7–10; Cairo, Egypt. New York: IEEE; 2009. p. 3501–4. 链接1

[57]  Diankov R, Kuffner J. OpenRAVE: a planning architecture for autonomous robotics. Pittsburgh: Robotics Institute; 2008. 链接1

[58]  Quigley M, Gerkey B, Conley K, Faust J, Foote T, Leibs J, et al. ROS: an opensource robot operating system. In: Proceeding of IICRA Workshop on Open Source Software; 2009 May 12–17; Kobe, Japan. New York: IEEE; 2009. p. 1–6. 链接1

相关研究