PDF(3258 KB)
户外空中双机械手抓取设计和视觉伺服
Pablo Ramon-Soria, Begoña C. Arrue, Anibal Ollero
工程（英文） ›› 2020, Vol. 6 ›› Issue (1) : 77-88.
PDF(3258 KB)
PDF(3258 KB)
户外空中双机械手抓取设计和视觉伺服
Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator
本文介绍了一种配备有RGB-D摄像机的使用带有双机械手的无人飞行器（unmanned aerial vehicle, UAV）抓取已知物体的系统。空中操纵仍然是一项极具挑战性的任务。本文主要从三个方面对这一任务进行了评价：目标检测与姿态估计、抓取设计、飞行中的抓取动作。人工神经网络（artificial neural network, ANN）首先被用来获得有关物体位置的线索。接下来，使用对齐算法获取对象的六维（six-dimensional, 6D）姿态，并使用扩展的卡尔曼滤波器进行滤波。然后，使用物体的三维（three-dimensional, 3D）模型来估计空中机械手可实现良好抓取的排列清单。检测算法的结果（即对象的姿态）用于更新手臂朝向对象的轨迹。如果由于无人机的振荡而无法达到目标姿态，则算法将切换到下一个可行的抓取。本文介绍了总体方法，给出了每个模块的仿真实验结果和实际实验结果，并提供了视频演示结果。
This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object's position. Next, an alignment algorithm is used to obtain the object's six-dimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object's pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV's oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.
Aerial manipulation / Grasp planning / Visual servoing
| [1] |
Pouliot N, Richard PL, Montambault S. Linescout technology opens the way to robotic inspection and maintenance of high-voltage power lines. IEEE Power Energy Technol Syst J 2015;2(1):1–11.
|
| [2] |
Matikainen L, Lehtomäki M, Ahokas E, Hyyppä J, Karjalainen M, Jaakkola A, et al. Remote sensing methods for power line corridor surveys. ISPRS J Photogramm Remote Sens 2016;119:10–31.
|
| [3] |
Wang L, Zhang Z. Automatic detection of wind turbine blade surface cracks based on UAV-taken images. IEEE Trans Ind Electron 2017;64(9):7293–303.
|
| [4] |
Almadhoun R, Taha T, Seneviratne L, Dias J, Cai G. A survey on inspecting structures using robotic systems. Int J Adv Robot Syst 2016;13(6): 1729881416663664.
|
| [5] |
Reinoso JF, Gonçalves JE, Pereira C, Bleninger T. Cartography for civil engineering projects: photogrammetry supported by unmanned aerial vehicles. Iran J Sci Technol Trans Civ Eng 2018;42(1):91–6.
|
| [6] |
Ruggiero F, Lippiello V, Ollero A. Aerial manipulation: a literature review. IEEE Robot Automat Lett 2018;3(3):1957–64.
|
| [7] |
Gevers T, Smeulders AWM. Color-based object recognition. Pattern Recognit 1999;32(3):453–64.
|
| [8] |
Bruce J, Balch T, Veloso M. Fast and inexpensive color image segmentation for interactive robots. In: Proceedings of 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2000 Oct 31–Nov 5; Takamatsu, Japan. New York: IEEE; 2000. p. 2061–6.
|
| [9] |
Lowe DG. Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer Vision; 1999 Sep 20–25; Corfu, Greece. New York: IEEE; 1999. p. 1150–7.
|
| [10] |
Zhou H, Yuan Y, Shi C. Object tracking using SIFT features and mean shift. Comput Vis Image Underst 2009;113(3):345–52.
|
| [11] |
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, et al. SSD: single shot multibox detector. 2015. arXiv:1512.02325.
|
| [12] |
Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition; 2014 Jun 23–28; Washington, DC, USA. New York: IEEE; 2014. p. 580–7.
|
| [13] |
Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. 2015. arXiv:1506.01497.
|
| [14] |
He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. 2017, arXiv:1703.068702017.
|
| [15] |
Redmon J, Farhadi A. YOLOv3: an incremental improvement. 2018. arXiv:1804.02767.
|
| [16] |
Xiang Y, Schmidt T, Narayanan V, Fox D. PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. 2018. arXiv:1711.00199.
|
| [17] |
Rai A, Patchaikani PK, Agarwal M, Gupta R, Behera L. Grasping region identification in novel objects using Microsoft kinect. In: Proceedings of 19th International Conference on Neural Information Processing; 2012 Nov 12–15; Doha, Qatar. Berlin: Springer; 2012. p. 172–9.
|
| [18] |
Lysenkov I, Eruhimov V, Bradski G. Recognition and pose estimation of rigid transparent objects with a Kinect sensor. In: Roy N, Newman P, Srinivasa S, editors. Robotics: science and systems VIII. Cambridge: MIT Press; 2013. p. 273–80.
|
| [19] |
Gordon I, Lowe DG. What and where: 3D object recognition with accurate pose. In: Ponce J, Hebert M, Schmid C, Zisserman A, editors. Toward categorylevel object recognition. Berlin: Springer; 2006. p. 67–82.
|
| [20] |
Collet A, Berenson D, Srinivasa SS, Ferguson D. Object recognition and full pose registration from a single image for robotic manipulation. In: Proceedings of the 2009 IEEE International Conference on Robotics and Automation; 2009 May 12–17; Kobe, Japan. New York: IEEE; 2009. p. 48–55.
|
| [21] |
Ramon Soria P, Arrue BC, Ollero A. Detection, location and grasping objects using a stereo sensor on UAV in outdoor environments. Sensors 2017;17 (1):103.
|
| [22] |
Marchand E, Bouthemy P, Chaumette F, Moreau V. Robust real-time visual tracking using a 2D-3D model-based approach. In: Proceedings of the 7th IEEE International Conference on Computer Vision; 1999 Sep 20–27; Kerkyra, Greece. New York: IEEE; 1999. p. 262–8.
|
| [23] |
Miller AT, Knoop S, Christensen HI, Allen PK. Automatic grasp planning using shape primitives. In: Proceedings of 2003 IEEE International Conference on Robotics and Automation; 2003 Sep 14–19; Taipei, China. New York: IEEE; 2003. p. 1824–9.
|
| [24] |
Xue Z, Xia S, Dillmann R. An efficient grasp planning algorithm based on decomposition of grasp regions. In: Proceedings of 12th IEEE-RAS International Conference on Humanoid Robots; 2012 Nov 29–Dec 1; Osaka, Japan. New York: IEEE; 2012. p. 686–91.
|
| [25] |
Song D, Huebner K, Kyrki V, Kragic D. Learning task constraints for robot grasping using graphical models. In: Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2010 Oct 18– 22; Taipei, China. New York: IEEE; 2010. p. 1579–85.
|
| [26] |
Saxena A, Driemeyer J, Ng AY. Robotic grasping of novel objects using vision. Int J Robot Res 2008;27(2):157–73.
|
| [27] |
Morrison D, Corke P, Leitner J. Closing the loop for robotic grasping: a realtime, generative grasp synthesis approach. 2018. arXiv:1804.05172.
|
| [28] |
Baier-Lowenstein T, Zhang J. Learning to grasp everyday objects using reinforcement-learning with automatic value cut-off. In: Proceedings of 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2007 Oct 29–Nov 2; San Diego, CA, USA. New York: IEEE; 2007. p. 1551–6.
|
| [29] |
Dragiev S, Toussaint M, Gienger M. Gaussian process implicit surfaces for shape estimation and grasping. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China. New York: IEEE; 2011. p. 2845–50.
|
| [30] |
Björkman M, Bekiroglu Y, Högman V, Kragic D. Enhancing visual perception of shape through tactile glances. In: Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2013 Nov 3–7; Tokyo, Japan. New York: IEEE; 2013. p. 3180–6.
|
| [31] |
Martens W, Poffet Y, Ramón Soria P, Fitch R, Sukkarieh S. Geometric priors for Gaussian process implicit surfaces. IEEE Robot Autom Lett 2017;2(2):373–80.
|
| [32] |
Roa MA, Suárez R. Grasp quality measures: review and performance. Autono Robots 2015;38(1):65–88.
|
| [33] |
Marturi N, Kopicki M, Rastegarpanah A, Rajasekaran V, Adjigble M, Stolkin R, et al. Dynamic grasp and trajectory planning for moving objects. Autono Robots 2019;43(5):1241–56.
|
| [34] |
Ruggiero F, Trujillo MA, Cano R, Ascorbe H, Viguria A, Peréz C, et al. A multilayer control for multirotor UAVs equipped with a servo robot arm. In: Proceedings of 2015 IEEE International Conference on Robotics and Automation; 2015 May 26–30; Seattle, WA, USA. New York: IEEE; 2015. p. 4014–20.
|
| [35] |
Suarez A, Jimenez-Cano AE, Vega VM, Heredia G, Rodríguez-Castaño A, Ollero A. Lightweight and human-size dual arm aerial manipulator. In: Proceedings of 2017 International Conference on Unmanned Aircraft Systems; 2017 Jun 13– 16; Miami, FL, USA. New York: IEEE; 2017. p. 1778–84.
|
| [36] |
Hutchinson S, Hager GD, Corke PI. A tutorial on visual servo control. IEEE Trans Robot Autom 1996;12(5):651–70.
|
| [37] |
Chaumette F, Hutchinson S. Visual servo control. I. basic approaches. IEEE Robot Autom Mag 2006;13(4):82–90.
|
| [38] |
Kim S, Seo H, Choi S, Kim HJ. Vision-guided aerial manipulation using a multirotor with a robotic arm. IEEE/ASME Trans Mechatron 2016;21 (4):1912–23.
|
| [39] |
Prada Delgado J, Ramon Soria P, Arrue BC, Ollero A. Bridge mapping for inspection using an UAV assisted by a total station. In: Ollero A, Sanfeliu A, Montano L, Lau N, Cardeira C, editors. Robot 2017: third Iberian robotics conference. Cham: Springer; 2018. p. 309–19.
|
| [40] |
Laiacker M, Huber F, Kondak K. High accuracy visual servoing for aerial manipulation using a 7 degrees of freedom industrial manipulator. In: Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2016 Oct 9–14; Daejeon, South Korea. New York: IEEE; 2016. p. 1631–6.
|
| [41] |
Korpela C, Orsag M, Oh P. Towards valve turning using a dual-arm aerial manipulator. In: Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2014 Sep 14–18; Chicago, IL, USA. New York: IEEE; 2014. p. 3411–6.
|
| [42] |
Suarez A, Heredia G, Ollero A. Design of an anthropomorphic, compliant, and lightweight dual arm for aerial manipulation. IEEE Access 2018;6:29173–89.
|
| [43] |
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. 2016. arXiv:1603.04467.
|
| [44] |
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, et al. Caffe: convolutional architecture for fast feature embedding. 2014. arXiv:1408.5093.
|
| [45] |
Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. In: Proceedings of Third International Conference on 3-D Digital Imaging and Modeling; 2001 May 28–Jun 1; Quebec City, QC, Canada. New York: IEEE; 2001. p. 145–52.
|
| [46] |
Men H, Gebre B, Pochiraju K. Color point cloud registration with 4D ICP algorithm. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China. New York: IEEE; 2011. p. 1511–6.
|
| [47] |
Liu W, Hu J, Fang Y, Shao Q, Zheng K, Zhu GN. Real time pose estimation based on extended Kalman filter for binocular camera. In: Proceedings of 2016 AsiaPacific Conference on Intelligent Robot Systems; 2016 Jul 20–22; Tokyo, Japan. New York: IEEE; 2016. p. 142–6.
|
| [48] |
Hajimolahoseini H, Amirfattahi R, Khorshidi S. Real-time pose estimation and tracking of rigid objects in 3D space using extended Kalman filter. In: Proceedings of 2014 22nd Iranian Conference on Electrical Engineering; 2014 May 20–22; Tehran, Iran. New York: IEEE; 2014. p. 1545–9.
|
| [49] |
Diankov R. Automated construction of robotic manipulation programs [dessertation]. Pittsburgh: Carnegie Mellon University; 2010.
|
| [50] |
Barrientos A, Peñín LF, Balaguer C, Aracil R. Fundaments in Robotics. New York: McGraw Hill; 1997.
|
| [51] |
Goldenberg A, Benhabib B, Fenton R. A complete generalized solution to the inverse kinematics of robots. IEEE J Robot Autom 1985;1(1):14–20.
|
| [52] |
Courty N, Arnaud E. Inverse kinematics using sequential monte carlo methods. In: Perales FJ, Fisher RB, editors. Articulated motion and deformable objects. Berlin: Springer; 2008. p. 1–10.
|
| [53] |
Wampler CW, Leifer LJ. Applications of damped least-squares methods to resolved-rate and resolved-acceleration control of manipulators. J Dyn Sys Meas Control 1988;110(1):31–8.
|
| [54] |
Buss SR. Introduction to inverse kinematics with Jacobian transpose, pseudoinverse and damped least squares methods. IEEE J Robot Autom 2004;17(1–19):16.
|
| [55] |
Ghosh M, Amato NM, Lu Y, Lien JM. Fast approximate convex decomposition using relative concavity. Comput Aided Des 2013;45(2):494–504.
|
| [56] |
Mamou K, Ghorbel F. A simple and efficient approach for 3D mesh approximate convex decomposition. In: Proceedings of 2009 16th IEEE International Conference on Image Processing; 2009 Nov 7–10; Cairo, Egypt. New York: IEEE; 2009. p. 3501–4.
|
| [57] |
Diankov R, Kuffner J. OpenRAVE: a planning architecture for autonomous robotics. Pittsburgh: Robotics Institute; 2008.
|
| [58] |
Quigley M, Gerkey B, Conley K, Faust J, Foote T, Leibs J, et al. ROS: an opensource robot operating system. In: Proceeding of IICRA Workshop on Open Source Software; 2009 May 12–17; Kobe, Japan. New York: IEEE; 2009. p. 1–6.
|
/
| 〈 |
|
〉 |
