Introduction
An aging society is widely considered to be one of the main socio-political challenges of the 21st century. Demographics, improved health care, increasing urbanization, and the trend towards smaller families imply that more and more elderly people are living alone. Recent data for Europe indicates that currently about 17.5% of the total population (88.5 million out of 505.7 million people in 2012) are aged 65+, but this percentage is expected to increase to 30% (164 million people) in 2050 [
1]. The number of elderly people living alone varies between countries, and is as high as 67% for people aged 80+ in Scandinavia. Although similar numbers apply for the USA, these numbers are dwarfed by estimations for Asia, due to the large populations and the expected rise in family income over the next decades. In Japan, the percentage of persons aged 65+ is expected to grow from 17.2% (21.6 million people) in 2000 to 36.5% (45.8 million people) in 2050, and the corresponding estimates for China are 6.9% (88 million people) and 23.9% (400 million people), respectively.
Mobile personal service robots, capable of helping people in their daily housekeeping tasks, are one approach to tackling the problem (Figure 1). The Robot-Era project, funded as part of the European Seventh Framework Programme (FP7), proposes a fully integrated system for ambient assisted living (AAL) [
2]. The project targets persons aged 65+ who are either living alone or with their families, but who do not have dedicated caregivers. These should be people who still maintain a high level of autonomy according to the usual medical classifications, although they may show the first signs of physical and cognitive disabilities. The system services are designed to help the users with their daily-life activities (communication, dressing, cleaning, cooking, etc.) and to stimulate and encourage mental and physical exercise.
The requirements for the system were derived from focus-group studies using interviews, questionnaires, and the observation of elderly users in Germany, Italy, and Sweden [
3]. Unlike typical laboratory robots that are used by experts, the Robot-Era system must be usable by elderly people with no prior knowledge of robots, who may have slight mental or physical disabilities. Therefore, the user interface of the robot is intentionally kept simple, and only exposes a set of robot services that are easy to understand and select.
Figure 2 outlines the overall concept. A set of multiple robots with dedicated roles is coordinated by a planner that has access to a variety of ambient sensors. The basic idea is not only to provide services within the apartment of a single user, but also to span the functionality across buildings and a residential area to outdoor zones and perhaps even a whole town. Data is exchanged and integrated using a common middleware, and contextual information is extracted from the stationary and robot sensors and the user commands and stored in the context-awareness module (CAM) database.
Thus far, three different robots are being considered in our demonstration facilities in Peccioli (Italy) and Orebro (Sweden):
•The outdoor robot is designed for autonomous outdoor transportation tasks (e.g., shopping, garbage), but also provides guidance and walking support to the elderly.
•The condominium robot handles object transportation tasks within a building or building complex.
•The domestic robot takes on the role of a dedicated indoor service robot and is designed for manipulation and cleaning tasks as well as object transportation. The robot system is operated by the elderly users via the tablet and speech interfaces.
In addition, elevators and doors have been instrumented to be used by the robots and the planner.
Due to its close interaction with the user and the need for it to be able to perform autonomous manipulation tasks, the domestic robot is by far the most complex component of the overall system. To keep down the cost of this robot, its hardware was assembled from proven standard components (Figure 1).
The rest of this paper describes the design of the domestic robot, and motivates the major design decisions. Section 2 summarizes the state of the art, and Section 3 describes the overall design of the domestic robot and the selection of its key hardware components. In Section 4, we introduce the software architecture of the robot and outline the main building blocks and the interfaces between them. Finally, Section 5 presents results from the first large-scale test of the system with elderly users, and highlights key design changes made according to our experiences. The paper concludes with a summary and outlook to future work.
Related work
The design of control architectures for autonomous robots remains at the heart of robotics research, with the plan-based approach first implemented on Shakey [
4] and on Brooks’ subsumption concept [
5], on both ends of the spectrum. The designers of many recent robots take a pragmatic approach, and integrate a multilayer architecture with some variant of symbolic planning and scripted skills on top of a robot middleware framework; examples include Player [
6], YARP [
7], ROS [
8], and MIRA [
9]. These frameworks provide a communication infrastructure and proven software for common robot tasks.
Developed since 1998, the Fraunhofer Care-O-bot robots are among the best-known examples of service robots targeting elderly assistance [
10¯
12]. The mobile platform uses four individually steerable wheels for omni-directional mobility. With one robot arm and a foldable tray, these robots can carry objects and perform simple pick-and-place tasks. Originally based on custom hybrid software [
13], these robots are now running the robot operating system (ROS).
Research on elderly-care robots began in Japan around the same time as the first Care-O-bot, with work on WENDY [
14] at Waseda University dating back to 1999. The PR2 robot [
15] also uses a steered-wheel mobile platform. Two compliant 7-DOF robot arms with grippers are mounted on a telescoping spine that enables the robot to lift or lower its arms to increase the workspace. The software is fully based on ROS [
16]. This robot has been sold to several universities and research institutes around the world, and provides a reference and benchmark platform from which several groups can share, exchange, and compare their algorithms. A recent design, the HoLLiE assistance robot [
17], integrates a wheeled platform with two robot arms for advanced manipulation. Like the PR2, an active torso increases the reach of the robot.
Domestic robot hardware
Any robot designed for household manipulation tasks must be able to reach objects and furniture, and its workspace must therefore be similar to the reach of humans. However, the size of the robot is limited by the width of typical doors, while its weight remains a difficult compromise between battery capacity, stability, and the payload of the manipulators. In most scenarios, these robots are expected to move and operate autonomously in direct proximity and even contact with humans, and safety is therefore of the utmost importance.
While it is difficult to estimate realistic prices for research prototypes, the reference robots listed above are all based on rather complex designs. In contrast, the stakeholder interviews and financial analysis carried out by project Robot-Era clearly indicated that only a lower cost robot would be affordable, even for large retirement homes. Given these budget limitations, a differential-drive platform with one robot arm was selected as the basic design. Figure 3 shows the evolution of this robot from the basic design to the prototypes used in the first and second experimental phases of the project.
Mobile platform
We selected the proven Scitos-G5 differential-drive mobile robot [
18] as the base platform for the domestic robot. The form factor allows the robot to pass through narrow doors and corridors, and the weight of the integrated lead batteries results in a low center of gravity that makes the platform safe for the walking-assistance and stand-up-helper scenarios. The runtime with fully charged batteries is several hours, and the robot will return to its charging station autonomously. Sensors for localization and obstacle detection include front and rear laser-scanners (Sick S300 and Hokuyo URG-05), a ring of sonar sensors, the wheel odometry, and a gyroscope.
Manipulator
Our user studies clearly showed that the robot should be able to grasp a variety of objects including soft objects and clothes. In addition to cleaning tasks, the users also expected help with heavy objects and with reaching objects on the floor and on high shelves. At the same time, the acceptability studies indicated the importance of a good design with an aesthetically pleasing appearance and outer form.
Therefore, we considered several lightweight robot arms for the robot and checked them against our functional requirements, which included the robot weight, payload, kinematics and reach, gripper options, force/torque and tactile sensors, mechanical and electrical interface, vendor-provided software, and, last but not least, costs. Figure 4 shows the shortlist of candidates, namely the Schunk Powerball arm [
20], the Kinova Jaco arm [
21], the Schunk modular system [
22], the compliant BioRob arm with memory-wire actuation [
23], and the Universal Robots UR5 arm [
24]. At that time, no vendor had a ROS interface to their arm ready, but all indicated that it was planned.
We finally decided on the Kinova Jaco arm because tele-operation tests demonstrated the versatility of its three-finger hand and indicated acceptable performance despite its low payload of 1.5 kg. Originally designed for helping disabled people in wheelchairs, the Jaco arm also has the advantage of existing medical certification and a pleasing appearance [
25]. System integration is especially simple for the Jaco arm, since it has just one USB-connection and relies on a 24 V DC supply. The arm lacks torque sensors or tactile sensors, but motor currents are measured and provide a rough estimate of forces and torques.
Given its reach of about 90 cm and its slightly unusual kinematics, no possible mount point allowed the Jaco arm to reach both the floor and high shelves. We decided to mount the arm at a height that allowed for good workspace dexterity on tabletops and for reaching the floor. The initial and final mount positions of the arm can be seen in Figure 3. Figure 5 shows two examples of our workspace analysis, one for cleaning tasks and one for a hospital scenario. We are considering mounting the arm on a moving spine (similar to the PR2 [
15] or HoLLiE [
17]) as a later upgrade, to increase the reach of the robot.
Sensors and calibration
The sensor suite of the robot is fairly standard and includes the sensors of the mobile platform (front and rear laser-scanners, sonar, wheel odometry, gyroscope, bumpers, and emergency-stop button). Two FireWire cameras and the XtionPro RGB-D sensor are mounted on the moveable (pan-tilt) head of the robot. Another USB camera on the side of the robot is used to detect objects on the floor to the right of the robot. Standard techniques are used to calibrate the intrinsic parameters of the cameras, and global calibration is done by the tools available in ROS.
Software architecture
This section describes the software architecture of the robot. An abstraction layer encapsulates the robot skills into services for use by the ambient sensor network. Most of the computation is performed by a large number of ROS nodes, with state-of-the-art algorithms for environment perception and motion planning.
Overview and main abstraction layers
The robot software is organized into four main abstraction layers (Figure 6). The topmost layer consists of the user interface (speech, tablet) and the PEIS middleware [
26,
27]. The PEIS system maintains the state of all sensors in the ambient assisted living environment, manages the high-level semantic information about objects and tasks, and provides the symbolic multi-robot planner that controls the different robots, sensors, and smart appliances [
28].
The second level is composed of a set of carefully chosen services that encapsulate the available robot skills. Inside this PEIS-ROS interface layer, the robot skills are further organized into a three-level hierarchy of basic skills (single sensor or actuator), intermediate skills (multiple sensors and actuators involved), and high-level services that require a combination of basic and intermediate skills [
29]. Table 1 lists some examples of robot skills, while Figure 7 illustrates the sequencing and interaction for the bring-object service.
These skills are implemented in turn on the third layer by combinations of interacting ROS nodes. Object grasping and manipulation is managed by the MoveIt! framework, which interfaces to the perception modules for collision-aware and self-filtered arm motion planning. Custom ROS nodes are used to control the Jaco hand for grasping. A dedicated supervisor ROS node manages the scheduling of ongoing services, and provides feedback about task and subtask progress to the PEIS layer and the user interface.
The fourth level is made up of the different sensor and actuator drivers, which in turn interface to the actual hardware devices. Robot localization and navigation is performed by the MIRA/CogniDrive software from Metralabs, which is tuned for the Scitos G5 platform and was found to perform much better than the common ROS navigation_stack.
ROS
Modeling a new robot for ROS begins with the URDF robot model [
30], which describes the geometry and kinematics structure of the robot, including all joints and wheels as well as the sensors and actuators. Suitable ROS device drivers were also available for most components, including the laser scanners, FireWire cameras, and the Xtion RGB-D sensor.
PEIS-ROS interface
The main purpose of the PEIS-ROS interface is a clear encapsulation and mapping of the implemented robot skills to the PEIS tuple space, where information is stored in pairs of keys and payload data. The PEIS middleware receives service requests from the users, keeps the semantic information about tasks and objects, and plans coordinated action sequences for the different robots.
Because most of the robot actions take several seconds or even minutes to complete, the PEIS-ROS layer must support asynchronous service requests with regular feedback during skill execution. Our interface is based on a hierarchy of C++ classes called
exekutors, one for every robot skill, which provide a ROS actionlib [
31] interface to the PEIS network. Each
exekutor subscribes to the PEIS network and is configured to monitor a set of specific target tuples. When the target tuple changes, the goal function is called with the tuple payload as the parameters, and service execution is started. While active, the
exekutor class provides periodic status and feedback messages to PEIS, until the service is completed.
Perception
The design of the multi-modal perception system for the domestic robot has been driven by the different manipulation scenarios. To grasp and manipulate an object, the existence of the object must be detected, along with its position and orientation with respect to the base coordinate system of the robot. In our scenarios, three kinds of objects are relevant:
• Box-shaped household objects (e.g., cereal box, milk carton, instant-coffee package).
• Cylindrical household objects (e.g., bottle, cup, soda pop can).
• Special-purpose buckets and other objects.
We have evaluated and implemented different algorithms for these objects, managed by our Visionhub ROS node. Figure 8 shows the detailed structure and integration of the perception system, and Figure 9 shows a typical recognition result.
The SIFT-algorithm can reliably detect box-shaped objects [
32], as long as at least one box surface provides a sufficient amount of optical features. It is necessary to provide a reference image of each relevant surface as well as the exact dimensions of the object. The full 6D pose of the object can then be reconstructed based on the camera calibration data and the dimensions of the reference images.
Cylindrical objects are best detected using the 3D point-cloud from the XtionPro camera. Since such an object is rotationally symmetric, only two of the three degrees of freedom of the orientation can be detected; however, due to the symmetry, side-grasps can be applied from an arbitrary angle.
In addition to common household objects, special-purpose boxes and buckets are defined for the shopping, laundry, and garbage scenarios. These objects have been designed with handles suited for the gripper of the Jaco arm (Figure 10), while AprilTags [
33] fiducial markers are used for robust detection and accurate pose estimation. This choice has the advantage that only a small marker and not the whole object must be in the field of view of the camera. Object grasp positions are specified in relation to the marker location, and can also be learned using the joystick.
The Visionhub node acts as the common interface between the different nodes for image processing (AprilTags, SIFT, point-cloud) and the manipulation and grasp-planning nodes. It provides a clean abstraction of the high-level task (grasping an object) and hides the low-level details of the underlying object-detection process. The Visionhub XML configuration file includes the following data:
• Dimensions of the bounding box.
• Surfaces (image files) and their exact position.
• AprilTag markers and their size and position.
• Information on grasps (grasp-points maximum force).
The node can also interface to an ontology or to the PEIS context-awareness module with semantic information about objects, their usage, and how to grasp them.
Experiments and results
During 2013 and 2014, the Robot-Era system and services were tested in a large experimental test with elderly users (35 persons in Peccioli, and 35 in Orebro; see Figure 11). Every test user was asked to test and evaluate each of the 11 services implemented at that time, using the tablet and speech interface to interact with the robots. The users evaluated each service for usability, acceptability, and perceived performance. The learning curve for the user interface had a significant impact, and the first services tested suffered from this.
Figure 12 shows example usability and acceptability results based on the SUS (system usability scale) [
34] analysis of the user questionnaires. The SUS method combines positively and negatively worded questions, and maps the replies into numbers ranged from 0¯100. To summarize the results, we also applied the following classification [
35]:
• 0¯64 points: not usable;
• 65¯84 points: usable;
• 85¯100 points: excellent.
The detailed analysis of the experiments has been published as a project report [
36]. For most of the services, no significant correlations were found regarding sex, education level, or wealth of the test persons. While the same hardware and software was used in Italy and Sweden, there were slight differences in overall scores and evaluation.
All feedback from the users as well as video recordings and experiences (e.g., crashes) from the experiment runs were analyzed carefully [
37]. This analysis resulted in a list of changes to the services and important robot and system updates:
• In general, users preferred the speech interface over the tablet, despite the need to carry a microphone. One interesting observation is that the text-to-speech system on the domestic robot was actually too good, so that users expected that they could simply talk to the robot. As a result, the speech-command interface was completely redesigned, switching grammar and commands depending on context.
• Several test persons criticized the size and appearance of the domestic robot. The cover was redesigned accordingly, using better materials and giving the robot a more slender look (Figure 3 shows both earlier and later models).
• Hardware changes for the robot included an updated arm-mount position, a larger object-transportation tray, and improved on-board computer performance.
• Several tests suffered from performance issues on the network and the PEIS middleware. These problems were identified and the software was improved.
• Perception of unknown objects was found to be too unreliable. Visual markers and known objects will be used in future experiments to improve robustness.
All change items were addressed and implemented, so that the system is now ready for the upcoming second experimental loop. Two test series will again be performed in our prototype apartments, but this time the users will be alone with the robots, and researchers will not help or re-start the robots if problems occur. The final test will take place in a third ambient assisted living apartment in Ancona (Italy), where recently recovered hospital patients will be expected to use the system without any help from developers at all.
Summary
In this paper, we outlined an AAL architecture that targets integrated robotic services for elderly users spanning indoor, residential, and outdoor spaces. We motivated the hardware design and the software architecture of the domestic robot, an affordable indoor-service robot. Our software encapsulates the robot functionality in a layered hierarchy of robot services. Advanced perception and manipulation functions of the robot are built from a large number of existing ROS software stacks. The system was tested during 2013 and 2014 in a large field test with 70 elderly users in Italy and Sweden, and the usability and acceptability scores for the robot and the overall system are encouraging. Several improvements were made to the robot as a result of the tests, and additional manipulation and cleaning tasks will be tested in the upcoming second large field test.
This work was partially supported by the European Commission in project Robot-Era under contract FP7-288899.()
PDF
(8114KB)
