《1. Introduction》
1. Introduction
The vision of autonomous vehicles has a long history. In the year 1925, a prototype of a radio-controlled vehicle was demonstrated by the “Linrrican Wonder” in New York [1]. Since then, autonomous driving has been a topic of science fiction and, more recently, within engineering science. Big car manufacturers and players in other industries have now announced their intention to introduce fully automated cars within the next 10 years.
Autonomous driving is thus on its road to market entry. It requires a set of high-end technologies in both cars and infrastructure that are comprehensively connected or even integrated. A key technology for autonomous driving is the real-time high-definition (HD) map. This technology covers three main challenges for autonomous driving.
The first challenge addressed is the capability of a vehicle to localize itself with high precision in relation to its environment. The second challenge is to solve the problem of recognition and reaction to events appearing on the road beyond the reach of onboard sensors, in a range of more than 200 m ahead or around corners. The third challenge concerns the vehicle’s capability to drive in accordance with the needs of passengers and other traffic participants [2].
Until now, maps in cars have been used mainly for navigation purposes, along with applications around points of interest. The resolution of these maps is not precise enough for autonomous driving. In addition, current maps do not meet the requirement of real-time information (i.e., live maps) and do not provide sufficient information for autonomous driving. Especially in urban environments with high traffic density, the requirements for safe, fully automated driving are tremendous-not only in terms of vehicle specifications but also in terms of infrastructure specifications. A good overview of European progress in this field of research gives the European roadmap smart systems for automated driving [3].
This article provides insights into the research of applied science institutions aimed at the realization of autonomous driving, based on HD maps in the city of the future: the iCity.
《2. High-definition (HD) maps and automated driving》
2. High-definition (HD) maps and automated driving
Fully automated driving requires an intelligent control system consisting of highly performant sensor and robotic technologies. The requirements for a technical system must obviously reflect the capabilities humans need to drive a car. Simply identifying where the roads are is insufficient for autonomous driving. A robotic car must have the ability to detect and avoid obstacles. In addition to immovable obstacles, roads contain dynamically moving traffic participants such as other cars and-particularly in urban areas-the more fragile pedestrians and cyclists. Especially in critical situations, the detection and control system of an autonomously driven vehicle must have fast reaction times. A robotic car requires a range of sensor technologies such as sonar devices, stereo cameras, lasers, radar, and car-to-X communication. All these technologies have different reaches of view, and each technology has a dedicated purpose that is comparable to one or more of the five human senses. The main “sense” of a robotic car is the LiDAR (an acronym for light detection and ranging), a laser-based process that senses objects in the near-field environment of the car [4].
Fig. 1 shows a point cloud image of a LiDAR device mounted on a self-driving car.
The importance of a LiDAR system comes from its accuracy, which is up to a range of 100 m, and its rotational ability of 360 degrees. With more than two million readings per second, a LiDAR system provides high-resolution information on the environment around the car.
The second core sense of a robotic car is the Global Positioning System (GPS), which allows a rough localization of the car. This localization is the basis for state-of-the-art navigation systems with 3D topographic displays, rendered with virtual reality buildings and near-time traffic information. Although it is an advanced technology, the accuracy of GPS is insufficient for the next generation of highly automated driving cars. In a best-case scenario, GPS achieves an accuracy of up to 5 m; however, autonomous driving requires accuracy on the centimeter level.
This need for accuracy means that a robotic car requires additional senses that have to be fused in order to provide a real-time high-resolution image of the car's environment. Radar, ultrasonic sensors, and stereo cameras are important “feelers” of a car's immediate surroundings. This fusion of sensor data and GPS data enables the positioning of a car's location to within 10 cm.
In addition to high-precision localization, vehicle-to-infrastructure and vehicle-to-vehicle communication provides further information that enhances the reach of perception of the surrounding environment to up to 1 km. By imagining millions of cars acting as connected, mobile detection devices, with information hubs collecting sensor data and providing these to cloud solutions, creativity for new business models will be stimulated. For this reason, the consortiums of Audi, BMW, and Daimler have acquired the Nokia spinoff, HERE. This deal gives these car manufacturers access to leading-edge navigation software to drive their next generation of autonomous cars.
The advantage of HERE, compared with its competitors, is its vast amount of data, collected by racking up more miles than anyone else. Exploiting this logic by applying a crowdsourcing business model, based on every car as a rolling measuring machine, will increase the quality of the evolving cloud service toward perfection. In addition to HERE, companies such as Bosch, TomTom, Uber with deCarta, Google, and Apple have joined the quest for autonomous driving.
One of the most promising strategies for autonomous driving is the application of simultaneous localization and mapping (SLAM) algorithms. The reconciliation of real-time gathered data from onboard sensors with cloud-based HD map data provides a virtual image of the car's vicinity to landmarks. This allows an exact determination of location and relation to other road users. SLAM technologies place high requirements on computing power and data transmission. At the moment, the main bottlenecks are the following:
• Data collection: One hour of drive time corresponds to one terabyte of data.
• Data processing: Interpreting one terabyte of collected data by means of high computing power requites two days to come up with usable navigation data.
• Data transmission: Although today's available LTE (4G) allows data transmission at 100 Mbit·s−1, 2.2 Gbit·s−1 are required. However, 5G enables 5 Gbit·s−1 and will be market-ready in 2020+.
• Latency time: For real-time execution, latency must be lower than 10 ms-which requires high-performance computing onboard the vehicle.
《Fig. 1》
Fig.1 A point cloud image of a vehicle approaching an intersection illustrates the complexity of data collected by a Velodyne LiDAR [5].
The current state of research on these problems demonstrates the capability of automated driving by means of prototypes. New players and research institutes such as Fraunhofer-Gesellschaft cooperate with car manufacturers and information technology companies on solutions that also take new propulsion concepts into consideration [6]. The progress of the different research initiatives must be considered as differentiated. New, upcoming tech firms operate differently than tier one suppliers of the automotive industry, considering all points on the original equipment manufacturer (OEM)’s requirements list. With the integration of Android or CarPlay with embedded systems, new tech firms develop automated driving solutions in an agile and flexible way. Of course, this development is still in preliminary stages; however, for the shift toward full automation, software becomes the driving force. Thus, corporate venturing in this field becomes more and more important, in order to secure access to key knowledge and technology-as the Nokia HERE case proves.
An analysis of the progress that has been achieved thus far with the main automated driving initiatives reveals interesting findings. For example, Google is not as close to a marketable solution as the media implies. Multiple thousand-mile trips with a small fleet of semi-robotic cars are far from sufficient to obtain data for fully automated driving. In comparison, Nokia HERE has a fleet of more than 400 cars equipped with LiDAR and four wide-angle 24-megapixel cameras, and has mapped more than three million kilometers. Additional data for the development of an HD map comes from a built-in gyroscope and high-end GPS. The data fusion of these different sources is done by sensor data fusion using high-performance computing power. The storage of the gathered data requires high-speed hard drives with multiple terabyte capacity. The data transfer from car to central cloud computers is done by the exchange of the complete hard drive. To date, this is the fastest and safest way. The data is used for rendering roads and for building up HD maps. Based on such an HD map, a fully automated Daimler prototype drove through Germany.
The necessity of HD maps for autonomous driving becomes obvious when trying to anticipate appropriate behavior in traffic beyond the sensors of a robotic car. To date, there is no sensor available that can localize and determine a car in reference to its surroundings. Although a few research projects are working on a solution based on quantum positioning, they are still in early stages, and the first prototypes are larger than a trunk of a station wagon. It will take a few more years before this quantum physics technology is ready for use [7]; therefore, autonomous driving relies on dynamic HD maps. Localization is ensured by the mapping of HD map data with landmarks such as buildings as reference positions. The position of a car in relation to various landmarks provides an exact localization. The next move is the integration of HD mapping in series of cars [8].
The sensor data collected by thousands of cars allows the establishment of dynamic HD maps with high accuracy and real-time information. With this kind of self-learning cloud service, the latest updates can be sent to all registered cars over a 4G cellular connection. The more cars participate, the more precise the HD map cloud solution will become.
The first prototypes have been put in place recently, in closed areas. BMW and Audi have employed HD maps on various race tracks in order to run a car on autopilot, and have achieved several lap records. The combination of HD maps with onboard sensors and control systems leads to high-performance driving at the limits [9].
The next level of autonomous driving involves leaving closed areas such as race tracks and performing automated driving on public motorways, which have a lower complexity than the dense traffic in urban environments. Highly automated driving (HAD) describes the capability of an automated vehicle to safely operate in real traffic, with speeds of up to 130 km·h−1 on motorways. In BMW's latest research project, fundamental technologies such as environment perception, localization, driving strategy, and vehicle control were developed for HAD [10].
The ultimate level for self-driving cars is fully automated driving under all imaginable ambient conditions, without human intervention. This is the biggest challenge, particularly in future urban traffic with its high complexity in terms of the variety of road users and traffic density.
Up to this point, this paper has mainly considered the car side of the transport system. However, in order to solve complex traffic situations from an out-of-the-car perspective-whether vehicles are being controlled by people or by machines-communication with surroundings is necessary. Such communication involves other traffic participants, as well as traffic signs and sensor-based roadside units (Fig. 2). Vehicle-to-vehicle (V2V) communication helps to improve safety by preventing collisions and providing warnings of problematic traffic conditions. Vehicle-to-infrastructure (V2I) communication is based on an infrastructure of roadside units that facilitates vehicular ad-hoc networks (VANETs); these are a long-term solution that contributes significantly toward intelligent transport systems (ITSs) in providing access to critical safety applications and services for autonomous driving. Although VANETs are becoming more and more interesting, current research has not sufficiently identified the constraints of VANET technologies under real conditions.
In order to ensure stable VANETs with seamless connectivity, full coverage over the best available channels is required. At the moment, managing the overlapping signals of adjacent roadside units is one of the biggest challenges. In a comprehensive simulation study, Ghosh and his research team [11] investigated "the feasibility and benefits of providing a ubiquitous communication in VANET under different mobile environments. The overlapping effect can be in vestigated using concepts such as Network Dwell Time, Time Before Handover and Exit Times."The first results in the research area of VANETs showed "that a hybrid routing protocol is the best choice for VANETs in both urban and highway environments" [12].
《Fig. 2》
Fig.2 Roadside units capturing real-time traffic situations and communicating with vehicles.
Powerful information and communication technology is required in the field of automation and robotics connected to dedicated internet zones such as the Industrial Internet. In a future scenario of a "swarm intelligent and highly interconnected iCity," robotic cars become cyber physical systems [13].
The combination of roadside unit infrastructure and onboard HD maps is a very promising approach to enable autonomous driving. Vehicular nodes obtain precise position and road information from the onboard part of the HD map system in real time, and provide it to a dedicated traffic internet by means of position-based routing protocols. A highly dynamic and rapid-changing network topology of VANETs is developing that enables exact and real-time images of the cars' surroundings. With this technology, swarm intelligent automated driving can be put into practice [14].
Running big data analytics on the acquired data of all participating robotic cars allows additional cloud services for road users, as well as for government institutions. Examples of these kinds of cloud services include:
• Deep machine learning for robotic cars, for optimal behavior in traffic situations;
• Continuous improvement of intelligent traffic control systems;
• Eco-path finder for pollution control; and
• Safety and security solutions based on surveillance [15].
Thus, cloud solutions for autonomous driving can be considered part of the infrastructure of a future traffic system that will consist of robotic cars, roadside units, HD maps, and high-performance computing and storage for cloud services (Fig. 3).
In the current state of technology, there is no alternative to HD maps for intelligent mobility solutions in the iCity. In order to overcome the existing bottlenecks, 5G transmission technology, central (infrastructure backend) and de-central (onboard the car) high-performance computing, and big data solutions are required [16].
In terms of further challenges, 3D mapping for autonomous driving in future iCities will be the next level. There are two main ways to acquire data for HD maps in 3D. The first way involves instrumented, high-tech equipped vehicles for efficient and accurate 3D mapping of roads [17]. The second way involves unmanned aerial vehicle systems equipped with similar lightweight sensors such as LiDAR or stereo cameras, providing a novel platform for photogrammetry. This allows autonomous fast data acquisition, low errors, and dense point clouds for the realization of precise 3D maps [18]. Once autonomous driving is implemented, HD maps in 3D will become essential, particularly for densely populated cities such as those in China. Fig. 4 shows a vision of a future iCity and illustrates the challenge of 3D mapping.
《Fig. 3》
Fig.3 The concept of traffic-related cloud services.
《Fig. 4》
Fig.4 The iCity of the future, with novel 3D mapping challenges [19].
《3. Summary》
3. Summary
Autonomous driving is on its way to becoming a reality. Advanced driver assistance systems (ADAS) of the latest premium car generation already allow a self-driving experience on highways for a few seconds. This kind of feature is based on optical and radar sensors that are able to scan the near-car environment. The results are input to steering, accelerating, and breaking actuators that permit automated driving in situations with low complexity. This is the first step toward autonomous driving. When transferred into urban traffic situations, the information from these sensors is insufficient to enable fully autonomous driving. Additional input such as the exact location, speed, and direction of the car, as well as the current traffic situation and the behavior of other traffic participants, is required. The basic references for this multi-dimensional information are HD maps with an accuracy of ± 10 cm, enhanced by updates on the real-time status of the wider car environment (to a range of about 1 km). Thus, HD maps are a key challenge for the automotive industry, especially because this topic has not been known as a core competence of car manufacturers until now.
This paper revealed some background information on the current state of research and provided an outlook on future scenarios.
《Compliance with ethics guidelines》
Compliance with ethics guidelines
Heiko G. Seif and Xiaolong Hu declare that they have no conflict of interest or financial conflicts to disclose.
京公网安备 11010502051620号
