1. Introduction
Our society evolves and is profoundly transformed by the advances in science and technology. However, grand engineering challenges remain open in environmental sustainability, urban infrastructure, health treatment, among others[1–2]. Nowadays, sophisticated unmanned systems have become increasingly capable at an affordable cost. Further blending such systems with cloud and edge computation, 5G, Internet of Things (IoT), and big data allows networked unmanned systems to collaboratively operate as a ‘‘team” to meet our society’s grand challenges [3]. As we can see in the daily life, unmanned systems, as the embodiment of artificial intelligence (AI), are step-by-step becoming commonplace. For example, multiple unmanned aerial systems aviate in national airspaces, connected autonomous vehicles drive on the roads, surgery robots operate in the hospital, and service robots walk into a home for the heath care of the elderly and children with special needs.
Yet, despite the remarkable achievements of networked unmanned systems, there is a gap between their current capabilities and the required capabilities for the grand challenges. We argue that the potential reasons are mainly twofold. First, the current networked unmanned systems are primarily designed for and applicable to a narrow range of domain-specific missions. This prevents the emergence of collective intelligence on a larger scale to deal with complexities in missions and environments. Second, the increasing trend of being ‘‘unmanned” leaves the ‘‘humanity” part un-inherited, which makes it hard for people to trust, accept, and use unmanned systems. Indeed, human attitude towards unmanned systems and the impact of unmanned systems on human behaviors are relatively less explored, which ultimately compromises the expected social and economic benefits of using unmanned systems in our society.
In fact, the high expectations associated with smart society requirements increasingly demand that unmanned systems leave their typical structured and controlled environments to work alongside humans. In other words, the unmanned systems should reach well-rounded human-level intelligence to perform diverse missions with humans in complex environments. To achieve satisfactory and harmonious co-existence of humans and unmanned systems for better living and working, the following questions arise: ① How to jointly gather the strength of various unmanned systems and regenerate services for humans as a response to demanding requirements and never-encountered events in uncertain and changing environments? ② How can such capable systems humanly collaborate, be positively perceived, and be widely accepted in society? The integration of unmanned systems into society is non-trivial, as our well-being or even lives to some extent depend on these unmanned systems (e.g., autonomous driving applications).
The urgent requirement is that multiple unmanned systems must socialize to recognize, understand and establish relationships with each other, humans, or objects (possibly unknown in prior) in unknown environments. Unmanned Intelligent Cluster (UnIC), proposed in this study to meet this requirement and serve our society better, is composed of socially aware unmanned systems. The core principle is to increase the cognitive ability of unmanned systems and to integrate human needs as an indispensable part, which allows the artificial unmanned systems to think humanly in the interactions and act compatibly with human preferences in perception, decision, and operation.
‘‘Un” of UnIC stands for a cohesive union of networked unmanned systems, and more specifically, it stands for a union of the capabilities from a large scope of unmanned systems. In a broad sense, regarding the scope, we do not explicitly distinguish between robots/machines and autonomous unmanned vehicles operating in multiple domains (air, sea, land, and space), for the following reasons. First, they share common characteristics in that they are usually mobile, self-sustained, self-directed, and rich in resources. Second, despite operating in different domains, there are overlaps in their safety-critical or mission-critical task characteristics. Third, they face a high level of challenge in cooperation, cognition, and sociability. Unlike intelligent virtual agents, unmanned systems in UnIC have physical bodies as the embodiment of AI and are mobile in multiple domains. To name a few, unmanned systems in UnIC include spacecraft, airship, unmanned aerial vehicle (UAV), unmanned ground vehicle (UGV), unmanned surface vehicle (USV), unmanned underwater vehicle (UUV), humanoid, and humanoid robots.
‘‘Intelligence” is at the heart of UnIC, which mainly refers to collaborative, interactive, and social intelligence. The source of such intelligence can be bio-inspired, brain-inspired, and societyinspired. The interpretation and source of intelligence of UnIC are different as compared to those of the current unmanned systems. When interacting with each other, humans, objects, and environments, the state-of-the-art unmanned systems have shown capabilities to solve the ‘‘what,” ‘‘when,” and ‘‘where” problems, but leave the essential ‘‘who,” ‘‘why,” ‘‘how,” and ‘‘what if” problems to humans [4]. With these limitations, it is not surprising to occasionally observe naive machine behaviors that humans perceive as awkward, strange, unsafe, or even dangerous. In fact, these deeplevel problems require unmanned systems to own strong humanlike cognitive capabilities such as inference and reasoning, as well as human-like social capabilities such as relationship understanding and establishing. To reach this goal, UnIC is expected to possess a high level of collaborative, interactive, and social intelligence, which are complementary with each other.
‘‘Cluster” is a fundamental form to address complexity [5]. Cluster in UnIC means an emerging pattern of cooperation, intuitively referring to teams, functions, and services arised from the networking of unmanned systems. Here we mainly highlight two aspects of the cluster: membership and structure. Concerning membership, the clustering pattern is open and dynamic, meaning that members in a cluster may adapt their strategy and reform new clusters across the physical, information, and social space. Thereby, an unmanned system might be involved in multiple clusters, playing variable roles. Concerning structure, the cluster could be in the form of hierarchically layered and nested groups, or teams of various qualities, sizes, and compositions. Note that these characteristics are also different from those of the traditional clusters in satellite cluster/constellation, service robots, UAV formation or swarms, connected autonomous cars, and UUV fleet. For the traditional clusters, unmanned systems are usually designed with fixed configurations for a specific mission, and the social interaction rules between machines or between humans and machines are often designed task-by-task. The drawback of such closed systems is that they cannot establish relationships with the right participants in large-scale networked unmanned systems and thus cannot generalize the respective capability for different missions in novel situations. UnIC distinguishes itself from the closed systems via the large-scale social connection and collaboration, which are enabled by the following: scalability with volatile group membership, interoperability with secure access, functionality and service virtualization, and flexibility of on-demand reconfiguration.
Unmanned systems have been traditionally employed for socalled dangerous, dull, and dirty (3D) tasks to work for humans. In a more digitalized modern world, however, they are expected to live in symbiosis with humans as cognitive, collaborative, and companionable (3C) partners in an organized and networked manner[6–8]. UnIC is expected to adapt to this role change and the new challenges coming along.
The research on networked unmanned systems has received considerable attention globally. China launched an eight-year, 200 million USD national research initiative called the coexisting–cooperative–cognitive (Tri-Co) Robot in 2016, for the purpose of exploring human–machine and machine–machine collaboration. The proposed coexistence will allow robots to ubiquitously and safely work alongside humans, whereas cooperation will enable robots to coordinate and collaborate effectively with other agents (either people or robots). In Europe, the H2020 project launched ‘‘Road-, Air- and Water-based Future Internet Experimentation” that focused on coordinating multiple cross-domain unmanned vehicles; likewise, the project of the cooperative–cogni tive–control for autonomous underwater vehicles (Co3 AUV) has been initiated [9]. Another European project named Robotic UBIquitous COgnitive Network, investigated the robotic ecology concept for the purpose of providing pervasive services in daily life by using collaborative platforms [10]. In the United States, the National Ubiquitous Collaborative Robots Initiative (NRI-3.0) and many global robotics projects were introduced towards a similar goal [11]. In the industry community, Starlink systems from SpaceX have been providing internet services via a large number of cooperative low orbit satellites. All these research efforts have contributed to a solid foundation for the implementation of UnIC.
The organization of this paper is as follows. Section 2 discusses related works and concepts. In Section 3, we outline the fundamental concepts of unmanned intelligent system clusters, highlighting the roles of humans and social interactions. Section 4 introduces the key-enabling technologies to support UnIC. In Section 5, we briefly introduce some research on social integration of machines. Section 6 lists the remaining critical open questions, and finally, Section 7 presents the conclusions.
2. Related works and concepts
Existing research related to collaborative unmanned systems is rich, with ubiquitous robotics and swarm being representative topics. Both topics and UnIC emphasize collaboration and networking of multiple unmanned systems. In addition, ubiquitous robotics shares in common with UnIC in that embedding unmanned systems in daily life and environments is proposed[10,12]. Yet, UnIC distinguishes with them in group formation and composition. Compared to ubiquitous robotics, UnIC can form dynamic clusters with varying sizes and memberships. As for swarm, it often refers to a group of homogenous members for a narrow range of tasks, and in many cases its members are less capable, intelligent, and expensive [13]. Compared to swarm, UnIC allows for heterogeneity of unmanned systems in multiple aspects such as autonomous level or social identity. Moreover, neither swarm nor ubiquitous robotics seems to actively consider the role of human in the system design [14], while the UnIC concept stresses an open, interoperable, reconfigurable, and particularly human-aware collaborative group structure.
Recently, a renewed trend of bringing humans into the engineering system has been observed [15]. Related works include human and system of systems[16,17] and human centered multiple agent system (MAS) [18]. In the following, we briefly cover two representatives: cyber–physical–human system (CPHS) and human-in-the-loop IoT, highlighting the similarities and differences compared to the proposed UnIC concept.
2.1. Cyber–physical–human system
Cyber–physical system (CPS) is not only a field closely related to UnIC, but also one of UnIC’s main precursors. CPS captures the interactions between cyber aspects (e.g., communication and computation) and physical systems, ranging from avionics on aircraft to connected mobile vehicles used in multiple systems. Based on CPS, CPHS further recognizes that humans are social beings and have social relationships. In this context, humans move within social connections, with cyber information augmentation overlaying on human activities and behaviors. The importance of human involvement is also recognized in industrial CPS [19], factory CPS, and residential CPS[20,21]. Extensions of these concepts to a larger scale include cyber–physical social system (CPSS)[22–24].
Compared to CPHS or CPSS, UnIC focuses more specifically on multiple unmanned systems. It features a high level of heterogeneity and mobility, and has outstanding capabilities in sensing and actuating. In addition, UnIC can respond to human social need by sharing humans’ physical or cognitive workload in a real-time fashion. In other words, humans play a more central role in bridging the physical and cyber worlds in UnIC.
2.2. Human-in-the-loop IoT
UnIC and human-in-the-loop IoT share the common vision that human needs and social networks are considered as essential factors to be coupled with IoT research. In the IEEE IoT committee, a working group initialized in 2018 explicitly considers human’s role in IoT development [25], paying particular attention to how the increasing usage of technology affects human. UnIC aligns with human-in-the-loop IoT in the basic idea that the objects around us should understand humans desires, requirements, and interests, so that they can provide reliable and trustworthy solutions[26,27]. The advantage of such human and IoT integration is to foster service discovery, resource visibility, object reputation assessment, and source crowding. Note that robots can also be seen as a special category of things, and in this context, Internet of Robotic Things (IoRT) is proposed to further converge sensing, actuation, connectivity, and services[28,29].
Compared to human-in-the-loop IoT, which generally explores networked objects, UnIC particularly focuses on human interactions with intelligent and mobile entities. Despite the fact that human-in-the-loop IoT can interact with humans via providing context information, the information required may easily exceed the perception range, sensing dimensions, and the precision of the sensor network. The problem, however, might not exist for UnIC, as it pursues information richness and broader contexts by mobility. Besides, UnIC allows to form open and dynamic clusters by inviting new socially connected unmanned systems that may contribute to the mission.
3. UnIC concepts
This section details the concept and intelligence-driven features of UnIC. We also elaborate on the physical, information, and social space from which UnIC emerges.
3.1. UnIC overview
UnIC refers to autonomous unmanned systems that are humancentered and networked through intelligent social collaboration. This human–machine collective optimizes the coordination of humans, machines, objects, and environments via knowledge sharing and social awareness, such that it continuously improves the capabilities of both individuals and teams by increasing mutual understanding and trust. The aim is to provide humans with a set of personalized, diversified, and socially acceptable functionalities and services. These services emerge around humans by clustering necessary virtual and physical resources. When required, both the clusters and the clustering approaches can be reconfigured according to human intention and adapt to the dynamic, unknown environment.
The following are essential features of the UnIC operation.
• Unmanned systems form clusters with adequate scalability to respond to events.
• Clusters can both emerge and be dismissed depending on the mission and human needs.
• Human preferences and social acceptance play essential roles.
• UnIC is both the service provider and the service consumer.
• UnIC can serve as mobile infrastructures for themselves and others.
The autonomous mobility of UnIC facilitates close-to-human capabilities not only in perception and cognition, but also in actions and behaviors. The prominent intelligent features include self-sensing, self-decision making, self-actuation, self-learning, self-healing, self-growing, and self-governance. UnIC uses unmanned system platforms to provide four basic blocks of collaborative functionalities, namely perception (UnIC–P), communication network ((UnIC–N), computation (UnIC–C), and service (UnIC–S). The concept of UnIC is illustrated in Fig. 1.
Fig. 1. UnIC concept. Info.: information.
We will now briefly introduce these four functional blocks. UnIC–P is compatible with existing systems like IoT but adds value to them. The cluster in UnIC can connect to the IoT networks for cooperative sensing of objects, the environment, and humans. Also, the knowledge acquired and refined is shared at both the human– machine and the machine–machine level. UnIC–N can provide ubiquitous networks, which allows UnIC to be implemented in a distributed manner or to be configured to form clusters out of existing distributed systems. UnIC–C is responsible for contextaware computations in the interpretation of situations, reasoning, and predictions, while UnIC–S manages the service pool and organizes services according to the mission objectives. These four blocks jointly enable the growing capabilities of the human– machine collective in services and missions.
The implementation of UnIC sits at the intersection of multiple disciplines: computer science, robotics, communications technology, automation, and social science. On the one hand, it exploits the advances in information and communication technology (ICT), IoT technologies, and AI to build smart ‘‘minds.” On the other hand, it benefits from achievements in energy, material sciences, and manufacturing for its physical ‘‘body.” The ‘‘body” and ‘‘mind” are integrated by social and life sciences to enable the ‘‘soul.” These indispensable elements are illustrated in Fig. 2. The interactions between humans, machines, objects, and environments can be analyzed in three spaces: the physical space, the information space, and the social space, as illustrated in Fig. 3. In contrast with many other paradigms, the social space is highlighted and plays a unique role in achieving the full benefits of UnIC.
Fig. 2. Enabling elements of UnIC.
Fig. 3. Human–machine–object–environment interactions in physical, information, and social space.
3.2. Sources of intelligence in UnIC
The intelligence of UnIC originates from three sources: bioinspired intelligence, brain-inspired intelligence, and societyinspired intelligence. The intuitive reason to investigate these three sources is that organisms in nature, networked neurons in the brain, and teams in human society all survive and prospect by working together. In the following, some recent inspirational results on the three sources of intelligence are introduced to inform the design and implementation of the intelligence feature of UnIC.
3.2.1. Bio-inspired intelligence
The emergence of collective intelligence in UnIC can be inspired by understanding nature. Biologically inspired principles have guided the design of multi-unmanned systems in aspects ranging from collective information processing to self-organized behavior patterns. Biologically inspired robotics [30] and swarm robotics [31] are two related research fields. Below, we will introduce some potential inspirations for UnIC in collective motion, perception, and decision-making. First, to achieve collective motion, the bottom–up approach to establish self-organized collective behavior can be taken. The natural systems indicate that local interactions lead to global patterns. For example, birds, fishes, and cells can use local interaction rules to form complex patterns or structures. Even when the local motion is stochastic, random motion of individual components can lead to deterministic behavior [32]. Likewise, the morphogenesis principles are used to build an intricate structure with each robot using identical rules. The robot swarm shows robustness to noise, damage, and adaptability to changing environments [33]. Second, a top–down approach to generate local rules from the desired macro collective behavior can be taken. Programmable swarm robots have the ability to achieve the desired collective shape, illustrating an example of highly self-organized behavior by design [34]. In a similar sense, to achieve collective sensing, individual local measurement can be modulated with social cues from neighbors [35]; to achieve efficient decisionmaking in dynamic groups, sparse knowledge is sufficient. Such knowledge accumulates and further improves collective intelligence [36]. Third, group size and hierarchy are found to be vital for decision-making performance. The larger the group size, the better the decision accuracy [37] and the smaller the proportion of informed individuals is needed for the given accuracy [38]. A well-defined hierarchy may also help efficient group decisionmaking from an evolutionary point of view. For example, the fat tail distribution of competence in the human group suggests better group performance [39].
The advantages of using bio-inspired intelligence include the following: ① higher degree of global competence emerging from a collection of simple distributed systems; ② improved resilience to failures of individuals and disturbances of the environment; and ③ increased flexibility and scalability. The findings above shed light on the design of multiple unmanned systems in UnIC with self-healing, self-growing, and self-governance capabilities in uncertain and harsh environments [30,31,40,41].
3.2.2. Brain-inspired intelligence
The breakthroughs in AI have their roots in the increasing understanding of human brain[4,42,43], which has attracted considerable attention in governmental funding support all over the world [44]. Such studies are accelerated by systematic nonintrusive imaging instruments, which allow us to observe neuron activities with unprecedented spatial–temporal resolutions [45]. Currently, two approaches are adopted to investigate brain-inspired intelligence. One approach is the inverse engineering of the brain with biologically plausible imitations. This includes the efforts to mimic neural circuits by using spiking neural networks. Recently, low-cost and large-scale simulations of spiking neural networks has been implemented matching a monkey’s brain [46]. The other approach takes the working principles of the brain as inspiration to provide new types of computational models (representation, abstraction, and reasoning) and architectures. This line of research has fueled the development of artificial cognition and the transfer of cognitive concepts into robotics[47,48]. The inspired models include spatial representation and navigation which leverage human place cells [49], affordance computation of applicable actions linking to human mirror cells [50], semantic cognition which supports the use and generalization of knowledge with neural bases [51], and hierarchy motor control architectures [52]. Recent neuroscience research has also identified several neural bases of social interactions, which include ‘‘social place cell” for identifying others’ location and ‘‘identity cells” for supporting the interactive behavior of social groups [53]. It’s worth noting that the inspiration is bidirectional between neuroscience and AI, as AI also offers new perspectives and tools for neuroscience research [54]. One encouraging example, is that the deep learning navigation agent shows a grid cell structure that resembles human brain [49].
It is important to note that a better understanding of the brain can inspire the human-like ability to discover knowledge. Despite the success of probabilistic machine learning approaches [55], brain-inspired connectionism using neural networks seems to prevail[56,57]. By using the insight from neural network, AI is able to discover knowledge. Currently, for example, AI has been shown to discover physic laws and new mathematical conjectures and theorems[58–60], to produce interpretable psychological theories [61], to assist in the discovery of new materials[62,63], and to find ways of achieving nuclear fusion for sustainable energy.
Researchers have also investigated how better systems can be built by learning. Three approaches may contribute to guiding an unmanned system how to learn: ① Cognitive science may shed light on what humans learn and how they learn it[47,52,64]; ② inspiration can be obtained from results in developmental research on how infants acquire skills in a progressive manner[65–67]; ③ brain–machine interface in both motor control and mood regulation has emerged as a new scientific tool for investigating learning [68].
3.2.3. Society-inspired intelligence
Human society sets a good example for UnIC of how complex problems can be collectively solved when the members possess human-level intelligence. Teams, groups, communities, or organizations are the primary forms of problem-solving in human society. Working in groups improves productivity, and it has been found that scientific teams produce more frequent knowledge and higher impact than individuals [69]. Generally speaking, a group of connected people outperforms a single individual in improving productivity and accurately answering complex questions. To this end, society-inspired intelligence refers to approaches learned from human society for solving complex problems, with social group structure and social intelligence playing central roles.
There is evidence that a crowd structured in small groups can outperform a large unstructured crowd via aggregating knowledge [70]. Beyond efficiency, the exploration of social network structure can provide predictions of group behavior. It has been revealed that people with close social proximity, like friends, usually possess similar ways of perceiving and responding to the surrounding world [71]; such findings can be used for the early detection and management of epidemics[72,73].
It is important to note that social intelligence is essential to maintain social group relationships. What is social intelligence? As suggested in Ref. [74], ‘‘Social intelligence is the ability to understand the feelings, thoughts, and behaviors of persons, including oneself, in interpersonal situations and to act appropriately upon that understanding.” Evidence shows that general intelligence (defined by verbal and abstract thinking) and social intelligence are independent regarding the following social intelligence measures: prosocial attitude, emotionality, social anxiety, social skills, and empathy skills [75]. It is beneficial to have both general and social intelligence. As reported, human groups characterized by both high general and social intelligence possess a remarkable ability to manage common resources. More specifically, these two kinds of intelligences indicate a high cognitive capability to recognize the usefulness of resources and social relationships, which enables practical, consistent, and sustainable collective actions [76]. As unmanned systems become increasingly capable of social interactions with humans, it is important for them to understand the feelings, thoughts, and behaviors of people in social settings. Tools in social science like social norms, judgment, reputation, and trust can be borrowed and used in regulating and optimizing social behaviors of multiple agents[77,78].
3.3. UnIC and the clustering spaces
The clusters in UnIC emerge from three spaces: physical, information, and social. The physical space contains all the physical systems and objects in the interactions, which ensure safe motion of unmanned systems in multiple domains. The information space involves perception, communication networks, and computation. The high-level outcome of UnIC activities in this space is knowledge generation and sharing. In the social space, UnIC promotes trustworthy social interactions with the support from the other two spaces.
3.3.1. Clustering in the physical space
In the physical space, UnIC clusters necessary resources around humans to set up an ecology in the context of human activities. The main actors are unmanned systems (e.g., UAV, UGV, and USV) or UxV, which accommodate payloads such as multiple modal sensors/communication/computation modules. The main benefit of using UxV platforms is access to the multi-domain physical space. Such access is enabled by a series of coordination, ranging from low-level components’ actuation, subsystems’ actions, platforms’ motion to the overall system behaviors. Since UxV shares the physical space with humans, tempo-spatial coordination among UxV platforms needs to guarantee collision-free motion. In general, UnIC is expected to act as human partners or delegates, without negative interferences with humans while following humans’ intention closely.
3.3.2. Clustering in the information space
In the information space, UnIC clusters around the cognitive needs of humans and augments humans for better thinking. UnIC processes the heterogeneous, unstructured, incomplete, and asynchronized raw data into knowledge, which is more stable over time than data and context [79]. In this process, semantic technologies enable interoperability and a human-centric view of the objects and the environment, allowing for the active participation of nonexpert humans. This is a solid basis for the collaborative ability to learn, think, and understand[80,81], which produces fused situation awareness. In general, humans benefit from augmented cognition with the help of UnIC. On the other hand, humans also contribute to knowledge generation through human mobility as sensing, or through human crowdsourcing as computation units.
3.3.3. Clustering in the social space
In the social space, members in UnIC are expected to actively understand, model, and infer the behaviors of other machines and humans. The clusters of UnIC in the social space can thereby emerge around humans by establishing relationships via social identity, preference, common ownership, brand, goal, and location. In fact, social space considerations can be coupled into the two other spaces. For example, the physical space can be socially partitioned using social cues, signals, and proxemics [82]; social semantic maps can be built for further reasoning, data sharing, and interoperability in the information space[83,84].
Clustering in the social space requires UnIC to possess a high level of social autonomy and to exhibit socially acceptable behaviors[85–88]. The automated social characteristics include deterministic courtesy [86], social value orientations [89], and other irrationalities using prospect theory (e.g., framing effect, riskseeking behavior, and loss-aversion behavior[90,91]). Social norms and reputation can also be used to help promote human understanding [92]. Learning techniques are proposed to obtain automated social characteristics such as altruism, curiosity, attention, and affordance[93–97]. Both short- and long-term learning help the pursuit of social autonomy [98].
Clustering in the social space also requires the interaction interfaces with humans to be personalized with multimodal, interactive, and multitasking properties [99]. Such interfaces can be physical (e.g., a service robot), virtual (e.g., a software assistant), bio-physical (e.g., brain machine interface or a body augmentation form of robotic limb) [100,101], cyberphio–chemical [102], tactile, and verbal (e.g., natural language interpretation).
4. Technology enablers of UnIC
UnIC takes a human-centric layered structure, as shown in Fig. 4. It comprises four core layers: ① A1: layer of unmanned system platform (UxV); ② A2: layer of sensing and network ; ③ A3: layer of computation; ④ A4: layer of service. These four layers interact with each other, with intelligence, cognition, and ethics embedded in each layer [86,103]. Key enabling technologies of the four layers are reviewed as follows.
Fig. 4. Human-centric layered architecture of UnIC. A1: layer of unmanned system platform (UxV); A2: layer of sensing and network; A3: layer of computation; A4: layer of service.
4.1. Layer of unmanned system platform
In this section, we introduce some latest development of unmanned systems operating in multiple domains.
4.1.1. Unmanned system as a physical entity
Unmanned systems (including robots) are the fundamental building blocks of UnIC. They come in different sizes [104], work in or even across different domains, and perform diverse tasks with multiple modes, as shown in Table 1. It is also common to use them in teams for better group performance [105]. In such cases, they can self-organize, self-assemble, and reconfigure their physical ‘‘body” shapes [33,106].
Table 1 Heterogeneous unmanned system platforms [110–115,118–130].
The latest advances of unmanned systems feature increasingly small-scale and human-friendly design. At the nano-scale, for example, the 2016 Nobel Prize in Chemistry was awarded to the researchers who pioneered building nano-machines on the molecular scale. Although in their infancy of autonomy, they are expected to be used for macro-level tasks such as drug delivery [107,108]. A deeper discussion on building molecular machinery can be found in Ref. [109]. At a relatively larger scale, a survey on the design and manufacture of drones can be found in Ref. [110]. Among these, the Robobee is noteworthy for its crossdomain operation and untethered flight with a weight of around 100 mg [111,112]. Concerning human-friendly design, soft robots have attracted a great deal of attention, as safe interactions with humans is one advantage of soft robots [113,114]. For example, lightweight and low-speed flapping drones have flexible wings and a natural appearance, making them ideal candidates for indoor applications. Some studies on soft robots show that sensing, computation, actuation, and energy supply with machine intelligence can be integrated with the constructing material of the soft body [115,116], and that technologies like 4D printing with programmed tempo-spatial behavior can be utilized for the dedicated motion [117].
4.1.2. Unmanned systems: From body to mind
The mobility of unmanned systems is enabled by locomotion, which is the mechanism to transport unmanned systems from place to place by various modes of interactions with the operating environment. At the macroscale, locomotion modes can be wheeled, wing-born, or bi-pedal. At the microscale, they can be very flexible with the possible choice of using smart programmable materials. These materials can be self-folding [131] and can further be used for an entire functional system [132]. Since single-modal locomotion has limitations when operating in dynamic environments for complex tasks, multi-modal locomotion has been under intensive investigation. Multi-modal locomotion can be accomplished either via a collection of domain-specific and distributed unmanned systems, or via a single integrated system with multimodal capabilities, such as a humanoid [133,134].
When autonomously moving in the physical space, unmanned systems need to guarantee safety first, both for themselves and for humans and objects in the environment. In most unmanned systems, depending on the level of autonomy, safety requirements often demand detect-and-avoid functions. Reviews concerning collision-free operations of various unmanned systems in unstructured environments can be found in the literature, including: UAV [135], USV [136], UUV [137], autonomous vehicle [138,139], space systems [140,141], and mobile robots [142,143]. Based on the collision-free motion, unmanned systems are able to explore the environments [144] and perform tasks with time-varying group formations [145,146].
The development of unmanned systems is also reflected by the ‘‘mind” building, which is the ‘‘see–think–act” cycle [147] (i.e., the perception, decision-making, and action cycle). Due to the current breakthroughs in AI, recent research advocates cognitive architecture again [148,149]. For unmanned systems, cognition not only plays a role in their active perception, but is an essential aspect for their anticipatory decision-making. For example, humancentered unmanned systems can be designed by integrating socially aware navigation and motion planning [68]. In general, the interaction of a cognitive mind and a physical body of an unmanned system would allow the exploration of challenging tasks (e.g., dexterous manipulation illustrated by the automatic assemblage of an IKEA chair) [150].
Overall, the current unmanned systems offer precision, efficacy, and even safety by working beyond the limits of human perception and dexterity. Yet, they suffer from limitations in the following two aspects. First, it is still a challenging task for the current unmanned systems to perform general thinking and social thinking. The stateof-the-art achievements and the grand challenges ahead in robotics have been identified and reviewed in Ref. [151], including social aspects and ethics. Second, unmanned systems are expected to be more physical-damage resilient, adaptive in perception and action, and scalable in system size. When collaborating with humans, they are desired to possess greater self-predicting, selfhealing, and self-growing capabilities [152,153].
4.2. Layer of sensing and network
Distributed sensing and networked communication technologies are converging. They are closely interdependent particularly in the setting of networked unmanned systems. This section discusses both sensing and communication networks in the traditional sense and their coupling with human-participated sensing and social networks.
4.2.1. Sensing and communication network
Sensing is achieved by using either proprioceptive or exteroceptive sensors. Raw sensor data is processed to provide situational awareness of the unmanned system itself, team members, objects, and the environment. Enhanced situational awareness can be further achieved by combing sensing and communication networks, as communication networks support the sharing and transmission of local measurement data among the static/mobile sensor nodes [154]. Such distributed perception from multiple systems allows for gathering richer information in shorter time intervals over vast space. In general, sensing and communication networks collectively promote collaboration of multiple unmanned systems, ensuring safety, efficiency, and convenience.
When designing communication networks for unmanned systems, we need to consider the platform mobility and an open architecture. The mobility of unmanned systems poses challenges in ensuring highly dynamic connections and mitigating environmental interferences. In this sense, communication requirements can be demanding regarding availability, continuity, integrity, latency, and communication transaction time. The emerging 5G technology can offer wide-area communications for mobility with ultra-short time delays and massive connectivity. Such characteristics also enable new sensing networking possibilities such as tactile internet [155], which provides humans with immersive situational awareness to remotely operate unmanned systems. The mobility also allows unmanned systems to provide dynamic coverage and seamless service, as unmanned systems are able to carry relay and base station payload for on-demand communication. Taken together, unmanned system networks serve dual roles: communication for mobility and vice versa mobility for communication [156]. Concerning communication architectures, ad hoc networks are often formed to scale multiple unmanned systems. Examples include mobile ad hoc network (MANET), vehicular ad hoc network (VANET), flying ad hoc network (FANET), and ocean networks [157–159]. In UnIC, unmanned systems require a more open and dynamic communication architecture to form an open machine society, where the members may represent different stakeholders. Challenging questions arise concerning who are the communication partners, what are the information needs, what to transmit at which rates, and what are the communication protocols (may not be hardwired) [160]. Computational interaction frames proposed in Ref. [161] explore in this direction, which may serve as a candidate solution to promote openness for ownership, membership, conventions, and the internal design.
4.2.2. Human involvement and social network
New advances emphasize the role of humans in sensing and communication networks. On the one hand, sensing and communications are employed for human well-being. It is possible to sense and collect data from digital footprints of social behaviors in the physical and information space, which can then be used for understanding human behaviors and activities [162–164]. Humancentric principles also extend to a new generation of communication network like 6G, emphasizing the user experience and social factors like privacy and security [165]. On the other hand, sensing and communications can be implemented via humans. As an example of communication via humans, the human body can be used as the networking channel media to form secure body area networks for biomedical sensors in the proximity of human users [166]. Concerning sensing via humans, participatory sensing and opportunistic sensing are representative examples. They are possible approaches to get humans’ cognitive capabilities and intelligence involved in large-scale socio–technical systems. As participatory and opportunistic sensing rely on human or human network to relay data, the communication pattern and sensing performance largely depend on human mobility, social network structure, and human social preference like altruism [167]. In this sense, mechanism design approaches in social science may provide a possible solution to improve the availability of social sensing data. Note that the social sensing and communication concept is not limited to humans but can be extended to networked unmanned systems (e.g., in social vehicular networks, opportunistic information sharing of events has been found to reduce emissions and alleviate traffic congestion) [168,169].
4.3. Layer of computation
The cognitive and collaborative features of UnIC require context-aware knowledge sharing, cognitive computation, and ubiquitous computation. This section briefly surveys the computation technologies supporting UnIC that concerns data, algorithms, and computational power. The purpose of computing (e.g., specific planning, decision-making, and control algorithms), is omitted here, as we focus only on knowledge representations, computational architectures, and social computation.
4.3.1. Knowledge generation and computational architecture
Data, algorithms, and computational power are core elements of the current AI. For UnIC, high-volume, heterogenous, and multimodal data from networked systems might lead to data management problems. To solve these problems, UnIC can adopt the semantic knowledge approach to refine the data. This approach not only allows reusing old data, adding new data, transferring unambiguous knowledge, but also facilitates better user visibility during data acquisition, processing, and representation. Such transparency provides a prerequisite for human engagement [170]. The outcomes of knowledge generation can be represented as maps showing both cognitive and social landscape: a geometric map representing spatial, temporal, and thematic features in the physical space; a semantic map representing concepts that human can interpret in the information space; a social affordance map representing networked relationships and social acceptance considerations [171,172]. In practice, ontology-based semantic technologies have supported interpretable knowledge representation in health care [173] and agile reconfiguration in flexible manufacturing [174]. The semantic knowledge representations can be domain-specific, emphasizing reliability, safety, and usability. They bring benefits like standardized situational information [175], temporal and spatial context update [176], and collaborative platform behavior [177], which increase the platform’s interoperability [178].
Computation algorithms can produce cognition, which enhances the capability of UnIC in interactions with machines, objects, environments, and humans. Cognition is not a module of mind but a process that entails robust and anticipatory behavior. While cognitive characteristics can be embodied in perception, computation (reasoning, inference, decision-making, intent interpretation), communication, action, and goal adjustment [179], cognitive systems need to be aware of the implications of perceptions and anticipate the consequences of actions. The concept of affordances, which describe the action possibilities that an environment can offer (or afford) a machine, fits into this scheme. Affordances associate the perceived objects with applicable actions, mapping understanding in perceptions to consequence aware behaviors. It has been suggested that such mapping can be obtained via endto-end learning without internal models [180]. As humans prefer to interact with systems with a certain degree of cognition, the cognitive features can help UnIC to maintain autonomy in interactions with humans by understanding human needs and goals [149].
Recent advances in ‘‘computational power” involve computation methods and computation architectures. The new computation methods range from quantum, DNA and biochemical computing to neuromorphic computing [181]. As mobile computation becomes more and more prevalent, ubiquitous computing is dispersed into the environment, devices, and unmanned systems. Regarding computation architectures, three main different types are available: cloud architectures, fog architectures, and edge architectures [182–187], with the latter two being especially important for UnIC’s role as the mobile infrastructure. UnIC can serve as computing centers using fog architectures. For example, slow-moving and parked vehicles can be used as mobile computation infrastructure [188]. Fog architectures also enjoy greater temporal–spatial advantages in the case of unmanned aerial systems, due to high mobility and fast geo-distribution, latency-sensitive knowledge handling in the information space, and possibly behavior-sensitive awareness in the social space. Recent advances in edge computing architectures promote data-centric computing at the edge, which is close to the data source (e.g., in sensors or memory) [189,190]. In UnIC, the three types of computation architectures can be jointly used and smoothly self-switched, depending on the connectivity and safety constraints, as well as mission objectives [191].
4.3.2. Social computation
From an evolutionary perspective, it has been considered that social learning plays a pivotal role in the evolution of intelligence in primates [192]. Likewise, computation approaches that implement social learning are critical for acquiring the expected capabilities of intelligent unmanned systems. Unmanned systems can learn from the information used by humans (e.g., the world wide web or from the demonstrations performed by humans/peer systems) [193,194]. In this regard, the platform Sensorpedia can connect incompatible sensor systems and share information across them [195]. For the same purpose, the platforms Roboearth and Robobrain can share knowledge between unmanned systems [183,196]. In fact, sharing information or knowledge, either from humans to machines or from machines to machines, can provide opportunities for skill transformation [197], which ultimately brings improvement in the success rate of solving problems [194,198]. An example of using social computation in accomplishing complex tasks is demonstrated in the Million Objects Challenge [199].
4.4. Layer of service
UnIC provides services for humans based on the fusion of the three spaces. This section introduces some essential aspects in providing human-centric services.
4.4.1. Human behavior patterns
Providing personalized, precise, and innovative services requires knowledge of human behavior patterns [200]. Computational models can be employed to build comprehensive pictures of these behavior patterns. For example, human behavior patterns can be characterized in the physical space as motion/crowd dynamics [201], in the information space as opinion dynamics and communication patterns, and in the social space as friendship networks and social preferences. Some studies further show that ① in the physical space, human mobility shows a high degree of regularity and reproducible patterns in migration, commuting between cities, and moving in crowds [202,203]; ② in the information and social space, human beliefs and decisions are coupled with their social networks, resulting in interdependent dynamics in the two spaces [204–206]. UnIC, which perceives, thinks, and works socially, can take advantage of these behavior patterns when scheduling services.
4.4.2. Service virtualization and management
UnIC is expected to deliver rich, personalized, and on-demand services with the support of the IoT network in missions like disaster response [207], smart home [208], urban service [209], smart manufacture [210], and smart agriculture [211]. These services can be provided by using a service-oriented paradigm, in which the functions of unmanned systems are virtualized at three levels. At the subsystem level, for instance, the virtualization of sensing results in sensing as a service [212]. At the individual system level, virtualized subsystem services can be packed into system-level services. For example, UAV can provide multiple services like data management, distributed computing, interaction with humans, security, and privacy [213]. Furthermore, the unmanned system can be virtualized as unmanned systems autonomy services (UxAS), and based on this, a software framework has been proposed for mission-level autonomy [214]. At the system of systems level, using task scheduling as service management is feasible even for individual agents with low levels of autonomy. Services virtualized at various levels can be efficiently managed via dynamic service matchmaking (e.g., clustering based on similarity) [215], discovery, replacement, and synthesis [216]. Altogether, service virtualization and management enable the reuse of functionalities and support the openness of the UnIC.
5. Social integration of machines
Even though the capabilities of unmanned systems advance to the point of matching or even surpassing those of humans in some cases, integrating multiple unmanned systems with AI in the society remains as one of the grand challenges in achieving the full social and economic benefits of UnIC [151]. This section briefly introduces some research addressing how machines influence the collaboration with human involvement and social acceptance of machines in general. The promising approaches of promoting acceptance by building trust in machines are highlighted. Such studies provide a path for social integration of UnIC.
5.1. The impact of machines on collaboration
Collaborating with machines has been shown to significantly reduce human workload both mechanically and cognitively. Scholars from different fields such as human–machine interaction (HMI) and social robots have devoted their efforts to identifying the influencing factors of human–machine collaboration. As reported in Refs. [217–219], the question of ‘‘how the task type, operating context, and machine behavior influence human–machine team collaboration” has been intensively investigated in the field of human–robot interaction [220]. Compared to human–machine collaboration, human collaboration via machine is much less explored. There is evidence that using machine as human delegate can enhance human collaboration. For instance, a common use of preprogrammed autonomous vehicles among people can promote cooperation relative to direct human–human interaction. In other words, human cooperation is enhanced when their interactions take place through autonomous machines [221]. Additionally, machine involvement has been found to improve human collaboration in terms of conversation participation and conflict mediation [222]. For example, having a machine in the team that expresses vulnerability can induce higher levels of participation in team conversations and improved perceptions of the team interaction experience [220].
5.2. Social acceptance of machines
The benefits that machines bring for collaborations would not be achieved if machines are not properly accepted and applied. The literature on social acceptance mainly involves three dimensions: ① positive evaluation and belief, as an attitudinal acceptance dimension; ② willingness or planned actions, as an intentional acceptance dimension; and ③ actions of using, as a behavioral acceptance dimension. Although the distinctions between the three dimensions are important, in this section we discuss social acceptance of machines in a general sense.
Existing research on acceptance of machines has addressed drones [223], personal robots [224], robots at work [225], robots in domestic environments [226], autonomous vehicles [227,228], connected vehicles [229], and so forth. Methodologies to investigate acceptance include surveys, questionnaires, acceptance models, field experiments, or controlled lab experiments. In general, barriers exist in accepting intelligent machines. For instance, in a survey conducted in Germany, the majority of people being questioned disagreed about the drone usage for tasks like advertising and parcel delivery. Typical concerns include safety and privacy. Those who have dealt with unmanned systems value safety more than privacy, whereas those who do not care more about privacy. However, both seem to care about privacy as long as their own privacy is considered [223,230]. One another important reason for the insufficient acceptance of machines is the perception of machines’ low level of emotions [231]. Also, even if no barriers exist for accepting the concept of living with unmanned systems, there is a risk of not taking the corresponding actions. For example, for people who agree that the self-driving vehicle in a crash should kill a passenger instead of a pedestrian, they would rarely buy such a vehicle for themselves [232].
5.3. Trust as indicator of acceptance
Obviously, the proper acceptance and social integration of machines are far less sufficient. One crucial factor is insufficient human trust in machines. Trust is considered to be a key determining factor in the acceptance of machines, both technically (e.g., reliability and dependency of functionalities) and socially (e.g., emotions, satisfactions, and preferences) [233,234]. Trust in machines can be influenced by many factors, including task characteristics, demographics, experience, and so forth. In this section we would briefly emphasize three important factors that have the potential to increase people’s trust in machines and could be taken into account in machine designing.
The first factor is identity building, which has always been an important tool to promote trust and cooperation within the human society. In fact, humans also show considerable in group favoritism towards machines with similar identities [235,236]. Hence, having machines build the same identity or group membership with the people interacted could help to increase human trust in machines [237]. Manipulating team structure to recognize machines as partners rather than tools can also improve subjective satisfaction and team performance [238].
The second factor is interpretability. Interpretability of machine behaviors, such as explanations from the machine, has been found to increase trust towards machines [239,240]. However, the limitations of human reasoning also raise the question of the content and level of explanations [240], since a mindless choice of information about what to communicate with people might risk causing information overload. To cope with this challenge, researchers have proposed to check the effectiveness of interpretability using designed experiments [238].
The third factor is human-like behavior. One important underlying reason that humans do not have much trust in machines or have emotionally different trust in machines vs humans lies in the fact that machines have been perceived as having no or little human-like emotions and social behavior. There is some evidence that humans prefer to interact with machines with a higher level of human-like behavior. For example, people have the tendency to prefer proactive machines to reactive machines [241], and people show higher trust in adaptively automated machines than in stable machines who always provide assistance irrespective of whether people need or not [242]. In fact, machine responsiveness and active listening are also found to lower people’s cognitive load and increase people’s willingness to be accompanied by the machine during stressful events [243,244].
6. Open questions and trends
This section lists the urging and unanswered questions which are important for applying UnIC in our society.
6.1. Process from design to verification and validation (V&V)
UnIC defines itself as an open, complex, and giant system [245], which requires system science to integrate both social and technological knowledge [246]. However, the systematic top–down development of such complex systems is to a large extent unmatured [247]. First, the fundamental mechanism of multi-agent interactions for desirable behavior is still not widely available [34]. Second, compared to other disciplines (e.g., automation), the methodologies of design, modelling, evaluation, implementation, operation, maintenance, and V&V in UnIC are still missing. The consequence is that the system development might risk in not knowing the potential adverse effects.
6.2. Behavior driven by general and social intelligence
The question of embodying general intelligence into unmanned systems for various tasks is still open. It might be beneficial to revisit the question of what accounts as AI and how AI takes a physical form and coevolves with its body. These perspectives might help to advance the current definition of unmanned systems and the understanding of AI itself [153,248–250]. As indicated above, unmanned systems equipped with AI still lack common sense, social intelligence, and skills that allow them to transfer capabilities [97,251]; such characteristics are in fact essential features of well-rounded AI [252].
Social aspects of unmanned systems have been considered mainly in the interactions between humans and service robots on a ‘‘build once, apply once” basis. Either these were loosely coupled designs, or social factors were only partially considered. Besides, although the functionality of autonomous levels has been investigated, social interactions at higher autonomous levels have been less explored. In fact, in domains like education and healthcare, having unmanned systems obtain skills to socially engaged and serve in a context is at the core of the design, since in such areas, the interactions involve highly personal attributes. A recent encouraging example is that educational robots have been shown to achieve similar cognitive and affective outcomes compared to human tutors or peer learners [218].
6.3. Safety
UnIC is a safety-critical system that makes real-time online decisions. However, many AI-based enabling technologies result in less predictable behaviors. For instance, it is still unclear how the dynamics of the emergent behavior of unmanned systems develop; it is also difficult to predict the behavior of learningbased adaptive systems or to have a clear idea of the conditions under which such systems work or fail [239]. Besides, human involvement in the networked unmanned system would bring the challenge of privacy concerns due to the cyber–physical nature of UnIC [253,254]. These unsolved issues hinder the application of these systems in safety-critical sectors.
6.4. Standards, guidelines, and regulations
The robot operating system (ROS) is designed by using practically common standards and has been shown to accelerate the autonomy of unmanned systems successfully [255]. It has recently been updated to the next generation ROS 2, which can support multiple-robot applications in nearly real time. Standardization is especially important for UnIC, as UnIC requires interactions across multiple platforms and many domains. Efforts in standardization have been observed in IEEE committees, for example, the IEEE Ontologies for Robotics and Automation Working Group aims to create standard knowledge representations and reasonings to allow for knowledge transmission among agents (e.g., autonomous robots and humans). The group provides a standard ontology for creating autonomous behavior [256]. There are also standards on semantic sensing networks [257] and joint communication architectures for unmanned systems [258]. However, standards, guidelines, and regulations that are highly desirable for the four layers of UnIC are yet to be established.
6.5. Bias, ethical, and moral concerns
The lack of a framework that enables the ethical and moral design of multiple unmanned autonomous systems hinders the social acceptance of UnIC. For instance, controversial discussions arise when unmanned systems decide who should live and die on their own [259]. Moral principles may not be universal and may vary across cultures, as suggested by Ref. [259]. Currently, results are mixed concerning whether social norms can be applied to cognitive agents [260]. Also, how machine bias can be avoided is an open question [261–263].
7. Conclusions
Despite the fact that networked unmanned systems have made remarkable achievements, they are primarily designed for and applicable to a narrow range of domain-specific missions, and have neither sufficient human-level intelligence nor adequate human needs fulfillment. To solve these problems, we propose in this paper a vision of human-centric networked unmanned systems: UnIC. UnIC is envisioned to provide superior power in functionality, adaptability, autonomy, efficiency, reliability, safety, usability, and user experience, which could empower intelligent transportation systems, fast disaster response, environment monitoring, smart manufacture, smart agriculture, health care, and daily assistance. In this paper we emphasize some enabling technologies and factors that can support this vision and its integration into society. More collaborative research and engineering practice from multiple disciplines are needed to promote the realization of this vision.
Acknowledgments
The work is supported by the National Natural Science Foundation of China (U1913602), the National Key Research and Development Program of China (2021YFF0601304), and the Civilian Aircraft Research (MJG5-1N21).
Compliance with ethics guidelines
Fubiao Zhang, Jing Yu, Defu Lin, and Jun Zhang declare that they have no conflict of interest or financial conflicts to disclose.