Human skin exhibits a remarkable capability to perceive contact forces and environmental temperatures, providing complex information that is essential for its subtle control. Despite recent advancements in soft tactile sensors, accurately decoupling signals—specifically separating forces from directional orientation and temperature—remains a challenge thus resulting in failure to meet the advanced application requirements of robots. This study proposes, F3T, a multilayer soft sensor unit designed to achieve isolated measurements and mathematical decoupling of normal pressure, omnidirectional tangential forces, and temperature. We developed a circular coaxial magnetic film featuring a floating mount multilayer capacitor that facilitated the physical decoupling of normal and tangential forces in all directions. Additionally, we incorporated an ion gel-based temperature-sensing film into the tactile sensor. The proposed sensor was resilient to external pressures and deformations, and could measure temperature and significantly eliminate capacitor errors induced by environmental temperature changes. In conclusion, our novel design allowed for the decoupled measurement of multiple signals, laying the foundation for advancements in high-level robotic motion control, autonomous decision-making, and task planning.
To equip robots with human-like environmental perception and adaptive interaction abilities, tactile sensors with multi-signal sensing capabilities, particularly for force and temperature, have recently attracted significant attention [1], [2], [3], [4], [5], [6], [7]. To date, various types of soft tactile sensors have been proposed, which could be categorized into electrical, ion gel, optical, and magnetic signal-based sensors [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22] (Table 1 and Section S1 in Appendix A). These sensors have demonstrated considerable application potential for force measurements, while some of them exhibited skin-comparable sensitivity [17], [22], [23], [24], [25]. However, recognizing that force is a vector comprising both magnitude and direction components is essential, and accurately determining these components is critical for estimating the operational states and taking appropriate actions. Nevertheless, owing to signal coupling and interference in existing soft tactile sensors, accurately decoding normal and omnidirectional tangential forces remains a significant challenge, particularly when considering temperature effects.
Over the years, scientists have attempted to address the issue of signal decoupling by focusing on materials, structures, and principles. Stretchable sensors based on carbon nanotubes and graphene oxide have been developed to decouple the one-dimensional (1D) tangential force and normal pressure [11]. Sensor arrays utilizing two-dimensional (2D) or three-dimensional (3D) field-coupled thin-film transistors enabled the simultaneous measurement of the spatial distribution of normal pressure and temperature [12]. In addition, sensing receptors based on ion relaxation dynamics could distinguish between thermal and mechanical information without signal interference [17]. Recently, Liu et al. [14] introduced a 3D force decoupling electronic skin (3DAE-Skin) using a heterogeneous encapsulation strategy that incorporated pressure and strain sensors oriented in different directions to measure multidimensional forces. They demonstrated their application in robotic manipulation and food freshness assessment. However, its piezoresistive-based 3D array design resulted in reduced accuracy and stability, higher structural complexity, increased production costs, and inability to decouple the effects of temperature on the measurements.
Previously, we proposed a Halbach-based magnetic film and demonstrated its theoretical capability to decouple normal and orientation-fixed shear forces [22]. However, achieving decoupled normal and omnidirectional tangential force sensing remains challenging. Moreover, the electrical components of tactile sensors are typically sensitive to temperature changes, and robots frequently manipulate objects at various temperatures [26], [27], [28], [29], [30], [31], [32]. In general, the incomplete 3D contact force information of the sensor limits the robot's ability to accurately interpret complex interactions. Multi-signal sensors face challenges in decoupling force and temperature, which makes operations involving temperature changes and monitoring impossible. Additionally, owing to the need for large computing power, traditional decoupling algorithms have the problem of slow response time and cannot meet the needs of high dynamic feedback. Therefore, the development of a highly integrated, compact, and low-cost soft tactile sensing unit with high-precision and dynamic response and 3D force and temperature decoupling capabilities is imperative for achieving adaptive, precise, and intelligent robotic operations.
In this study, we proposed a skin-inspired, multi-material, four-layer sensory unit capable of mathematically decoupling 3D force and temperature, referred to as the F3T unit (Fig. 1, and Figs. S1 and S2 in Appendix A). The sensor dynamically measured the normal force, tangential force, and temperature of the operated targets using electromigration, magnetic induction, and ion translocation, respectively. We demonstrated its ability to decouple static signals and recognize dynamic operations in intelligent robotic manipulation and human-robot cooperation.
Our advancement opens up significant possibilities for enhanced environmental perception, motion control, autonomous decision-making, and task planning for robots [33], [34], [35], [36], [37], [38].
2. Methods and Results
2.1. Skin-like multilayer design of F3T sensory unit
The skin is the largest organ in the human body, featuring a multilayer structure composed of the epidermis, dermis, and hypodermis; each of them contain specialized receptors with distinct functions (Fig. 1(a)). Mechanoreceptors, which are categorized as Pacinian corpuscles, Meissner corpuscles, Krause end bulbs, and Ruffini endings, enable the sensation of 3D force, whereas thermoreceptors detect temperature. This unique architecture allows the human skin to effectively decouple different signals, facilitating the accurate recognition of both the magnitude and direction of force, as well as temperature, without interference from one another, even under complex environmental conditions.
To achieve a skin-like decoupling ability, we designed a tactile sensing unit (F3T) with multiple layers (Figs. 1(b) and (c)). The topmost layer (1 mm thick) was created by pouring an ion gel mixture (poly(ethylene glycol) diacrylate:2-hydroxy-2-methylpropiophenone:phosphate buffered saline = 5:5:1; Sigma, Germany) onto a porous mesh fabric. After curing under ultraviolet (UV) light, the ion gel film was demolded and encapsulated with cyanoacrylate to improve reliability during contact (Fig. 1(d)). The second layer consisted of a thin magnetic film with a circular Halbach pattern (0.5 mm thick). To create this layer, we fabricated a rectangular magnetic film using the standard Halbach array magnetization by mixing polydimethylsiloxane (PDMS) and neodymium (NdFeB) magnetic powders. A laser was then used to cut the film into small isosceles triangles with vertex angles of 22.5°, which were then assembled into a circular Halbach magnetic film (Fig. 1(e)). The third layer (5 mm thick) was a floating capacitor composed of four circular electrodes with a gap of 1 mm between them. The diameters of the electrodes decreased from the bottom to the top, measuring 9, 7.5, 6, and 4.5 mm, respectively. A silicone elastomer containing phosphate ions served as the medium between the electrodes and was molded to fit precisely (Fig. 1(f)). This design ensured that the capacitor was sensitive only to the applied normal force, thereby effectively eliminating the influence of the tangential force.
Consequently, the F3T unit was fabricated into a compact cylindrical structure with a radius of 6 mm and height of 8 mm. It featured an ionic gel-based top layer for temperature sensing via ion translocation, a circular Halbach magnetic film-based middle layer for tangential force sensing through magnetic induction, and a floating capacitor-based bottom layer for normal force sensing via electromigration. The inner silicone elastomer resembled the hypodermis of the human skin, facilitating connections between components and buffering impact forces. In addition, a rigid printed circuit board (PCB) with a 3D Hall-effect sensor served as the backbone, providing fixation and support for the F3T sensing unit, similar to the bone structure of a finger. Electrical signals from the temperature-sensitive ion gel layer and floating capacitor were transmitted to the bottom PCB via two pairs of thin wires. Further details about the architecture can be found in Section S2 in Appendix A.
2.2. Decoupling principle for temperature, normal force, and all-directional tangent force
Unlike existing multi-signal sensing units that struggle to accurately measure and decouple 3D forces and temperatures, our multilayer design allows for the mathematical decoupling of these signals. As illustrated in Fig. 2(a), when the F3T sensor contacts the target object, a composite signal (denoted as f(T, Fz, Fx, Fy)), which includes the temperature, normal force in z direction, tangential force in x direction, and tangential force in y direction, is transmitted to the sensor. The topmost ion gel layer directly contacts the object to gather the temperature information. When the temperature changes, the distance between the polymer chains within the gel also changes, leading to the conversion between non-free and free ions and resulting in a change in resistance (Section S3 of Appendix A and Fig. 2(b)). Consequently, the temperature T of the target object can be effectively decoupled from the force F and accurately measured [39].
The floating capacitor comprised four intersecting disc electrodes (two positive and two negative) with varying radii made of a soft elastomer as the medium. The total measured capacitance was estimated by the following equation:
$C=C_{12}+C_{14}+C_{23}+C_{34}=\varepsilon r\left(3 S_{1}+2 S_{2}+2 S_{3}\right) / 8 \pi k d$ (Section S4 and Fig. S3 in Appendix A); $ C_{12}$,$ C_{12}$,$ C_{12}$,and $ C_{34}$ mean the capacitance formed by electrode 1 and 2, electrode 1 and 4, electrode 2 and 3, electrode 3 and 4, respectively. S1, S2, and S3 are the effective capacitor area of electrode 1, 2, and 3 respectively. k is the electrostatic force constant. This capacitance was insensitive to the tangential force components because the effective capacitor area S and electrode distance d did not change under a tangential load. Notably, both the electrode distance d and relative permittivity εr were correlated to the temperature T owing to the thermal expansion (cte: $ d=f\left(F_{n}, \text { cte }\right)$, Fn is the external normal force) and ionization (it: εr=f(it)) properties of the material. Because these factors had opposing effects on the capacitance C, we could design the $ \frac{\text { cte }}{\text { it }}$ ratio of the material to achieve temperature–sensitive adaptive compensation by regulating the ion content in the soft elastomer (Fig. 2(c) and Section S4). Consequently, the change in capacitance C was solely related to the variations in the electrode distance d caused by the normal force, allowing for an accurate measurement of the normal force.
The magnetic flux distribution on the Hall sensor was generated by the magnetic film and correlated with the deformation of the soft elastic layer caused by either tangential or normal forces. In addition, the measurement accuracy of the magnetic field of a Hall sensor was affected by temperature changes. Thus, the initially measured magnetic field could be represented as $ \boldsymbol{M}=f\left(T, F_{Z}, F_{x}, F_{y}\right)$ (Fig. 2(d)). For a circular coaxial Halbach magnetic film, the tangential magnetic field distribution Mr can be expressed as
where $ M_{0}\left(T, F_{z}\right)$ is the tangential magnetic field strength amplitude as a function of correction factor kr, temperature T, and normal force Fz. In the previous step, the temperature T0 and normal force Fz0 were obtained independently using an ion gel and a floating capacitor. These values could then be substituted into Eq. (1) to obtain the distribution of the tangential magnetic field Mr with displacements Δx and Δy under specific conditions of temperature T0 and normal force Fz0. Given that the displacement in the x and y directions can be directly converted into the tangential forces Fx and Fy using shear strain γ and shear modulus G, the relationship between the tangential forces Fx and Fy and the tangential magnetic field Mr can be determined as follows:
where γx and γy represent the shear strain in x and y direction, respectively, θr means the angle of the tangential force in the xy plane, and h means the height of the elastomer.
Furthermore, the calculation of the normal and tangential forces was closely related to the material properties of the elastomer, allowing for adjustments in the measurement range and accuracy. Generally, a smaller elastic modulus E and shear modulus G of the material result in a reduced sensor range and increased measurement accuracy. For characterization in this study, the elastomer used in the sensor was dragon skin (dragon skin 30, Smooth-On, Inc., USA) doped with phosphate ions.
2.3. Characterization of F3T in decoupling multiple tactile signals
To calibrate the relationship between the contact temperature T and the current I within the gel, we developed an equivalent voltage divider circuit on the bottom PCB substrate. As shown in Fig. 3(a), the current I exponentially increases with the increase of temperature T, modeled by a polynomial equation (Section S5 in Appendix A) with an error of less than ±0.5 °C within the range of 20-80 °C. We measured the temperature T under different normal loading (from 0 to 7 N) and tangential loading (from 0 to 2 N) conditions. We then calculated the ratio of the measured temperature under force loading conditions to the temperature under free load conditions. As shown in Fig. 3(b), the ratio remains close to 1 with an error of less than 1%, while the environmental temperature changes from 20 to 80 °C. This verified that the effect of the external force F on the temperature measurement was negligible, which was consistent with the theoretical prediction.
The normal force Fz was measured based on the value of the floating capacitor C. Here, we converted the capacitance signal measurement into a frequency signal f using an NE555 timer integrated circuit circuit to increase the accuracy (Fig. 3(c) and Section S4). The calibration result indicated that f was approximately linear to the normal displacement ΔH with a slope of −888.2 Hz∙mm−1 (Fig. 3(d)), while it had two linear segments corresponding to Fz. This behavior reflected changes in the apparent stiffness owing to different deformation regimes, that is, initially elastic at small deformations and more complex responses at larger deformations owing to geometric effects, contact conditions, or changes in the internal structure (Fig. 3(e)). By using piecewise linear functions with slopes of -1545 and -1150 N∙Hz−1 to model the relationship between the normal force Fz and the frequency signal f (Section S6 in Appendix A), we effectively decoupled the response into distinct linear regions. This approach achieved a measurement accuracy of 0.01 N in the range from 0 to 1 and 0.03 N for values greater than 1 N. Furthermore, the experimental results indicated that the capacitance C was independent of the environmental temperature T and tangential force (Fig. 3(f)). These characterization results verified the effectiveness of the mount capacitance design for the decoupled normal force Fz measurement. Moreover, a cycling test of the sensor in the pressure mode was provided. As shown in Fig. S4 in Appendix A, the force sensing performance under an applied force strength of 5 N was stable over 10 000 cycles.
To evaluate the accuracy of the tangential force Fr, we first characterized the tangential magnetic field distribution Mr. The results showed that the strength of Mr varied sinusoidally with the tangential displacement ΔR at a fixed normal displacement ΔH (Figs. 3(g) and (h), and Fig. S5 in Appendix A), which resulted from the circular Halbach magnetization pattern. Within the defined tangential force measurement range (ΔR≤2.5mm), the intensity of the magnetic field increased monotonically with ΔR over the ascending part of the sine wave. This behavior indicated a unique correspondence between ΔR and Mr when ΔH was known, allowing us to establish a mathematical model for calculating the tangential force Fr (Fig. 3(i), Section S7 and Fig. S6 in Appendix A).
When a normal force Fz = 7.2 N was applied (i.e., ΔH=2mm), the calibration results showed that the tangential magnetic field increased with the tangential force initially, with an average slope of k1=10531 μT∙N−1 when the tangential displacement ΔR>2mm (i.e., tangential force greater than 1 N), the average slope changed to k2=377 μT∙N−1owing to the sinusoidal distribution characteristics of the tangential magnetic field. Notably, by leveraging this characteristic, the magnetic field changed more significantly in the initial phase, providing a sensor with a higher sensitivity (0.01 N) for measuring Fr even when the load was small (0-1 N). The orientation of the tangential force θ was determined by the signs of the tangential magnetic fields in the x and y directions, specifically using the arctan of the ratio My/Mx (My and Mx are the magnetic field in x and y direction, respectively). The calibration results indicated that θ (the angle between the tangential force and the x axis) was independent of the normal force Fz (Fig. 3(j)) and magnitude of Fr (Fig. 3(k)), which aligned well with the actual situation and θ could be expressed using piecewise linear functions (Section S8 and Fig. S7 in Appendix A). Furthermore, the respective component strengths of the tangential force in the x and y directions could be calculated by Fx=Fr∙cosθ and Fy=Fr∙sinθ, respectively.
The dynamic response of normal force and tangential force sensing was almost timely, whereas its recovery time depended on the applied force strength and stiffness of the elastic materials. More rigid elastic materials implied a larger force sensing range, lower sensitivity, and quicker dynamic response. For the silicon (dragon skin 30) used in our demonstration, the dynamic response of the sensing unit was 50 ms under a 5 N force and a 100 Hz acquisition frequency (Fig. S8 in Appendix A). The influence of the temperature T on the magnetic field measurement Mx was calibrated, as shown in Fig. 3(l), which exhibited a linear relationship. Thus, the influence of T on the magnetic field could be mathematically normalized to achieve compensation and correction during measurement of the tangential force. Consequently, we could achieve the mathematical decoupling of the temperature T, normal force Fz, tangential force in x axis Fx, and tangential force in y axis Fy using the F3T sensor.
2.4. Performance evaluation of F3T sensor under static and dynamic conditions
Effective human-robot interaction requires not only precise, interference-free measurement of tactile signals under static conditions but also the ability to dynamically decouple these signals to accurately recognize human actions and intentions. Although the existing soft tactile sensors are highly sensitive, they often struggle with decoupling forces from different directions owing to signal interference. This limitation hinders the recognition of human actions, especially in subtle interactions such as gentle taps, sliding motions, or object handovers. To demonstrate the capabilities of the F3T sensor, we evaluated its performance in multi-signal measurements and decoupling under both static and dynamic conditions.
First, we evaluated its static performance. As shown in Fig. 4(a), a 100 g weight was placed on the sensor as a fixed normal load at t = 50 s. Subsequently, tangential forces were applied sequentially along the x axis at t = 100 s and along the y axis at t = 150 s. During this process, the ambient temperature progressively increased from 20 to 80 °C. For benchmarking purposes, we also conducted the same experiment using two widely used commercial sensors: the strain-force sensors (VISTE VSZ020, RoHS, China) with an accuracy of 0.01 N, and the thermocouple temperature sensors (TEKTRONIX 2110-220, Tektronix, USA) with an accuracy of 0.25 °C.
As depicted in Fig. 4(a), the temperature readings from the F3T sensor closely align with those from the standard thermocouple, demonstrating the F3T’s robustness and stability (average error less than ±0.8 °C; Figs. S9-S12 in Appendix A) even when subjected to external load variations. Regarding the force measurement, the F3T maintained consistent readings as the temperature changed from 20 to 80 °C, with an average error of 0.03 N (within a 3% error margin). In contrast, the traditional strain-force sensor exhibited significant thermal drift, with errors reaching up to 13.2% at 80 °C. These results confirmed that the F3T unit could accurately detect the magnitude and direction of 3D forces, while simultaneously providing precise decoupled measurements of both force and temperature under static conditions.
To evaluate the performance of the F3T sensor under dynamic conditions, it was integrated into a robotic gripper to hold a plastic block. We then used our hands to apply forces with arbitrary magnitudes and specific directions, interacting with the robot by pulling it up, down, left, right, and pressing (Fig. 4(b)). During this process, the robot dynamically gained tactile information from the F3T sensor and accurately identified the human movement direction and intention. Furthermore, by integrating tactile information as control feedback, the grasper could achieve an adaptive response to disturbances and various interactive actions in the environment.
As shown in Fig. 4(c), in the initial stationary phase (t = 0-3 s), the gripper holds the target with a normal force of Fz = 5.52 N, and a y direction tangential force of Fy = 0.80 N caused by the object’s gravity (Fx = 0 N). During this stage, the static grip force (Ff = uFz > |Fr| = sqrt((Fx)2 + (Fy)2, μ means coefficient of friction) ensures that the gripper maintains a stable hold of the object. In the jamming phase (3-7 s), a dynamic external pulling force in an arbitrary direction is applied, increasing the tangential loads, which may exceed the static grip force Ff. To maintain stability and prevent unintended slipping, the gripper automatically increases the normal gripping force Fz in response to the changes of 3D tangential force Fr. In this demonstration, the application of external interference makes the tangential load in the y direction change in the range from 0.08 to 1.56 N, while the tangential load in the x direction changes in the range from −0.62 to 1.24 N. As a result, the normal gripping force of the gripper changes in the range from 2.64 to 8.28 N accordingly. This adjustment ensures that the static grip force from the gripper always exceeds the tangential load, thereby allowing the grasper to maintain a stable grip on the target object. During the recovery phase (7-10 s), with the withdrawal of external interference, the tangential load is only in the x direction and the same as the initial value of 0.80 N. Likewise, the normal gripping force of the grasper also decreases, returning to the initial level of 5.52 N.
This adaptive grasping ability not only allows the gripper to adapt to external disturbances and maintain manipulation stability, but also allows stable grasping with the smallest force, helps reduce power consumption, and prevents excessive gripping force that may potentially damage the target object. These demonstrations highlight the F3T sensor's superior capability to handle diverse and dynamic 3D tactile interactions, which distinguishes it from other sensors that are typically limited to simpler scenarios such as direct pressing.
2.5. Demonstration of F3T for automated chemical reaction procedure
Automatic laboratories play a critical role in enhancing the efficiency, accuracy, and productivity of both research and industry. They streamline processes, reduce human error, and ensure consistent experimental conditions, leading to faster results and improved reproducibility. However, a significant challenge in realizing automatic laboratories is the effective handling and manipulation of delicate and varied samples, which require a high level of dexterity and precision. Soft tactile sensing units with multi-signal decoupling capabilities have the potential to enhance robotic manipulation in automated laboratories by enabling precise 3D force measurements and temperature sensing, which are essential for handling sensitive chemical tasks and maintaining optimal reaction conditions.
Polyvinyl alcohol (PVA) is commonly used to fabricate hydrogels, adhesives, and coatings. However, highly hydrolyzed PVA is almost insoluble in water at room temperature and typically requires a combination of heating and stirring for complete dissolution. However, if the water temperature increases too rapidly or exceeds 90 °C, a significant risk of agglomeration exists. Careful preparation is essential to ensure consistency in experiments and maintain the performance of PVA-based materials. This process involves real-time temperature monitoring, staged heating, and controlled shaking to distribute the heat evenly, which is a challenge for novice technicians. To showcase the capabilities of the F3T in an automatic laboratory setting, we demonstrated the automatic preparation of an 87% alcoholysis degree PVA solution using a robot arm equipped with our sensor (Fig. 5(a)).
As illustrated in Figs. 5(b) and (c) and Movie S1 in Appendix A, we integrated the F3T tactile sensing unit into the robotic gripper. The gripper first approached the beaker containing the PVA particles and deionized water based on visual feedback within 6 s. Between 7 and 9 s, the gripper’s posture and the distribution of tangential forces exerted on the beaker were detected and relayed in real time. By analyzing the tangential force data, the system detected any imbalance and adjusted the position of the gripper to ensure that the beaker remained at this level.
Then, the beaker was transferred to a spirit lamp for heating, with the F3T sensor continuously monitoring the solution’s temperature to reach the optimal 60-75 °C (10-219 s). Unlike traditional thermometers, the F3T sensor was seamlessly integrated with a robotic system, allowing simultaneous temperature sensing and manipulation without manual intervention or separate equipment. This integration was essential for maintaining a controlled heating process and preventing rapid temperature increases that could lead to agglomeration.
After reaching the target temperature, the robotic system removed the beaker from the heat source and initiated controlled shaking at 1 Hz (220-459 s). Throughout this phase, the F3T sensor monitored the orientation of the beaker and the tangential forces acting on it to ensure stable handling and prevent unintended slippage or spillage.
This real-time feedback loop, enabled by F3T, was essential for maintaining precise control over the agitation of the solution, which was critical for preventing PVA particle agglomeration. As the temperature gradually decreased from 75 °C to approximately 60 °C, the system paused shaking and resumed heating to maintain the optimal dissolution environment. After approximately 15 min of alternating heating and shaking, the PVA particles were completely dissolved without agglomeration. This achievement underscored the exceptional flexibility of the F3T system, which offered an adaptive task-solving strategy that was unmatched by standard chemical equipment.
2.6. Demonstration of F3T for effective human-robot cooperation
Beyond its role as a laboratory assistant, the robotic gripper integrated with the F3T soft tactile sensing unit also opens new possibilities for complex operations and more natural human-robot interactions in service scenarios. To illustrate its potential, we used the example of a “tea delivery” task, that is, a simple and common activity for humans but a significant challenge for robots owing to multiple dynamic interactions, such as varying liquid levels, cup orientation, temperature changes, and most importantly, sensing human intent for handover (Fig. 6 and Movie S2 in Appendix A).
Initially, the robotic gripper, guided by visual positioning, successfully grasped a glass containing 20 ml of liquid at room temperature (18 °C) with a initial grasp force of 7.7 N (0-12 s). The robot then transported the glass to the water dispensing station (13-34 s) in preparation for pouring hot water. The pouring process was divided into three stages: high-flow (35-45 s), low-flow (46-63 s), and micro-flow (64-113 s).
During the high-flow stage, the rapid pouring of hot water caused significant changes in the tangential force (from 1.4 to 2.9 N) in the direction of gravity and a quick rise in temperature from 18.5 to 25.7 °C in 10 s. As the process shifted to the low-flow stage, the hot water flow rate decreased, and the pouring included brief pauses, resulting in slower changes, such as, 1.3 N in tangential force and a 14 °C rise over 17 s. Once the tea reached 40 °C, the process transitioned to the micro-flow stage, where the hot water dripped slowly into the cup. Here, the changes in the tangential force and temperature were minimal, allowing for precise temperature control and thermal equilibrium within the tea. When the temperature reached 55 °C, the pouring stopped, and the cup was ready for delivery to the user (114-136 s). During the entire hot water pouring process, to maintain a stable grip, the gripping force of the gripper was gradually increased from the initial 7.7 to 12.5 N.
Unlike traditional robots, which relied on placing the cup on a flat surface for a noncontact handover, a robotic system equipped with an F3T sensor could detect subtle shifts in force and accurately interpret human intent. At 137-138 s, a rapid decrease in the tangential force in the direction of gravity indicated that the human hand securely grasped the cup handle, enabling a smooth, natural handover.
The soft tactile sensing unit significantly enhances the robot's ability to recognize and adapt to human actions in real time, far beyond the capabilities of traditional visual recognition, which can suffer from delays and occlusions. This breakthrough in integrating advanced tactile sensing technology into robotic systems paves the way for more sophisticated, automtic, and human-like interactions, thereby establishing a new standard for the future of robotics in both industrial and service applications.
3. Conclusions
In this study, we introduced a novel soft tactile sensing unit, that is, the F3T, designed to enhance robotic manipulation by providing precise, decoupled measurements of normal and tangential forces, as well as temperature. Inspired by the human skin’s ability to perceive multiple stimuli, the F3T sensor featured a multilayer structure that integrated a circular coaxial magnetic film and a floating mount multilayer capacitor for effective decoupling of force signals across all directions. Additionally, an ion gel-based temperature-sensing layer was incorporated to accurately monitor thermal changes without interference from external pressure or deformation. The proposed design allowed for the accurate detection and mathematical decoupling of multiple signals, significantly advancing the robot’s capabilities in real-time environmental sensing, complex manipulation tasks, and dynamic human-robot interactions. Through various experiments including tasks such as solution preparation and tea delivery, the F3T sensor demonstrated its ability to perform complex and delicate operations by recognizing human intentions and adapting its responses accordingly.
Our current research has demonstrated the feasibility of mathematically decoupling temperature from omnidirectional force in terms of measurement theory and manufacturing; however, many aspects can be optimized before it can be widely used in industrial and robotics scenarios. For example, ensuring firm bonding between stacked layers in the sensor is critical, as weak interfaces can lead to interlayer peeling, which not only degrades the performance but also introduces signal noise and measurement errors. In future work, techniques such as plasma treatment, UV-ozone exposure, or chemical priming on the surface of each layer can significantly improve adhesion by increasing the surface energy, thus enabling better mechanical interlocking, and enhancing chemical bonding between layers. Performance consistency across different sensor batches is also essential because consistency, reliability, and stability directly affect a sensor’s usability in real-world applications. To achieve that, material procurement and mixing should be strictly controlled, standard molds should be used for manufacturing, and a self-correction algorithm should be applied to achieve uniform sensor performance. In addition, a certain temperature measurement delay occurred after the temperature-sensing layer was encapsulated. To address this issue, we can adopt a material with a high thermal conductivity, reduce the sensor layer thickness, and design a microstructure for faster heat transfer. For applications requiring high temporal accuracy in temperature measurements, predictive algorithms can be incorporated based on the rate of temperature change, which allows faster response times by mitigating delays through estimations.
Overall, the proposed technology represents a major step forward in developing more automatic and adaptable robotic systems, thus facilitating a broader range of applications from laboratory automation to service scenarios, and ultimately contributing to the advancement of robotics toward more human-like perception and interaction capabilities.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was support by Hong Kong RGC General Research Fund (16217824, 16213825, 16203923, and 16217824), National Natural Science Foundation of China (N_HKUST638/23), Research Grants Council Joint Research Scheme (62361166630), and Guangdong Basic and Applied Basic Research Foundation (2023B1515130007).
OvertoomEM, HoremanT, JansenFW, DankelmanJ, SchreuderHWR. Haptic feedback, force feedback, and force-sensing in simulation training for laparoscopy: a systematic overview. J Surg Educ2019; 76(1):242-61.
[2]
LinZ, WuH, JiaH, LiuH, AiX, ZouY.Fixed and sliding FBG sensors-based triaxial tip force sensing for cable-driven continuum robots. In:Proceedings of the 2022 International Conference on Robotics and Automation (ICRA); 2022 May 23-27; Philadelphia, PA, USA. Piscataway: IEEE; 2022. p. 9593-9.
[3]
ZeidanH, RyoE, SuzukiY, LijimaH, KajiwaraY, HaradaK, et al. Detailed analysis of the transverse arch of hallux valgus feet with and without pain using weightbearing ultrasound imaging and precise force sensors. PloS one2020; 15(1):e0226914.
[4]
XiaoZG, MenonC. A review of force myography research and development. Sensors2019; 19(20):4557.
[5]
Arman Kuzubasoglu B, KursunBS. Flexible temperature sensors: a review. Sens Actuators A2020;315:112282.
[6]
ZhaoY, WangX, ZhangY, LiY, YaoX. Optical temperature sensing of up- conversion luminescent materials: fundamentals and progress. J Alloys Compd2020;817:152691.
[7]
RorizP, SilvaS, FrazãoO, NovaisS. Optical fiber temperature sensors and their biomedical applications. Sensors2020; 20(7):2113.
[8]
MuC, SongY, HuangW, RanA, SunR, XieW, et al. Flexible normal-tangential force sensor with opposite resistance responding for highly sensitive artificial skin. Adv Funct Mater2018; 28(18):1707503.
[9]
YinY, WangY, LiH, XuJ, ZhangC, LiX, et al. A flexible dual parameter sensor with hierarchical porous structure for fully decoupled pressure-temperature sensing. Chem Eng J2022;430:133158.
[10]
LeeHK, ChungJ, ChangS, YoonE. Real-time measurement of the three-axis contact force distribution using a flexible capacitive polymer tactile sensor. J Micromech Microeng2011; 21(3):035010.
[11]
KanaoK, HaradaS, YamamotoY, HondaW, ArieT, AkitaS, et al. Highly selective flexible tactile strain and temperature sensors against substrate bending for an artificial skin. RSC Advances2015; 5(38):30170-4.
[12]
LiW, LinC, RasheedA, IranmaneshE, ZhouQ, WangK.A force and temperature sensor array based on 3-D field-coupled thin-film transistors for tactile intelligence. IEEE Trans Electron Dev 2020; 67(7):2890-5.
[13]
HuX, YangW. Planar capacitive sensors-designs and applications. Sensor Review2010; 30(1):24-39.
[14]
LiuZ, HuX, BoR, YangY, ChengX, PangW, et al. A three-dimensionally architected electronic skin mimicking human mechanosensation. Science2024; 384(6699):987-94.
[15]
LeiZ, WuP. Zwitterionic skins with a wide scope of customizable functionalities. ACS Nano2018; 12(12):12860-8.
[16]
ShiJ, DaiY, ChengY, XieS, LiG, LiuY, et al. Embedment of sensing elements for robust, highly sensitive, and cross-talk-free iontronic skins for robotics applications. Sci Adv2023; 9(9):eadf8831.
[17]
YouI, MackanicDG, MatsuhisaN, KangJ, KwonJ, BekerL, et al. Artificial multimodal receptors based on ion relaxation dynamics. Science2020; 370 (6519):961-5.
[18]
Ward-CherrierB, PestellN, CramphornL, WinstoneB, GiannacciniME, RossiterJ, et al. The tactip family: soft optical tactile sensors with 3D-printed biomimetic morphologies. Soft Robot2018; 5(2):216-27.
[19]
PalliG, MorielloL, ScarciaU, MelchiorriC.Development of an optoelectronic 6- axis force/torque sensor for robotic applications. Sens Actuators A 2014;220:333-46.
[20]
YuanW, DongS, AdelsonEH. GelSight: high-resolution robot tactile sensors for estimating geometry and force. Sensors2017; 17(12):2762.
[21]
TomoTP, RegoliM, SchmitzA, NataleL, KristantoH, SomlorS, et al. A new silicone structure for uSkin—a soft, distributed, digital 3-axis skin sensor and its integration on the humanoid robot iCub. IEEE Robot Autom Lett 2018; 3 (3):2584-91.
[22]
YanY, HuZ, YangZ, YuanW, SongC, PanJ, et al. Soft magnetic skin for super- resolution tactile sensing with force self-decoupling. Sci Robot2021; 6(51): eabc8801.
[23]
GuoX, JinC, LiX, ZhangA, HongW, YanZ, et al. Electromechanical performance of flexible sensor based on 3D printed biomimetic all-PDMS-coated multistage structure: toward kinesiology monitoring and human- machine interaction. Sens Actuators A2024;369:115193.
[24]
HouM, HongW, ChenH, ZhangT, ZhuX, HaoJ, et al. Simple fabrication of high-sensitivity capacitive tactile sensor based on a polydimethylsiloxane dielectric layer using a biomimetic gray kangaroo leg structure. J Phys D Appl Phys2024; 57(21):215401.
[25]
HongW, GuoX, ZhangT, LiuY, YanZ, ZhangA, et al. Bioinspired engineering of fillable gradient structure into flexible capacitive pressure sensor toward ultra-high sensitivity and wide working range. Macromolecular Rapid Commun2023; 44(22):2300420.
[26]
GaoW, QiaoZ, HanX, WangX, ShakoorA, LiuC, et al. A MEMS micro force sensor based on a laterally movable gate field-effect transistor (LMGFET) with a novel decoupling sandwich structure. Engineering2023;21:61-74.
[27]
BillardA, KragicD. Trends and challenges in robot manipulation. Science2019; 364(6446):eaat8414.
[28]
EdsingerA, KempCC.Manipulation in human environments. In:Proceedings of the 2006 6th IEEE-RAS International Conference on Humanoid Robots; 2006 Dec 4-6; Genova, Italy. Piscataway: IEEE; 2006. p. 102-9.
[29]
HendrichN, BistryH, ZhangJ. Architecture and software design for a service robot in an elderly-care scenario. Engineering2015; 1(1):27-35.
[30]
AjoudaniA, ZanchettinAM, IvaldiS, Albu-SchäfferA, KosugeK, KhatibO. Progress and prospects of the human-robot collaboration. Auton Robots2018; 42(5):957-75.
[31]
YousefH, BoukallelM, AlthoeferK. Tactile sensing for dexterous in-hand manipulation in robotics—a review. Sens Actuators A2011; 167(2):171-87.
[32]
LiG, LiuS, WangL, ZhuR. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci Robot2020; 5(49):eabc8134.
[33]
ZhaoX, ChenY, QianL, TaoB, DingH. Human-robot collaboration framework based on impedance control in robotic assembly. Engineering2023;30:83-92.
[34]
SheridanTB. Human-robot interaction: status and challenges. Hum Factors2016; 58(4):525-32.
[35]
YangHD, ParkAY, LeeSW. Gesture spotting and recognition for human-robot interaction. IEEE Trans Robot2007; 23(2):256-70.
[36]
LiuH, ZhangZ, JiaoZ, ZhangZ, LiM, JiangC, et al. A reconfigurable data glove for reconstructing physical and virtual grasps. Engineering2024;32:202-16.
[37]
Severinson-EklundhK, GreenA, HüttenrauchH. Social and collaborative aspects of interaction with a service robot. Robot Auton Syst2003; 42(3-4):223-34.
[38]
FreedyA, DeVisser E, WeltmanG, CoeymanN.Measurement of trust in human-robot collaboration. In:Proceedings of the 2007 International Symposium on Collaborative Technologies and Systems; 2007 May 25; Orlando, FL, USA. Piscataway: IEEE; 2007. p. 106-14.
PDF
(4430KB)
