Humanoid robot hand is sensed as part of ‘extended’ human body schema

A team from the Istituto Italiano di Tecnologia and Brown University has shown that people can ‘sense’ a humanoid robot’s hand as part of their body schema when they work together on a task using other parts of their boy – such as to slice a bar of soap

Researchers at the Istituto Italiano di Tecnologia (IIT) in Genoa, Italy, and Brown University in Providence, United States, have discovered that people can incorporate a robot hand into the perception of their body’s schema, particularly when they perform a task in partnership with a living limb such as cutting up soap. The study opens the possibility for robot design that must function in close contact with humans, including those applied in rehabilitation.

The project, led by Dr. Alessandra Sciutti, who is principal investigator of the CONTACT unit at IIT, worked in collaboration with Dr. Joo-Hyun Song, a professor at Brown University, examined whether unconscious mechanisms that guide human-to-human cooperation also arise in interactions between a person and a humanoid robot.

The researchers concentrated on a phenomenon known as the near-hand effect, in which the presence of a hand near an object alters a person’s visual attention because the brain is preparing to use the object. The study also considered the brain’s ability to construct a body schema that enables more efficient movement in surrounding space by integrating objects into this internal representation.

Unconsciously the brain builds a body schema that allows a person to avoid obstacles or seize objects without direct visual focus. Tools can be absorbed into this map when they prove useful for a task, just as a tennis racket may feel like an extension of the arm to a player who uses it daily. Since body schema is dynamic, Sciutti’s team investigated whether a robot could be assimilated into it as well.

Giulia Scorza Azzarà, doctoral candidate at IIT and first author of the study, designed and analysed experiments in which participants performed a joint task with iCub, a child-sized humanoid robot developed at IIT (pictured). Human volunteers and iCub sliced a bar of soap together with a steel wire, alternately pulled by the person and the robotic partner.

After the task, the team assessed whether the robotic hand had been integrated into the body schema by quantifying the near-hand effect through the Posner cueing task. This test requires participants to press a key as quickly as possible to indicate the side of the screen on which an image appears, while attention is influenced by an object placed beside the screen. Data from 30 participants showed a clear pattern: they responded more rapidly when images appeared next to the robot’s hand, indicating that their brains had processed it as a near hand. Control experiments confirmed that this effect arose only in those who had worked with the robot on slicing the soap.

The intensity of the near-hand effect varied according to the way the robot moved. When iCub’s gestures were broad, fluid and well synchronised with the human movements, the effect strengthened, resulting in more robust integration of the robot’s hand into the body schema. Physical proximity also mattered: the closer the robotic hand was to the participant during the task, the stronger the effect became.

“Participants who perceived iCub as both competent and pleasant showed a more powerful cognitive effect,” said the researchers. They explained that attributing human-like qualities or emotions to the robot enhanced the assimilation of its hand into the body schema. Partnership and empathy increased the cognitive bond with the machine.

The experimentation was performed under controlled laboratory conditions and have advanced the body of understanding of human–machine interaction. Psychological factors will be central to design of robots able to adapt to human behaviour and to provide more intuitive and effective experiences. These features are vital for robotic applications in motor rehabilitation, assistive technologies and virtual reality.

The research forms part of the wHiSPER project, funded by the European Research Council and coordinated by IIT’s CONTACT (Cognitive Architecture for Collaborative Technologies) unit.

For further reading please visit: 10.1016/j.isci.2025.112791