A robot called Emo that senses when a human is about to smile and simultaneously responds with one of its own could represent a big step towards developing robots with enhanced communication skills more conducive to building human trust, a new study suggests.
While advancements in large language models (LLM) like OpenAI’s ChatGPT have enabled the development of robots that are pretty good at verbal communication, they still find nonverbal communication challenging, especially reading and responding appropriately to facial expressions.
Researchers from the Creative Machines Lab at Columbia Engineering, Columbia University, have addressed this challenge by teaching their blue-silicon-clad anthropomorphic robot head, Emo, to anticipate a person’s smile and respond in kind.
Designing a robot that responds to nonverbal cues involves two challenges. The first is creating an expressive but versatile face, which involves incorporating complex hardware and actuation mechanisms. The second is teaching the robot what expression to generate in a timely manner so as to appear natural and genuine.
Emo may be ‘just a head,’ but it comprises 26 actuators that allow a broad range of nuanced facial expressions. High-res cameras in both pupils enable Emo to make the eye contact necessary for nonverbal communication. To train Emo how to make facial expressions, the researchers placed it in front of the camera and let it perform random movements – the equivalent of us practicing different expressions while looking in the mirror. After a few hours, Emo had learned what motor commands produced corresponding facial expressions.
Emo was then shown videos of human facial expressions to analyze frame by frame. A few more hours of training ensured that Emo could predict people’s facial expressions by watching for tiny changes. Emo predicted a human smile about 840 milliseconds before it happened and simultaneously responded with one of its own (albeit looking rather creepy doing it).
Human-Robot Facial Co-expression
“I think that predicting human facial expressions accurately is a revolution in HRI [human-robot interaction],” said the study’s lead author, Yuhang Hu. “Traditionally, robots have not been designed to consider humans’ expressions during interactions. Now, the robot can integrate human facial expressions as feedback.”
“When a robot makes co-expressions with people in real-time, it not only improves the interaction quality but also helps in building trust between humans and robots,” he continued. “In the future, when interacting with a robot, it will observe and interpret your facial expressions, just like a real person.”
Currently working on integrating an LLM into Emo to enable verbal communication, the researchers are keenly aware of the ethical implications of developing such an advanced robot.
“Although this capability heralds a plethora of positive applications, ranging from home assistants to educational aids, it is incumbent upon developers and users to exercise prudence and ethical considerations,” said Hod Lipson, director of the Creative Machines Lab and corresponding author of the study.
“But it’s also very exciting – by advancing robots that can interpret and mimic human expressions accurately, we’re moving closer to a future where robots can seamlessly integrate into our daily lives, offering companionship, assistance, and even empathy. Imagine a world where interacting with a robot feels as natural and comfortable as talking to a friend.”
The study was published in Science Robotics.
Source: Columbia Engineering
