Post-Doctoral Research Visit F/M Postdoc in Foundation Models for Human-Robot Interaction

Context
The HUCEBOT team is dedicated to advancing algorithms for human-centered robots: robots that are not working autonomously in isolation, but that instead react, interact, collaborate, and assist humans. To do so, these robots need to intertwine a multi-contact whole-body controller, a digital simulation of the interacting humans, and machine learning models to predict and respond to human movements and intentions. In a crescendo of complexity, the team tackles scenarios that involve collaboration with cobots,
assistance with exoskeletons, and collaboration with humanoid robots. The application domains span from industrial robotics to space teleoperation.
The main robots of the team are the mobile robot Shelfy, TiagoPro bimanual mobile manipulator, the Unitree G1 humanoid, and the Talos humanoid robot. The team also works with Franka cobots and exoskeletons.
The team currently consists of about 20+ members, including permanent researchers, PhD students and post-doctoral students.
Serena Ivaldi, head of HUCEBOT, is holding the chair in Robotics and AI of the Cluster IA ENACT project (
https://cluster-ia-enact.ai/
) that is funding this PhD thesis. In the chair, she wants to push the research in Natural Language to assist humans in different scenarios of collaboration with robots, where safety is paramount. The ambition is to create a foundation that bridges natural language commands into interpretable commands for the robot, leading to robot actions that are contextualized and intrinsically safe.
Assignment
Recent advancements in foundation models in VLMs and VLAs has drastically improved the contextual understanding and motion control for simple tasks. They allow the robot to execute plans and execute a task from visual input alone. However, there are known problems of trajectory hallucinations, safety and scaling to the long horizon problems. These problems make it difficult to make robust and reliable autonomous robot architectures. Tuning these generalized architectures and combining them with classical motion planning and control significantly improves the success rate.
When the robots have to work along with humans, strong safety guarantees are required along with clear understanding of how to interact with humans. We want to exploit the contextual understanding and action planning skills of VLMs and VLAs for motion planning in HRI with focus on safety, interaction and communication modelling and environmental adaptability.
Socially aware robot navigation across various kinds of robots and environments will be the application area of this research position.
This postdoctoral researcher will be funded by the ENACT project, but will contribute more generally to the research of the team in robot learning, HRI.
Main activities
Main activities: implement, test and develop novel algorithms for real robots that use language models and foundation models. Write papers and present them at conferences. Write, test, validate and document its associated software. Experiments with real robots are mandatory.
This position involves contributing in the supervision of the team engineers and PhD students, together with the supervisor.
The postdoc will also be involved in the activities organized by the Cluster-AI project ENACT, which may involve dissemination actions, meetings and presentations to relevant stakeholders (Europe, France, industries, etc).
Skills
Good skills in Python (Pytorch). Ideally, prior experience with LLM, VLM and Foundation Models.
Good knowledge of robotics.
Languages: English (English is the official language of the team and many members do no speak French).
Proactivity and curiosity, daily communication, ability to work in a team are fundamental.