Human-robot interactions help better understand language learning

CY INITIATIVE: Could you introduce yourself and tell us more about your career and your main research themes? 

I completed all my studies at CY Cergy Paris University with a degree in computer science, followed by a Master's in intelligent and communicating systems. I then joined the ETIS laboratory for my thesis, which focused on human-robot interaction, with an emphasis on emotional interaction. The question I asked myself was: should emotions be central to the development of “intelligent” robots?     

I then pursued a post-doc at Sorbonne University, where I focused on human-robot interaction, but more with typical and atypical children via the Pitié Salpêtrière Hospital. In particular, we studied the interaction between the robot and children with autism. The central question was: what is the impact of the individuals interacting with the robot, and in particular their impact on the robot's learning process? We had a robot to which we aimed to teach a postural recognition task and demonstrate that the person who interacts with the robot can influence the latter's learning.   

Overall, my research in developmental robotics aims to understand the developmental process in living organisms, using the robot as a model of a young child in development. The aim is to explore how robotics can be used to create models to analyze children's behavior. For example, we know that children suffering from autism have difficulty recognizing facial expressions. Therefore, I have developed a model for facial expression recognition. This model allows us to manipulate certain neurons to observe their impact on the recognition of facial expressions. 

The robot is thus a tool for understanding the brain. 

You are one of the recipient of the CY Initiative's 2023 funding program. What does this project involve? And what is its objective? 

The project is closely linked to my previous research on human-robot interactions. In the context of this call for proposals, I wanted to address a central question: how a robot can develop language acquisition, and ultimately propose a model to explain this process in children. The aim is to propose a model of this skill and also to understand whether the people who interact with the robot will have an impact on its learning.   

We've already explored this with a fairly simple skill - learning postures. We discovered that, depending on the people who interacted with the robot, the organization of its artificial neural network was not at all structured in the same way. Building on this hypothesis, we will now focus on a more complex form of learning: language. 

We have designed a model with an artificial neural system implemented on a robot, where the robot will gradually learn vocabulary words through interaction with humans, particularly atypical children.

The second aspect of this project is to try and use the robot to improve language acquisition in children. Our original approach involves learning by teaching. In this scenario, the child teaches the robot vocabulary that he or she has not yet mastered. In this experiment, the aim is to see if the child is perhaps more motivated to learn because he or she has a “teaching” role and therefore a responsibility. This method aims to be an effective and engaging way for children to learn. 

The third part of the project does not involve robots but focuses on Serious Games that enable children to learn vocabulary words in various methods. Rooted in learning by teaching, the child corrects an avatar's answers in the game, allowing the avatar to learn the task based on the child's responses. By assuming the role of a teacher, we believe this approach will make the child feel more like an instructor than a learner. 

In all three areas, we'll be working with atypical children, particularly those with autistic or language disorders. Our aim is then to extend our research to typical children, so that we can compare the behavioral trajectories of these different children. This will enable us to extend the model to any type of children.   

What are the main stages of the project? 

We have made significant progress on the project, even though we're still in the early stages. On November 1, 2023, a PhD student joined us for a 3-year period to work on modeling the early stages of language acquisition. This will enable us to begin test phases quickly. 2024 will be a crucial year for this project. We'll soon be publishing an article on the subject to share our initial results.   

Regarding the serious game, for the moment we have developed a program framework, and we will soon recruit a post-doctoral student to support us in this development. He should join us in September 2024.   

Why did you apply for this call for proposals? 

My ITC project was the winner of the HORIZON program of the call for proposals launched by CY Initiative in 2023. This program aims to support ambitious research programs in strategic research areas for a four-year period. It enables the organization to establish international visibility in the targeted strategic research area, presenting a significant opportunity for my research. Thanks to this support, we have been able to move this project forward, and our aim is to build on it with colleagues from Paris 8 University to set up an ANR project on learning by teaching.   

Our colleagues at Paris 8 University have experience of working with children with handwriting difficulties, and of using robots to improve this skill. Although our focus is on language, we share a similar framework. Our objective is to conduct a joint experiment to seek ANR funding.