Human Robot Interaction using Natural Language
This module gives a basic introduction into the benefits and challenges of natural language interaction with robotic systems. You will get to know the basic building blocks for creating your own Human-Robot interaction system, some theoretical background about the concepts behind the processing modules, like speech recognition, natural language understanding, dialogue processing, etc. With the help of a small example in the course itself, you will be able to create a fully functional HRI system on your own. The basic software modules for this will be freely accessible.
Open Access Educcation pilot
Lesson Description
Introduction (Lesson 1): Motivation for Natural Language Human-Robot Interaction, when and why to use NL Interaction (and when not)
Keywords: Human-Robot Interaction systems, Dialogue Systems, (S)UI Design
Building Blocks (Lesson 2): Required Software Modules, Speech Recognition, Natural Language Understanding, Dialogue Processing, Text Generation and Text-To-Speech
Keywords: HRI Components, Natural Language Processing
Information state / Update dialogue systems (Lesson 3): This lesson starts with some theoretical background of dialogue analysis and processing, and the Information State/Update approach to dialogue. Then, the presented concepts will be illustrated using an example from an implemented dialogue system in a CoBot setting.
Keywords: dialogue analysis and systems, information state/update approach
Ontologies and RDF for dialogue state representation and other implementation issues (Lesson 4): Ontologies and RDF databases are extremely flexible representations of data that are also equipped with additional semantic knowledge. That makes them ideal for implementing the information state of a dialogue system. Finally, we well look into some other implementation issues of HRI systems, especially concerning asyncchronicity, synchronization, and middlewares
Keywords: ontologies, RDF databases and reasoners, synchronization
Example implementation of a small HRI system (Lesson 5): A small example dialogue system will be presented, showing concrete implementation of the NLU, Information State representation in an ontology, rule creation for dialogue processing and text generation.
Recent approaches to NLP for Human-Robot Interaction (Lesson 6): Some recent approaches concerning interactive systems are presented. We’ll look at POMDP approaches to dialogue state tracking and the capabilities of LLMs to serve as interactive systems generating plans for robot execution.
Keywords: Machine Learning for Interactive Systems, LLMs
Learn more
Contact Ernest for more information