Cognitive systems and human-machine interaction

A thinking camera that explains its functions to amateur photographers or an upgraded car that winds its way through inner-city traffic with complete confidence – without a driver. Such smart technical systems are not just a distant reality. Engineers, computer scientists and psychologists at Ulm University are conducting research into these partnership services, and are already able to demonstrate a number of prototypes.

in the simulator

Research Project

CRC/Transregio 62: A Companion-Technology for Cognitive Technical Systems


Prof Susanne Biundo-Stephan | Institute of Artificial Intelligence


> 10 million euros


German Research Foundation (DFG)

Funding Period

2009 - 2016

Institutions involved

Ulm University
Otto von Guericke University Magdeburg
Leibniz Institute for Neurobiology, Magdeburg

Smart technology can be so human!

Technical systems are becoming increasingly smart, and yet they often suffer from a lack of usability. To begin with, who has the time and inclination to wade through the lengthy operating manual of a home cinema system? What’s more, senior citizens and less tech-savvy individuals are often overwhelmed by the many features offered by their smartphone. In the key area of cognitive systems and human-machine interaction, computer scientists, engineers and psychologists at Ulm University are working together to resolve this problem: smart technical systems, also known as companions, act as partnership services, adapting to their users, including their individual abilities, needs and preferences. This way, the hurried passenger can make quicker use of the ticket machine, and the car will intervene when the driver is exhausted or loses track at some point.

“In the future, all technical systems – from the television set and the fitness app to the medical diagnostic tool – will be companions that adapt to the highly individual needs of their users and their situations around the clock,” states Professor Susanne Biundo-Stephan, spokesperson of the CRC/Transregio 62 entitled “Companion-Technology for Cognitive Technical Systems”.

Since 2009, more than 70 scientists from Ulm and Magdeburg – including computer scientists, engineers, psychologists, neurobiologists and medical scientists – have been working within this CRC to transfer central human cognitive faculties, such as perception and recognition, interaction and communication, and planning skills and decision behaviour, to technical systems. The second phase, involving 10 million euros in funding, kicked off in 2013.

The first step involves analysis by the companion: using cameras, microphones and other sensors, the technical system records and learns the language characteristics, facial expressions, gestures and psychobiological features of the user. By taking an inventory, the machine is able to recognise the current situation and emotional state of its counterpart.

And if the user shares additional data with his companion, such as data from his smartphone, the system will also consider his preferences and goals. All of a sudden, computers, ticket machines or medical diagnostic tools that are capable of independent planning, reasoning and acting take on a human dimension. Let us return to the example of the home cinema system. In the case, the system will guide the user through its own installation process, for instance. And the smartphone will provide support to individuals who have restricted cognitive skills. In this connection, the researchers never lose sight of data protection and privacy aspects.

Adopting the motto “Technology Made For Me”, CFC 62 has been recognised as a showpiece project by the German Research Foundation. It was also one of the award winners of the German-wide competition “Landmarks in the Land of Ideas” in 2015.

Human-machine interaction as a recurring theme

Interdisciplinary research into cognitive systems and human-machine interaction emphasises the unique feature of the faculty: in 2009, Psychology was added to the technical disciplines of engineering and computer science, complementing and reinforcing them. One important joint research area that overlaps with the CFC is highly automated/autonomous driving, centred around Ulm’s Institute of Measurement, Control and Microtechnology. The institute’s head, Professor Klaus Dietmayer, is convinced that it will only be a matter of years before cars will safely navigate through inner-city traffic without a driver´s involvement, warning each other about congestion and sources of danger. The aim of such high-performance assistance systems is to provide support to drivers, not to incapacitate them. Martin Baumann, Professor of Human Factors, is exploring how both sides can communicate with each another, ensuring sources of error are avoided. The psychologist is testing existing systems and providing advice to colleagues from the technical subject areas. One of the means he applies to achieve this is to conduct experiments in the driving simulator: How many errors does the driver make? And what distracts him? Questions like these are answered using data from the on-board computer and cameras, as well as with the help of eye tracking.

Research into all matters concerning smart cars of the future are institutionalised in the interdisciplinary group of researchers F3 . Additional support is provided by driveU, a collaborative centre involving the Institute of Measurement, Control and Microtechnology and Daimler AG.

Whether in the form of assistance systems for everyday life or highly automated vehicles, application-oriented research into cognitive systems and human-machine interaction is a recurring theme throughout the faculty.

A test vehicles which is able to coast along public roads without any intervention by the driver.
A test vehicles which is able to coast along public roads without any intervention by the driver.

Simulation games and research training programmes to develop young scholars

The Serious Games Research Group, the only one of its kind in Germany, receives funding from Carl Zeiss Stiftung to create a link to teaching and learning research. This group develops simulation games that help users to learn how human organs interact or simply to remember vocabulary. An overarching objective pursued by the scientists is to understand how games foster learning, and which game elements have a particularly motivating effect.

Measures are in place to ensure that sufficient young scholars will be available to continue research in the field of cognitive systems and human-machine interaction: besides the PhD programme offered at the CFC, Ulm University launched its English-language Master’s course in Cognitive Systems, at the interface between computer science and psychology, in winter semester 2014/15. This degree programme is yet another indication of the successful symbiosis between technical subject areas and psychology at Ulm University.