Colloquium Cognitive Systems

Future Eye-based and Multimodal User Interfaces

Dr. Ken Pfeuffer (Bundeswehr University Munich)

Abstract. User Interfaces define how we interact with computers, and they are primarily controlled via hands. With advances in input devices, motion tracking, and natural interaction, future interfaces have the possibility to integrate and fuse multiple user inputs to advance the existing, and create new ways of interaction. Information from advanced modalities such as eye gaze are linked with human cognitive processes, and allow systems to better understand the user's intent and act before users would to ultimately make interactions easier. Our research explores new, potential multimodal systems, to approach questions such as 1) what natural ways are there to fuse multiple input signals for a more powerful UI, 2) what are advantages and drawbacks of such a system compared to traditional manual interaction, or 3) how can complex multi modal systems be presented and designed in a way intuitive for users? To investigate these questions, we explore the input theory, we design prototypes, and we assess their usability in studies. Our insights indicate how interfaces can be advanced in the background to facilitate users by observing their natural behaviour, and in the foreground where explicit user actions are enhanced by intelligent systems interpreting multimodal signals.  

Bio. Ken Pfeuffer is interested in expanding the frontiers of human-computer interaction. He is working as Postdoc at the Bundeswehr University Munich in Germany, and graduated as PhD in 2017 at Lancaster University, UK. His research focuses on the eyes, how integrating visual attention can provide novel interaction concepts for mobile touchscreen devices, large screen systems, and virtual reality. During his studies, Ken has published 15 papers, interned at Google and Microsoft Research, and received a UIST Best Paper Honorable Mention award. In his spare time, he enjoys travelling with his wife, classic real time strategy games, and doing & watching sports.