Colloquium Cognitive Systems

Efficient use of spectral cues for spatial hearing by Marc Ernst, Timo Oess, Heiko Neumann

Prof. Dr. Marc Ernst (Ulm University)

Abstract: Animals perceive the world through multiple sensory systems, yet the information provided by these systems is often inherently ambiguous. In vision, for example, the retinal image alone cannot uniquely determine an object’s physical size because different combinations of size and distance can produce the same projection. The visual system resolves such ambiguities by integrating additional cues, including binocular disparity, motion parallax, and prior knowledge. A similar challenge arises in auditory spatial perception. The sound spectrum measured at the eardrum reflects both the spectral content of the sound source and direction-dependent filtering by the head and outer ears, described by the head-related transfer function. Because both the source spectrum and the applied filtering are typically unknown, determining the elevation of a sound source from the acoustic signal alone constitutes an ill-posed inference problem. Here we show that humans resolve this ambiguity by integrating multiple sources of information. Behavioral experiments reveal that elevation localization becomes extremely difficult for unfamiliar sounds during monaural listening without head movement, but improves dramatically under binaural listening conditions. Familiarity with the sound source and small head movements provide additional, though smaller, benefits. A Bayesian model explains these findings by showing how binaural comparisons can largely cancel the influence of the source spectrum while prior knowledge and motion cues further constrain the estimate. Together, these results suggest that spatial hearing resolves ambiguity through cue-integration strategies closely analogous to those used in vision.

About: Marc O. Ernst is Professor of Applied Cognitive Psychology at Ulm University (Faculty of Computer Science, Engineering and Psychology). He studied physics in Heidelberg and Frankfurt, recieved his PhD from the University of Tuebingen (2000) for work conducted at the Max Planck Institute of Biological Cybernetics - earning both the Attempto Prize and the Otto Hahn Medal - and subsequently completed a postdoc with Martin Banks at UC Berkeley. He then returned to the MPI Tuebingen as Principal Investigator and later led the Max Planck Research Group on Human Multisensory Perception and Action. In 2011 he joined Bielefeld University as Chair of Cognitive Neuroscience and Director of the ZiF, before moving to Ulm University in 2016, where he also served as co-dean and head of the Psychology and Education Institute (2018-2020). His research focuses on multisensory perception, sensorimotor integration, and virtual/mixed reality, with over 100 publications in journals including Nature, Science and Current Biology. He has coordinated and contributed to numerous EU projects (e.g., CyberWalk, ImmerSence, WEARHAP) and is Editor-in-Chief of Multisensory Research. He is a founding member and former President of the EuroHaptics Society and has served on editorial boards, review panels and advisory committees internationally.