The goal of this project is to explore how much we can learn about physical objects a user is looking at by observing gaze depth. We envision a symbiotic scenario, where current technology (e.g. depth cameras) is extended with "human sensing data". Here, a depth camera is able to create a rough understanding of a static environment and gaze depth is merged into the model by leveraging unique propetries of human vision:

• the human eye as a near-range depth camera
• using smooth pursuit eye movements to recognise dynamic objects

The fusion between human abilities and physical sensors can potentially leverage each individual advantages to overcome current technical difficulties and is also a start to explore future human-machine symbiotic sensors.

The gaze-scans were obtained by one author, who scanned the objects with their eyes, i.e. by consciously looking at the objects' outlines and main features at a distance of approx. 0.5m. For a proof of concept, we tested our approach with three objects, differing in geometric complexity and contained depth cues:

• a simple three dimensional geometric object (Scan 1: black box)
• simple geometric objects at different depth levels (Scan 2: letters built in Lego bricks)
• an organic object that includes several depth levels itself (Scan 3: head of mannequin)

< Back to projects