Investigating Mid-Air Pointing Interaction for Projector Phones

Projector phones, mobile phones with built-in projectors, might significantly change the way we are going to use and interact with mobile phones. The potential of combining the mobile and the projected display and further the potential of the mid-air space between them have yet to be explored. In this paper we assess these potentials by reporting two user studies: First, an experimental comparison of four tech- niques for target selection on the projection, including in- teraction on the touchscreen of the projector phone as well as performing pointing gestures in mid-air around the phone. Our results indicate that interacting behind the phone yields the highest performance, albeit showing a twice as high error rate. Second, a follow-up experiment where we analyzed the performance of the two best tech- niques of the first study within realistic mobile application scenarios such as browsing and gaming. The results show that mobile applications benefit from the projection, e.g., by overcoming the fat-finger problem on touchscreens and increasing the visibility of small objects. Our findings speak for the integration of a tracking camera at the bottom of the projector phone to enable mid-air pointing interaction.