Large screens and projections have become part of our daily lives. They support collaboration as they allow multiple users to simultaneously access information. For instance, in a meeting presentation, information is shared with multiple users. However, the control of what is being displayed is usually limited to a single user (e.g., in presentation). Others cannot share, access, or manipulate virtual objects or data on the remote display. Direct pointing has been investigated for interaction with remote displays. Direct pointing interaction is a natural way for users to select and interact with objects on a remote screen (see Figure 1(left)). However, such settings enable only few options for interactions and are limited to basic operations. As mobile phones are ubiquitously available, they enable users to access remote displays in diverse ways. For instance, downloading information from a remote display to the mobile phone for further inspection or sharing information on a remote screen with others. Pointing-based interactions offer an easy-to-use way of allowing users to interact with an object by pointing to it. Hence, mobile phones with integrated pointing abilities enable diverse novel options for interaction in a natural and seamless way. Using the mobile phone as a pointing device and general interaction device has not been investigated, thus raising questions regarding how the phone’s specific characteristics (e.g., options for input and output) and attributes (e.g., user data context) can be integrated into the interaction process. In this work, we contribute the detailed investigation of the novel design space of PointerPhone. PointerPhone uses mobile phones as pointing devices for new direct and natural pointing interactions with remote screens (see Figure 1). We present a classification consisting of low-level, widget-level, and high-level interaction techniques. Further, we show application examples and demonstrate the integration of diverse techniques into a collaborative meeting support application which we implemented based on a prototype system that uses laser-pointer-equipped mobile phones. Furthermore, we present observations and results of a qualitative user study and we provide a catalogue of design guidelines as well as lessons learned that need to be considered when designing applications based on PointerPhone interactions.