Use of humanoid robots in customer service

Pepper as a style advisor

As part of the research project "Use of humanoid robots in customer service", Ulm University, in cooperation with Optik Mersmann, developed an innovative, technology-driven addition to personal sales conversations. The aim is to demonstrate that artificial intelligence, humanoid robots and human product specialists complement and support each other perfectly.

Traditional sales processes often reach their limits in terms of data availability as well as analysis and processing capabilities. In stationary retail, employees usually cannot know all product and customer details by heart. Even if this information were available to service experts (e.g., via a traditional IT system), processing would be almost impossible to manage manually due to the large amount of relevant data and its complexity, or would be distorted by subjective personal views. Similarly, online shops reach their limits because they cannot fully access crucial information directly from the physical world (e.g., about customers' physical characteristics or emotional reactions to recommended products). In this context, humanoid robots could combine the benefits of both the digital and physical worlds. Robots like Pepper have the ability to perceive and interact with customers in the physical world, and also have the computing power to process the resulting data more objectively, quickly and accurately than humans.

In a specific use case, Pepper took on the task of welcoming hundreds of customers at Optik Mersmann in the Ulmer Hafengasse. He made contact with the customers, greeted them and then provided an initial consultation with regard to a suitable shape of glasses. To do this, he analysed the individual facial geometry using Deep Learning and, depending on the customers' wishes, picked a more modern or more traditional suitable spectacle shape. In addition to the general shape of the glasses, pictures of various celebrities wearing glasses that look similar to the customer were also displayed as inspiration. Pepper's ability to interpret basic emotions of the human face and tone, and to distinguish between "like and dislike" came in handy during this task. This allowed him to make further improved suggestions to customers if they did not like the initial suggestion. Pepper then passed his customer on to a member of the Optik Mersmann team for spectacle selection and further advice.

Cooperation partner: Optik Mersmann, Ulm, Germany

Project period: May 2020 - July 2020