The workshop will feature three special sessions, for which submissions are encouraged. The papers to the special sessions will be evaluated according to the same quality standards as the main conference papers but the authors should indicate that their paper is for the special session so that one of the reviewers can be assigned as a representative of the special session theme.

Dialogue State Tracking Challenge

Dialog state tracking is one of the key sub-tasks of dialog management, which defines the representation of dialog states and updates them at each moment on a given on-going conversation. In this challenge, participants will use TourSG corpus to develop the components. TourSG consists of dialog sessions about touristic information for Singapore collected from Skype calls between three tour guides and 35 tourists. The challenge includes a main task on Dialog State Tracking at Sub-dialog Level; four optional pilot tasks on Spoken Language Understanding, Speech Act Prediction, Spoken Language Generation and the implementation of End-to-end Systems; and an optional open track for participants to explore any task of their interest over the provided dataset.

For more information please see http://www.colips.org/workshop/dstc4/

Organisers:

• Luis F. D’Haro
• Seokhwan Kim
• Rafael E.Banchs
• Matthew Henderson
• Jason Williams

Evaluation of Human-Robot Dialog in Social Robotics

With the significant advancements in robotics, the robots will be soon dialoguing with us. Human-Robot Dialog can offer different levels of social interaction. The question is how to evaluate and benchmark these social dialogic capabilities? This needs inputs from different disciplines to enable better understanding of the issues related to interaction with social robots.

Our aim is to make a bridge between social robotics and dialog communities, to gather the corresponding experiences and to find a way to collect large amounts of data, including the multicultural and multilingual dimensions. The Spoken dialog community mostly considers verbal interaction evaluation (spontaneous speech recognition and understanding, dialog acts). The Social robotics community most often evaluate engagement measures in interaction with many non verbal features such as acoustics, gestures, etc. but without spontaneous speech information.

Several dialogs competitions have been already proposed by Darpa, such as ATIS or Communicator, but there are no established protocol and no on-going evaluation campaign. Several robotics competitions are already incorporating Human-Robot Interaction aspects, such as RoboCup@Home, which has a set of benchmark tests to evaluate robot abilities in realistic home environment or the AAAI Grand Challenge, in which the robot is attending and delivering a conference talk. But these competitions do not focus on the spoken dialog challenges. In the IC-FP7 RocKIn project(2014), spoken interaction is evaluated with simplistic dialogues of the command and control type.

Spoken dialog with robots is an emergent field of research: various aspects are still open problems including remote signal acquisition, speech recognition, open-domain understanding, emotion detection, etc., and we wish to take into consideration those various aspects.

Organisers:

• Laurence Devillers
• Kristiina Jokinen
• Joseph Mariani
• Haizhu Li
• Alex Waibel
• Wolfgang Minker

Sociocognitive Language Processing

Sociocognitive Language Processing (SLP) is the idea of coping with everyday language, including slang and multi-lingual phrases and cultural aspects, and in particular with irony / sarcasm / humor and paralinguistic information such as the physical and mental state and traits of the dialogue partner (e.g., affect, age groups, personality dimensions) and social aspects. By that, multimodal aspects such as facial expression, gestures or bodily behaviour should ideally be included in the analysis where possible. At the same time, SLP can render future dialogue systems more "chatty" by not only feeling natural but being truly emotionally socially competent, ideally leading to a more symmetrical dialogue. For that, the computer should itself have a "need for humor", an "increase of familiarity", etc, i.e., enabling computers to experience or at least better understand emotions and personality such that they understand these concepts.

Beyond these ideas, the scope includes verbal behavior analysis, a closer coupling between language understanding and generation incorporating social and affective information, and new language resources to meet these ends. Overall, this includes the idea of bringing together more closely psychologists and the language processing community.

Organisers:

• Björn Schuller
• Michael McTear