M. Sc. Anastasiia Spirina

Research

PhD Thesis

Embedded Aigaion Query

Interaction Quality Modelling for Human-Human Conversations

While research related to automatic speech and language processing mainly focuses on speech recognition and language understanding, other information from spoken language may be extracted. For example, “paralinguistics” targets phenomena such as emotion, age, and gender. On a higher level, there still exists a multiplicity of not yet covered, but highly relevant issues related to an automatic processing of spoken language, namely the automatic analysis of spoken communication. The proposed work broadens the scope of automatic extraction of information from spoken language by addressing less researched topics in the field. Thereby, several research questions have not been sufficiently addressed related to human-human conversation.

The proposed work is centred on communication and speech analytics and deals with statistical modelling and classification. Spoken Dialogue Systems (SDS) allow users to interact with computer systems using speech. To assess the performance of an SDS during an ongoing interaction, the Interaction Quality metric  has been introduced. It allows estimating at any point in an interaction, how the system is performing. If the quality drops below a certain threshold, dialogue strategies may be changed  to increase the quality again. The metric is based on a large set of acoustic, lexical, exchange- and dialogue-level features. They describe how an interaction has been going on. A statistical model maps the observations to a target variable, the IQ score, which quantifies the quality of the interaction. While the Interaction Quality metric has been proven to work in human-machine interaction scenarios, the approach may also help to assess the quality of human-human communication. This is for example of particular interest in call centres, where calls with low quality have to be found for training and evaluation purposes.

Based on speech recognition and speaker recognition technology, transcripts of the spoken communication shall be created. First, relevant acoustic, prosodic, linguistic and interaction-related parameters in human-human communication are defined and the interaction is modelled in an appropriate feature space. Second, target variables relevant for an automatic assessment of conversations are defined, annotated, predicted and evaluated.

The proposed work shall model the quality of an interaction as an IQ score, and moreover other qualitative aspects such as cooperativeness between dialogue partners and the probability of task completion. The approaches are intended to be used online, i.e. during an ongoing conversation, and offline, i.e. based on completed conversations.

The main field of applications is the automatic classification of calls in call centres and the automatic filtering of troublesome conversations.

The proposed approaches will be evaluated on large data sets of real, i.e. non-acted dialogues.

Publications

Embedded Aigaion Query 2017

A. Spirina, O. Vaskovskaia, T. Karaseva, A. Skorokhod, I. Polonskaia and M. Sidorov
Analysis of Interaction Parameter Levels in Interaction Quality modelling for Human-Human Conversation
Proceedings of the 19th International Conference on Speech and Computer (SPECOM), Hatfield, UK, September 2017
Bibtex

A. Spirina, O. Vaskovskaia and M. Sidorov
Analysis of Overlapping Speech and Emotions for Interaction Quality Estimation
Proceedings of the 19th International Conference on Speech and Computer (SPECOM), Hatfield, UK, September 2017
Bibtex

A. Spirina, W. Minker and M. Sidorov
Could Emotions Be Beneficial for Interaction Quality Modelling in Human-Human Conversations?
Proceedings of the 20th International Conference on Text, Speech and Dialogue (TSD), Prague, Czechia, August 2017
Bibtex

A. Spirina, A. Skorokhod, T. Karaseva, I. Polonskaia and M. Sidorov
Significance of Interaction Parameter Levels in Interaction Quality Modelling for Human-Human Conversation
Proceedings of the 20th International Conference on Text, Speech and Dialogue (TSD), Prague, Czechia, August 2017
Bibtex

2016

A. Spirina, O. Vaskovskaia, M. Sidorov and A. Schmitt
Interaction Quality as a Human-Human Task-Oriented Conversation Performance
Proceedings of the 18th International Conference on Speech and Computer (SPECOM 2016), Budapest, Hungary, August 2016
Link to Document
Bibtex

A. Spirina, M. Sidorov, R. Sergienko and A. Schmitt
First Experiments on Interaction Quality Modelling for Human-Human Conversation
Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016), Lisbon, Portugal, Vol. 2, pp. 374-380, July 2016
Bibtex

A. Spirina, M. Sidorov, R. Sergienko, E. Semenkin and W. Minker
Human-Human Task-oriented Conversations Corpus for Interaction Quality Modeling
Bulletin of Siberian State Aerospace University, Vol. 17, Num. 1, pp. 84-90, March 2016
Link to Document
Bibtex

2015

A. Spirina, A. Schmitt, E. Semenkin and W. Minker
Interaction Quality in Human-Human Conversations: Problems and Possible Solutions
Journal of Siberian Federal University. Mathematics & Physics., Vol. 8, Num. 2, pp. 217-223, May 2015
Link to Document
Bibtex

2013

A. Spirina, S. Zablotskiy and M. Sidorov
Two-steps system in searching similar words for fast and reliable automatic concatenation of Russian sub-word units
Bulletin of Siberian State Aerospace University named after academician M.F. Reshetnev, Vol. 4, Num. 50, pp. 144-148, 2013
Link to Document
Bibtex

MSc/BSc Topics

Open Topics

For open topics for Diploma, Master or Bachelor Theses and Studienarbeiten, please come and see me in my office or write an e-mail.

In Progress / Completed

Embedded Aigaion Query 2016

Master Thesis
Comparison of the efficiency of classification algorithms for the Interaction Quality modelling for human-human conversations
2016
Link to Document

Bachelor Thesis
Interaction Quality modelling for human-human conversations: significance of the dialogue parameter level
2016
Link to Document

Bachelor Thesis
Interaction Quality modelling for human-human conversations: significance of the window and the dialogue parameter levels
2016
Link to Document

Bachelor Thesis
Interaction Quality modelling for human-human conversations: significance of the window parameter level
2016
Link to Document

Master Thesis
The most informative features selection for the Interaction Quality modelling for human-human conversations
2016
Link to Document

2015

Master Thesis
Interruption detection in human-human conversations in audio files
2015
Link to Document

Research Assistant