Dr.-Ing. Alexander Schmitt

rgtabs: Main Menue

PhD Thesis

Topic

Statistical Modeling for Online Monitoring of Adaptive Spoken Dialog Systems

Status

completed

Description

The aim of this thesis is the development and testing of domain-independent algorithms and tools for predicting the dialogue outcomes in the framework of a spoken language dialog system. The application, in which the algorithms and methods will be demonstrated and evaluated, will be that of an automated agent providing customer care for technical problems, e.g. with cable TV and broadband internet. The envisioned methods are stochastically based and require huge amounts of log scripts as well as utterance transcripts representing interactions between the system and the user.
The thesis will be carried out under the auspices of the Graduate School Mathematical Analysis of Evolution, Information and Complexity and in cooperation with SpeechCycle, NYC, USA.

Research Interests
  • Information Technologies
  • Spoken Language Dialogue Systems
  • Machine Learning
Projects

ATRACO: Adaptive and Trusted Ambient Ecologies, funded by the European Union
roles: Member

A Companion-Technology for Cognitive Technical Systems, SFB/Transregio 62, DFG
role: Member of Project B1

Publications
Embedded Aigaion Query 2017

S. Ultes, A. Schmitt and W. Minker
Analysis of Temporal Features for Interaction Quality Estimation
Dialogues with Social Robots: Enablements, Analyses, and Evaluation, Springer Singapore, Singapore, pp. 367--379, 2017
Link to Document
DOI
Bibtex

M. Sidorov, K. Brester, S. Ultes and A. Schmitt
Salient Cross-lingual Acoustic and Prosodic Features for English and German Emotion Recognition
Dialogues with Social Robots: Enablements, Analyses, and Evaluation, Springer Singapore, Singapore, pp. 159--169, 2017
Link to Document
DOI
Bibtex

2016

A. Spirina, O. Vaskovskaia, M. Sidorov and A. Schmitt
Interaction Quality as a Human-Human Task-Oriented Conversation Performance
Proceedings of the 18th International Conference on Speech and Computer (SPECOM 2016), Budapest, Hungary, August 2016
Link to Document
Bibtex

A. Spirina, M. Sidorov, R. Sergienko and A. Schmitt
First Experiments on Interaction Quality Modelling for Human-Human Conversation
Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016), Lisbon, Portugal, Vol. 2, pp. 374-380, July 2016
Bibtex

R. Sergienko, I. Kamshilova, E. Semenkin and A. Schmitt
Weighted Voting of Different Term Weighting Methods for Natural Language Call Routing
Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016), Lisbon, Portugal, Vol. 1, pp. 38-46, July 2016
Bibtex

M. Sidorov, A. Schmitt, E. Semenkin and W. Minker
Could Speaker, Gender or Age Awareness be beneficial in Speech-based Emotion Recognition?
Proceedings of the 10th edition of the Language Resources and Evaluation Conference (LREC 2016), Portorož (Slovenia), May 2016
Bibtex

R. Sergienko, M. Shan and A. Schmitt
A Comparative Study of Text Preprocessing Techniques for Natural Language Call Routing
Proceedings of the 7th International Workshop On Spoken Dialogue Systems (IWSDS), January 2016
Bibtex

2015

A. Schmitt and S. Ultes
Interaction Quality: Assessing the Quality of Ongoing Spoken Dialog Interaction by Experts---And How It Relates to User Satisfaction
Speech Communication, Vol. 74, pp. 12--36, November 2015
Link to Document
DOI
Bibtex

M. Sidorov, K. Brester and A. Schmitt
Contemporary Stochastic Feature Selection Algorithms for Speech-based Emotion Recognition
Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Dresden, Germany, September 2015
Bibtex

S. Ultes, M. Kraus, A. Schmitt and W. Minker
Quality-adaptive Spoken Dialogue Initiative Selection And Implications On Reward Modelling
Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), ACL, Prague, Czech Republic, pp. 374--383, September 2015
Link to Document
Bibtex

R. Sergienko and A. Schmitt
Verbal Intelligence Identification Based on Text Classification
Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Dresden, Germany, pp. 2524-2528, September 2015
Link to Document
Bibtex

R. Sergienko, O. Akhtiamov, E. Semenkin and A. Schmitt
A novel approach to neural network design for natural language call routing
Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2015), Colmar, France, Vol. 01, pp. 102 - 109, July 2015
Link to Document
Bibtex

A. Spirina, A. Schmitt, E. Semenkin and W. Minker
Interaction Quality in Human-Human Conversations: Problems and Possible Solutions
Journal of Siberian Federal University. Mathematics & Physics., Vol. 8, Num. 2, pp. 217-223, May 2015
Link to Document
Bibtex

S. Ultes, M. Platero Sánchez, A. Schmitt and W. Minker
Analysis of an Extended Interaction Quality Corpus
Natural Language Dialog Systems and Intelligent Assistants, Springer International Publishing, pp. 41-52, 2015
DOI
Bibtex

M. Sidorov, A. Schmitt and E. Semenkin
Automated Recognition of Paralinguistic Signals in Spoken Dialogue Systems: Ways of Improvement
Journal of Siberian Federal University, Mathematics and Physics, Vol. 8, Num. 2, pp. 208-216, 2015
Link to Document
Bibtex

2014

M. Sidorov, S. Ultes and A. Schmitt
Automatic Recognition of Personality Traits: A Multimodal Approach
Proceedings of the 2014 Workshop on Mapping Personality Traits Challenge and Workshop (MAPTRAITS), Istanbul, Turkey, pp. 11-15, November 2014
Bibtex

M. Sidorov, S. Ultes and A. Schmitt
Comparison of Gender- and Speaker-adaptive Emotion Recognition
International Conference on Language Resources and Evaluation (LREC), Reykjavik, Iceland, pp. 3476-3480, May 2014
Link to Document
Bibtex

M. Sidorov, S. Ultes and A. Schmitt
Emotions Are A Personal Thing: Towards Speaker-Adaptive Emotion Recognition
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Florence, Italy, pp. 4836-4840, May 2014
Bibtex

2013

M. Sidorov, A. Schmitt, S. Zablotskiy and W. Minker
Survey of Automated Speaker Identification Methods
Proceedings of the Intelligent Environments Conference (IE), Athens, Greece, pp. 236-239, July 2013
Link to Document
Bibtex

S. Ultes, A. Schmitt and W. Minker
On Quality Ratings for Spoken Dialogue Systems -- Experts vs. Users
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Atlanta, Georgia, pp. 569--578, June 2013
Link to Document
Bibtex

S. Ultes, R. ElChabb, A. Schmitt and W. Minker
JaCHMM: A Java-Based Conditioned Hidden Markov Model Library
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013, Vancouver, British Colombia, Canada, pp. 3213--3217, May 2013
Bibtex

A. Schmitt and W. Minker
Towards Adaptive Spoken Dialog Systems
Springer, Boston (USA), 2013
Link to Document
Bibtex

2012

S. Ultes, A. Schmitt and W. Minker
Towards Quality-Adaptive Spoken Dialogue Management
NAACL-HLT Workshop on Future directions and needs in the Spoken Dialog Community: Tools and Data (SDCTD 2012), Association for Computational Linguistics, Montreal, Canada, pp. 49--52, June 2012
Link to Document
Bibtex

A. Schmitt, S. Ultes and W. Minker
A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let's Go Bus Information System
International Conference on Language Resources and Evaluation (LREC), Istanbul, Turkey, pp. 3369--3373, May 2012
Link to Document
Bibtex

S. Ultes, A. Schmitt, R. ElChabb and W. Minker
Statistical Modeling of Interaction Quality in Spoken Dialogue Systems: A Comparison of (Conditioned) Hidden-Markov-Model-based Classifiers vs. Support Vector Machines
Bulletin of Siberian State Aerospace University named after academician M.F. Reshetnev, 2012
Bibtex

2011

S. Ultes, A. Schmitt and W. Minker
Attention, Sobriety Checkpoint! Can Humans Determine by Means of Voice, if Someone is Drunk... and can Automatic Classifiers Compete?
Proc. of the 12th Annual Conference of the International Speech Communication Association, Florence, Italy, pp. 3221--3224, August 2011
Bibtex

A. Schmitt, A. Zgorzelski and W. Minker
Tackling a Shilly-Shally Classifier for Predicting Task Success in Spoken Dialogue Interaction
Proc. of the International Conference on Speech and Language Processing (ICSLP), Florence (Italy), August 2011
Bibtex

S. Ultes, T. Heinroth, A. Schmitt and W. Minker
A Theoretical Framework for a User-Centered Spoken Dialog Manager
Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop, Springer New York, New York, NY, Granada, Spain, pp. 241--246, 2011
DOI
Bibtex

A. Schmitt, B. Schatz and W. Minker
Modeling and Predicting Quality in Spoken Human-Computer Interaction
Proceedings of the SIGDIAL 2011 Conference, Association for Computational Linguistics, Portland, Oregon, USA, pp. 173--184, 2011
Link to Document
Bibtex

2010

A. Schmitt, M. Scholz, W. Minker, J. Liscombe and D. Suendermann
Is it Possible to Predict Task Completion in Automated Troubleshooters?
Proc. of the International Conference on Speech and Language Processing (ICSLP), Makuhari (Japan), September 2010
Link to Document
Bibtex

A. Zgorzelski, A. Schmitt, T. Heinroth and W. Minker
Repair Strategies on Trial: Which Error Recovery Do Users Like Best?
Proc. of the International Conference on Speech and Language Processing (ICSLP), Makuhari (Japan), September 2010
Link to Document
Bibtex

T. Polzehl, A. Schmitt and F. Metze
Salient Features for Anger Recognition in German and English IVR Portals
Springer, Boston, USA, In: Spoken Dialogue Systems Technology and Design, Chapter 4, pp. 83-105, August 2010
Link to Document
Bibtex

A. Schmitt, U. Tschaffon, T. Heinroth and W. Minker
Inter-Labeler Agreement for Anger Detection in Interactive Voice Response Systems
6th International Conference on Intelligent Environments (IE'10), Kuala Lumpur (Malaysia), July 2010
Link to Document
Bibtex

T. Polzehl, A. Schmitt and F. Metze
Approaching Multilingual Emotion Recognition from Speech - On Language Dependency of Acoustic/Prosodic Features for Anger Detection
Proc. of the Fifth International Conference on Speech Prosody, 2010. Speech Prosody 2010, Chicago, U.S.A., May 2010
Link to Document
Bibtex

T. Heinroth, D. Denich, A. Schmitt and W. Minker
Efficient Spoken Dialogue Domain Representation and Interpretation
Proceedings of the Seventh Conference on International Language Resources and Evaluation (LREC'10), European Language Resources Association (ELRA), Valetta, Malta, May 2010
Link to Document
Bibtex

A. Schmitt, T. Polzehl and W. Minker
Modeling A-Priori Likelihoods for Angry User Turns with Hidden Markov Models
Proc. of the Fifth International Conference on Speech Prosody, 2010. Speech Prosody 2010, Chicago, U.S.A., May 2010
Link to Document
Bibtex

A. Schmitt, T. Polzehl, J. Liscombe and W. Minker
The Influence of the Utterance Length on the Recognition of Aged Voices
International Conference on Language Resources and Evaluation (LREC), Valetta, Malta, May 2010
Link to Document
Bibtex

A. Schmitt, G. Bertrand, T. Heinroth, J. Liscombe and W. Minker
WITcHCRafT: A Workbench for Intelligent exploraTion of Human ComputeR conversaTions
International Conference on Language Resources and Evaluation (LREC), Valetta, Malta, May 2010
Link to Document
Bibtex

T. Heinroth, D. Denich and A. Schmitt
OwlSpeak - Adaptive Spoken Dialogue within Intelligent Environments
8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), Mannheim (Germany), pp. 666 - 671, March 2010
Link to Document
DOI
Bibtex

T. Polzehl, A. Schmitt, F. Metze and M. Wagner
Anger Recognition in Speech Using Acoustic and Linguistic Cues
Speech Communication, Vol. Special Issue: Sensing Emotion and Affect - Facing Realism in Speech Processing, 2010
Link to Document
DOI
Bibtex

T. Polzehl, F. Metze and A. Schmitt
Factors for Linguistic and Prosodic Emotion Recognition
36. Jahrestagung für Akustik (DAGA), DEGA e.V., 2010
Bibtex

A. Schmitt, R. Pieraccini and T. Polzehl
"For Heaven's Sake, Gimme a Live Person!" Designing Emotion-Detection Customer Care Voice Applications in Automated Call Centers
Advances in Speech Recognition, Springer US, pp. 191-219, 2010
Link to Document
Bibtex

A. Schmitt, T. Polzehl and W. Minker
Facing Reality: Simulating Deployment of Anger Recognition in IVR Systems
Spoken Dialogue Systems for Ambient Environments, Springer Berlin / Heidelberg, Series: Lecture Notes in Computer Science, Vol. 6392, pp. 122-131, 2010
Link to Document
Bibtex

A. Schmitt, W. Minker and N. Sharaf
Advances in the Witchcraft Workbench Project
Proceedings of the SIGDIAL 2010 Conference, Association for Computational Linguistics, Tokyo, Japan, pp. 261--264, 2010
Link to Document
Bibtex

2009

T. Polzehl, A. Schmitt and F. Metze
Comparing Features for Acoustic Anger Classification in German and English IVR Portals
First International Workshop on Spoken Dialogue Systems (IWSDS), Kloster Irsee (Germany), December 2009
Link to Document
Bibtex

S. Mowafey, A. Schmitt, H. Hagras and W. Minker
Creating an Ambient Intelligent Environment with an Emotion-Aware System
Proceedings of the 5th International Conference on Intelligent Environments, IOS Press, Barcelona (Spain), September 2009
Bibtex

T. Heinroth, A. Schmitt and G. Bertrand
Enhancing Speech Dialogue Technologies for Ambient Intelligent Environments
5th International Conference on Intelligent Environments (IE'09), IOS Press, Barcelona (Spain), Series: Ambient Intelligence and Smart Environments, Vol. 2, pp. 42-49, July 2009
Link to Document
DOI
Bibtex

A. Schmitt, T. Heinroth and G. Bertrand
Towards Emotion, Age- and Gender-Aware VoiceXML Applications
5th International Conference on Intelligent Environments (IE’09), IOS Press, Barcelona (Spain), Series: Ambient Intelligence and Smart Environments, Vol. 2, pp. 34-41, July 2009
Link to Document
DOI
Bibtex

G. Bertrand, T. Heinroth and A. Schmitt
CHAD - Constraint Handling Architecture for Dialoguemanagement
5th International Conference on Intelligent Environments (IE'09), IOS Press, Barcelona (Spain), Series: Ambient Intelligence and Smart Environments, Vol. 2, pp. 50-56, 2009
Link to Document
DOI
Bibtex

A. Schmitt, D. Zaykovskiy and W. Minker
Speech Recognition for Mobile Devices
International Journal of Speech Technology, Vol. 11, Num. 2, pp. 63--72, 2009
Link to Document
DOI
Bibtex

A. Schmitt, T. Heinroth and J. Liscombe
On NoMatchs, NoInputs and BargeIns: Do Non-Acoustic Features Support Anger Detection?
Proceedings of the SIGDIAL 2009 Conference, Association for Computational Linguistics, London, UK, pp. 128--131, 2009
Link to Document
Bibtex

2008

O. Herm, A. Schmitt and J. Liscombe
When Calls Go Wrong: How to Detect Problematic Calls Based on Log-Files and Emotions?
Proc. of the International Conference on Speech and Language Processing (ICSLP), Brisbane, Australia, September 2008
Link to Document
Bibtex

J. Pittermann and A. Schmitt
Integrating Linguistic Cues Into Speech-Based Emotion Recognition
4th IET International Conference on Intelligent Environments, Seattle (USA), July 2008
Link to Document
Bibtex

J. Pittermann, A. Schmitt and W. Minker
Comparing Evaluation Criteria for (Automatic) Emotion Recognition
4th IET International Conference on Intelligent Environments, Seattle (USA), pp. 1-4, July 2008
Link to Document
Bibtex

D. Zaykovskiy and A. Schmitt
Java vs. Symbian: A Comparison of Software-based DSR Implementations on Mobile Phones
4th IET International Conference on Intelligent Environments, Seattle (USA), July 2008
Link to Document
Bibtex

D. Zaykovskiy and A. Schmitt
Deploying DSR Technology on Today's Mobile Phones: A Feasibility Study
4th IEEE Tutorial and Research Workshop Perception and Interactive Technologies for Speech-Based Systems, Irsee (Germany), June 2008
Link to Document
Bibtex

A. Schmitt, C. Hank and J. Liscombe
Detecting Problematic Calls With Automated Agents
4th IEEE Tutorial and Research Workshop Perception and Interactive Technologies for Speech-Based Systems, Irsee (Germany), June 2008
Link to Document
Bibtex

2007

D. Zaykovskiy, A. Schmitt and M. Lutz
New Use of Mobile Phones: Towards Multimodal Information Access Systems
3rd IET International Conference on Intelligent Environments, Ulm (Germany), September 2007
Link to Document
Bibtex

D. Zaykovskiy and A. Schmitt
Java to Micro Edition Front-End for Distributed Speech Recognition Systems
The 2007 IEEE International Symposium on Ubiquitous Computing and Intelligence (UCI'07), Niagara Falls (Canada), May 2007
Link to Document
Bibtex

Lectures
Embedded Aigaion Query 2012

W. Minker, H. Lang, A. Schmitt, S. Ultes and F. Nothdurft
Assistive and Adaptive Dialogue Systems
Graduate Course within the ERASMUS/SOCRATES Mobility Programme, Department of Computer Science, University of Granada (Spain), June 2012

2011

W. Minker, T. Heinroth and A. Schmitt
Dialogue Success Estimation and Adaptive Spoken Dialogue Management
Graduate Course within the ERASMUS/SOCRATES Mobility Programme, Department of Computer Science, University of Granada (Spain), May 2011

2009

W. Minker, A. Schmitt, A. Schmeiser and D. Zaykovskiy
DAAD Summer Course: Speech-Based Human-Computer Interfaces
Ulm, Germany, August 2009

W. Minker, A. Schmitt and S. Zablotskiy
Spoken Language Dialogue Systems
Graduate Course within the ERASMUS/SOCRATES Mobility Programme, Department of Computer Science, University of Warsaw (Poland), May 2009

2008

W. Minker, A. Schmitt and A. Schmeiser
Advanced Media
Compact Course: German University in Cairo (Egypt), October 2008

2007

W. Minker and A. Schmitt
Introduction to Spoken Language Dialogue Systems
Compact Course: German University in Cairo (Egypt), October 2007

W. Minker, A. Schmitt and P. Strauss
Spoken Dialogue Technology - Current Themes in Academic Research
Summer School, ELSNET Summer School on Advanced Dialogue Systems: Affectivity, Adaptability and Multimodality, Belfast (Northern Ireland), July 2007

W. Minker, A. Schmitt, A. Schmeiser and B. Wiegel
DAAD Summer Course: Speech-Based Human-Computer Interfaces and Multimedia Networks
August

In Progress / Completed
Embedded Aigaion Query 2010

B. achelor Thesis
A Java Workbench for Prediction Model Analysis for Spoken Dialogue Systems
2010
Link to Document

M. aster Thesis
Online Detection of Interaction Quality in Spoken Dialogue Systems
2010
Link to Document

M. aster Thesis
Towards Intelligent Voice User Interfaces: Online Call Quality Estimation for Interactive Voice Response Systems
2010
Link to Document

2009

B. achelor Thesis
A Sliding Window Approach to Statistical Problematic Call Recognition
2009
Link to Document

B. achelor Thesis
Detection of Angry Callers with Hidden Markov Models
2009
Link to Document

B. achelor Thesis
Rolling out graphical user interfaces in Arabic
2009
Link to Document

B. achelor Thesis
Voice user interface in Arabic
2009
Link to Document

M. aster Thesis
Speech-based Emotion Recognition for Interactive Voice Response Systems
2009
Link to Document

2008

M. aster Thesis
Masking Noise for Automatic Speech Recognition
2008
Link to Document

M. aster Thesis
Verteiltes Spracherkennungssystem zur Verwendung von verschiedenen Onlinediensten
2008
Link to Document

2007

M. aster Thesis
Detecting problematic Phone Calls in an SLDS-based Call-Center using Machine Learning Approaches
2007
Link to Document

Talks

External Research Associate