ituu_logo
Verlauf

Dialogue Systems Project

Description

Lecturer: Prof. W. Minker
Supervisors: Members of the Dialogue Systems Group
Program of Studies: Master Elektrotechnik, Communications Technology, Informationssystemtechnik, Informatik, Medieninformatik
Hours: 0/0/6/0 (8 ECTS)
Contents: Research in multimodal spoken language dialogue systems is performed on the basis of end-to-end systems including components for acoustic processing, speech signal analysis, recognition, spoken natural language understanding, dialogue processing and speech synthesis. In the framework of this component development several individual topics are proposed as practical studies. They may depend on the current development status of the prototype system demonstrator of the Dialogue Systems Group. Each student/group will work on one of the proposed topics.
Semester: Summer (ENGC8012)/Winter (ENGC7012)
Requirements: Bachelor. No prerequisites from other lectures required. Some programming (Java, C, C++) and operating systems (Unix/Linux) knowledge would be helpful. Due to the limited number of participants, preference is given to students having attended the lectures Automatic Recognition of Speech and Emotion, Benutzerschnittstellen, Dialogue Systems or Natural Language Understanding and Dialogue Management. Suitable for students from other university departments.
Language: Englisch / German
Exam: Certificate after fulfillment of the following criteria: at least three meetings with the supervisor throughout the semester to discuss progress of work; final presentation (demo) and discussion; short description and illustration (max. 10 pages); submission of the final version (hard/software and documentation) of the project to the supervisor.
Additional Information:

Topics announced before the beginning of each semester. Information and registration by sending an email to the respective topic supervisor.
Link to Hochschulportal

Open Topics

Online Emotion Recognition

Supervisor:

S. Ultes

Description:

As words and their semantics are not the only information held by human speech, research moved its focus towards emotion recognition. Various approaches for offline emotion recognition exist by now. Based on this work, an online emotion recognizing unit has to be implemented and tested.

(1 Person)

Requirements:
  • Good JAVA Programming Skills
  • Interest in Machine Learning 

Clustering Algorithm Investigation

Supervisor:

T. Gasanova

Description:

Clustering algorithms have a lot of important applications in real life. There are different approaches having their own advantages and disadvantages. The task of this work is to test the performance of the chosen algorithm varying its parameters on different datasets.

The algorithms to chose from:

  • Artificial neural networks
  • Non-parametric models
  • Genetic algorithm
  • Classical approaches (Hierarchical clustering, k-means, etc.)

(4 Persons)

Requirements:
  • Good Programming Skills in C++

Multiobjective genetic programming

Supervisor:

S. Sopov

Description:

Multiobjective genetic programming is a new and perspective approach for the intellectual analysis in modeling and developing of dialogue systems. Now you can participate in improvement of this approach. The following themes are suggested:

1) To develop the universal multiobjective algorithm with any number of criteria.

2) To develop the universal genetic programming algorithm with the possibility of using any kind of the functional and term sets.

(2 Persons)

Requirements:
  • to be ready to solve difficult but interesting problems;
  • programming skills in any OOP language;
  • basic knowledge in the module programming;
  • project result should be implemented as a dll-module with corresponding manuals.

SS 2012

Analyzing and Adapting Let's Go Dialogue to OwlSpeak

Supervisor:

S. Ultes

Description:

Probably the only research dialogue systems applied in real life is the Let's Go system of Carnegie Mellon University in Pittsburgh. People can call the system to get information about public transportation. The task of this work is to analyze the system descriptions and to adapt the dialogue for the ontology-based OwlSpeak dialogue manager.

(1 Person)

Requirements:
  • Good JAVA Programming Skills
  • Good Programming Skills in a Scripting Language, e.g. Perl
  • Interest in Dialogue Systems

WS 2011/2012

1. A Java Workbench for Analyzing Large Dialogue Corpora 

Supervisor:

A. Schmitt

Description:

The task of this project is the implementation of new features for our Eclips-RCP-based Java workbench that allows for a detailed analysis of logged dialogue conversations. Furthermore, the current project is to be enhanced in such a way that it can be made publically available on an open-source platform.

(1 Person)

Requirements:

Strong Java programming skills, interest in Spoken Language Dialogue Systems, interest in Machine Learning

2. Providing dialogue acts for usage in different dialogue strategies 

Supervisor:

G. Bertrand

Description:

Adapting dialogues to user emotions, profile and the current situation is a complex task, which is needed to create a more human-like dialogue system. For testing how to adapt dialogues in an appropriate way, we want to do Wizard-of-Oz Experiments (Experiments where parts of the system are simulated by a human person). In order to be able to conduct Wizard-of-Oz experiments, we need a framework, where the wizard (human person) who simulates the system part, can choose which dialogue fragment is appropriate for the user. In this Bachelor Thesis the student can design a database which provides different dialogue acts for coping with different situations in user interaction. Different ways of interacting with the user via speech should be designed and implemented in a database which will be the foundation of a Wizard-of-Oz system.

(1 Person)

Requirements:

Interest in Spoken Language Dialogue Systems, basic knowledge in database programming

3. Server interface for presenting adaptive dialogue strategies 

Supervisor:

G. Bertrand

Description:

Adapting dialogues to user emotions, profile and the current situation is a complex task, which is needed to create a more human-like dialogue system. For testing how to adapt dialogues in an appropriate way, we want to do Wizard-of-Oz Experiments (Experiments where parts of the system are simulated by a human person). In order to be able to conduct Wizard-of-Oz experiments, we need a framework, where the wizard (human person) who simulates the system part, can choose which dialogue fragment is appropriate for the user. In this Bachelor Thesis the student has the opportunity to create a servlet using state-of-the-art servlet technologies which is able to receive queries for adaptive dialogue acts. The servlet must have access to a database which contains dialogue fragments for creating different dialogue strategies. The servlet presents the information from the database in an xml-conform file format so that the client can interpret the information.

(1 Person)

Requirements:

Basic knowledge in web technologies, programming skills in Java, familiarity with XML technologies

4. User Interface for simulating adaptive dialogue strategies 

Supervisor:

F. Nothdurft

Description:

Adapting dialogues to user emotions, profile and the current situation is a complex task, which is needed to create a more human-like dialogue system. For testing how to adapt dialogues in an appropriate way, we want to do Wizard-of-Oz Experiments (Experiments where parts of the system are simulated by a human person). In order to be able to conduct Wizard-of-Oz experiments, we need a framework, where the wizard (human person) who simulates the system part, can choose which dialogue fragment is appropriate for the user. In this Bachelor Thesis the student has to design an User Interface for the Wizard. The User Interface should communicate to the server side, where the dialogue fragments which are stored in a database, and create the system utterance, which is presented via speech to the user. The User Interface should be able to let the wizard choose the system utterance according to emotional, situational states and the user profile.

(1 Person)

Requirements:

Interest in Spoken Language Dialogue Systems, basic knowledge in client-server-communication, programming skills in User Interfaces and XML-Technologies

5. Mapping Between Portlets and JSP Pages

Supervisor:

H. Lang

Description:

Java Server Pages (JSR-245) allow the integration of java code into html documents. A set of those server pages along with a number of additional java classes make up a Portlet (JSR-168). Usually the jsp files belonging to a Portlet are referenced from within compiled java classes. Hence there is no explicit mapping between a Portlet and its associated jsp files. Since such a mapping is indispensable for an avatar based help system currently developed within the scope of the bwGRiD portal project, a way to implement a Portlet to jsp file mapping has to be devised and integrated into the existing portal framework.

(1 Person)

Requirements:

Profound knowledge of Java, interest in J2EE web applications, basic knowledge of web standards and XML

6. Model-based testing of user interfaces for infotainment applications 

Supervisor:

H. Hüning

Description:

The focus work will be the generation of test cases from the model-based specification for user interfaces in cars. The functionality of the user interfaces is increasing rapidly, so the time for writing correct specifications is critical. In order to be able to validate the specifications, we investigate the model-based format of UML 2.0 state charts. We develop tools for a simulation from these state charts, including the behaviour of screen elements.

(1 Person)

Requirements:

Background in computer science, engineering or similar, Programming skills (C++, Java, ActionScript, …), Strong interest in model-based specification and testing of user interfaces, Analytic skills and ability to work single-handedly

SS 2011

1. Model-based testing of GUIs for in-car infotainment (GUC internship)

Supervisor:

H. Hüning

Description:

The team "HMI Implementation" at Daimler in Ulm, located next to the University of Ulm, is working on model-based specification methods and testing of user interfaces, or human-machine interfaces (HMI). The user interfaces are for the so-called head-units in Mercedes passenger cars, which integrate infotainment applications such as audio, telephone and navigation. The user interfaces typically comprise a screen, multi-functional buttons, and in some cases speech dialogues (i.e. multi-modal user interfaces). At Daimler, specifications of the user interfaces need to be written, and head-unit devices from suppliers need to be tested for conformance to the specifications. We develop tools for model-based testing of the user interfaces, so both the specification and test models can be derived from a system model. A test case is a particular path through a UML state chart. The aims of testing are twofold, first to check the model, and second to compare the behaviour of the model to the head-unit implementation. Tools for model based testing are either extensions of a UML tool like Enterprise Architect or combinations of tools like EB Guide and Conformiq (see URLs below). Such tools will be provided to the student, or, as a fallback possibility, we have our own source code for automatic test case generation from UML state charts. The focus of this bachelor thesis is planned to be the correct design of test models, such that test cases can be generated automatically. Either, another student will work on automatic test generation in parallel, or this thesis may comprise more aspects of automatic test generation. Challenges of this work will be: A systematic way to deal with different goals of testing, definition of formats for all necessary elements in the test models, validation of the results, and study of literature on model-based testing of user interfaces.

(1 Person)

Additional Information:
  1. For an overview of model-based testing and test case generation see: http://en.wikipedia.org/wiki/Model-based_testing
  2. For a UML tool enterprise architect see http://sparxsystems.eu/
  3. For the EB Guide example see http://www.elektrobit.com/what_we_deliver/automotive_software/products/eb_guide_-_hmi_development
  4. Conformiq http://www.verifysoft.com/en_conformiq_automatic_test_generation.html
Requirements:Background in computer science or engineering Programming skills (C#, java, C++ or similar, and XML) Language skills, fluent English or German, strong interest in model based testing, ability to work in a team, and single-handedly

2. Visualization of Adaptive Dialoguemodel

Supervisor:

G. Bertrand

Description:

This work is concerned with a displaying adaptive dialogue acts (parts of a spoken human machine dialogue) in 3D. That means to transform parts of an adaptive dialogue model (XML-Format) into a corresponding 3D Coordinate representation. For this purpose the student may take advantage of jzy3d or any other 3D library he/she finds appropriate.

(1 Person)

Requirements:Programming skills (Java/XML), interest in dialogue modeling.

3.  Intoxication Recognition with Hidden Markov Models (University of Toronto internship)

Supervisor:

S. Ultes

Description:

Speech not only contains of spoken words bud holds more information about the speaker. In this task, a Hidden Markov Model (HMM) should be used to find out if a speaker is drunk or sober. For that, a suitable feature set based on Automatic Speech Recognition has to be found and the topology of the HMM has to be designed.

(1 Person)

Requirements: 
Interest in statistical pattern recognition

WS 2010/2011

1. Design and Evaluation of a New Way of Modeling Assistive and Adaptive Dialogue

Supervisor:

G. Bertrand

Description:

In connection with the SFB-TRR62 a new form of Dialogue Modeling has been developed. It reflects assistivity and adaptivity by providing new ways to integrate emotional strategies and explanatory dialogues into the dialogue model. The model itself is based on state-of-the-art technologies (Eclipse Modeling Framework, Eclipse Graphical Modeling Framework and above all Java).


The task related to this new model is to design a dialogue in a given domain (there already is the opportunity to use a graphical modeling tool) and to evaluate this dialogue (meaning to write some test code to prove its usability). Because of the novelty of our model we also welcome ideas to improve the model and to refine it.


The task is intended for two students but can be reduced for a single student in its breadth. 

((1-)2 Persons)

Requirements:

For the task a firm knowledge of Java programming is required and some experience with the Eclipse Platform is recommended.

2. Iterative Linear Regression for User Capabilities Determination

Supervisor:

K. Zablotskaya

Description:

The objective of this work is to create a linear model for determination of speakers’ capabilities.  The capability score is calculated as a weighted sum of linguistic features extracted from monologues of speakers. A linear regression with Lagrange restriction must be implemented for determination of the weights in the linear model.

(1 Person)

Requirements:

analytical skills, programming skills (MatLab C/C++)

3. Connecting Knowledge Base and Dialogue Management

Supervisor:

F. Nothdurft

Description:

The focus of this work will be the design and implementation of an exemplary intermodule communication in multimodal dialogue systems, and in particular the information exchange between dialogue management and knowledge-base. The communication will be based on the SEMAINE API, an open source framework for building emotion-oriented systems.

As this work is related to the SFB-TRR62 project, the student will be part of up-to-date scientific work on companion technologies.

(1 Person)

Requirements:

Programming skills (Java). Interest in developing ideas for dialogue systems. Dialogue Systems lecture would be good.

SS 2010

1. Affective Grammar(s) - Emotion Recognition from Texts (GUC internship)

Description:

The main objective is the development of an affective grammar that is able to determine a user’s emotional state by spotting “emotion words” within utterances. The grammar should be written in theW3C grxml format to be included in a VoiceXML-based dialogue system. Optionally, speech-signal based emotion recognition shall be investigated. The recognizer output shall be integrated in a dialogue system which optionally shall be evaluated in user tests.

(1 Person)

Requirements:

Programming skills (Java), interest in model-based specification and development of user interfaces

2. Symbian C++ Client for Distributed Spoken Language Dialogue Systems (GUC internship)

Description:

The objective is the implementation of Symbian C++ client for distributed spoken language information retrieval system. The focus of the work should be placed on the development of the front-end for Distributed Speech Recognition (DSR) systems. The front-end extracts characteristic features out of speech signal, sends them to the DSR server, gets the recognized utterance back and presents the result in appropriate form to the user.

(1 Person)

3. Speaker Adaptation in Automatic Speech Recognition (GUC internship)

Description:

Speech recognition is becoming a standard interface in the human-machine communication. However, the performance of the automatic speech recognition (ASR) systems is highly affected by the mismatch in training and testing data. The differences in the vocal tract of speakers, articulation, microphone transfer function, etc. lead to performance degradation. The study of adaptation and normalization techniques to increase the reliability of an ASR-system and further implementation of these algorithms constitute the main objective of this work.

(1 Person)

4. Analysis of Term and Sentence (Dis)Similarity Measures for Unsupervised Categorisation of Spoken Language Utterances (GUC internship)

Description:

The main objective is to improve the unsupervised categorization of speech utterances by analysing new metrics for capturing similarities (or distances) between the utterances. Since components of speech utterances are the words, the purpose is to carry the mentioned analysis to the word level, i.e. search for new formulations that provide the semantic affinities between different words. If the semantic similarity among words is appropriately captured, it can be used to estimate the similarities between utterances whose component words are not identical.

(1 Person)

5. Dialogue Modelling in Multi-User Spoken Language Dialogue Systems (GUC internship)

Description:

This work deals with dialogue modelling for a multi-party spoken language dialogue system. Each user utterance is integrated in the dialogue model as a new dialogue state with information such as the semantic content, a dialogue act, speaker and addressee. The system then decides upon this dialogue state how to further proceed in the dialogue using update rules. These rules would e.g. provoke that the information
is stored in a so-called task model which describes the constraints for the database queries. Task model and update rules have to be developed and implemented.

(1 Person)

WS 2009/2010

1. Hidden Markov Models in RapidMiner

Supervisor: A. Schmitt
Description:

Hidden Markov Models (HMMs) are considered to be the most powerful machine learning algorithms for speech recognition and other time-variant learning tasks. RapidMiner, the popular machine learning framework, however, does not provide a possibility to integrate HMMs. In this project, a machine learning operator will be implemented based on RapidMiner (open-source Java) and JAHMM (a Java HMM implementation of Hidden Markov Models). To proove the functionality, a simple speech-based gender recognizer will be realized in RapidMiner.

(1-2 Persons)

Requirements:

Java programming skills, interest in Spoken Language Dialogue Systems, interest in Machine Learning

2. Repair Strategies for Advanced Spoken Dialogue Systems

Supervisor: T.Heinroth & A. Schmitt
Description:

Within advanced spoken dialogue domains such as preparing a dinner together with an autonomous Intelligent Environment various fallback and repair strategies are neccessary when the task is about to fail. In this project the main task is to explore already existing approaches towards such dialogue enhancements. Several different approaches should be examined and the most promising approach will be implemented within e.g. VoiceXML.

(1 Person)

Requirements:

Interest in Spoken Language Dialogue Systems, some VoiceXML skills

SS 2009

1. Development of a Call Browser in Java

Supervisor: A. Schmitt
Description:

With growing task complexity of telephone-based speech applications, we develop new tools that support us in analyzing human machine dialogues. Especially in larger conversations where dialogs not rarely consist of 50 system and user turns or even more, we quickly loose track of how the conversation between the user and the system happened. We are thus developing a framework for Call Analysis in Java, which allows for a detailed exploration of call center calls.

You can contribute to this framework by implementing a Call Browser Plugin that enables to visualize the path, the user took within the conversation in a tree-like structure. The tree can be built based on log-data. Our framework is implemented on Eclipse RCP basis and the plugin itself should be an Eclipse Plugin.

Requirements:

Java programming skills, some SQL skills, interest in Spoken Language Dialogue Systems

2. Age Recognition

Supervisor: A. Schmitt
Description: We as humans are basically able to determine gender and the rough age of persons that we hear over telephone. We also want to train computers to be able to determine age and gender of callers in order to change the dialogue strategy of voice computers. The task of this project is to implement an age recognizer based on acoustic data within the RapidMiner framework.
Requirements:

Java programming skills, some SQL skills, interest in Spoken Language Dialogue Systems and Machine Learning

3. Using genetic algorithms to solve the cluster ensemble problem

Supervisor: A. Albalate
Description:

Cluster ensembles refers to the combination of different cluster solutions for a data set that yields a more robust partition of the data.
Several quality criteria have been proposed for detecting the optimum solution, consisting, for example, of maximising the average mutual information (ANMI) across the pool of clusterings. In this project, the student will address the optimisation problem through greedy approaches, in particular Genetic algorithms (GA). The GA efficiency vs. the dataset size is also a fundamental issue to be finally investigated.

Requirements:

The proposed approach should be implemented in R software/Java. This project can be easily extended for a Master/Bachelor thesis by considering the efficiency aspect of the implemented genetic algorithm. In this context,  hybrid solutions with SVM clustering or Principal-Component-Projection clustering could be explored.

4. Semi-supervised co-training of classifiers with prior clustering

Supervisor: A. Albalate
Description: Supervised classifiers are algorithms trained with large data, previously labeled by humans. The high labeling costs are however, an important factor wich becomes prohibitive for the rapid adaptation or portability of the systems using these classifiers. In this project, the student will develop a semi-supervised algorithm with only makes use of few labeled data. To this aim, clustering algorithms or cluster ensembles should be used to provide a partition of unlabelled data in an unsupervised manner. Then, a small subset of data labels will be used to automatically annotate the obtained clusters. The automatic cluster annotation can be modeled as the "Travelling salesman" problem, for which greedy approaches (i.e., Genetic Algorithms) as well as other heuristics  are applied to optimise a quality parameter. Finally, some classifiers (SVM classifier and nearest neigbour classifier)  will be trained with the automatically expanded annotations and tested.

Requirements:

The project should be implemented in R software/Java. The project can be extended for a bachelor thesis, by conducting a more exhaustive evaluation, or by increasing the evaluated set of classifiers.

5. Hiper-graph  and similarity-based techniques for cluster ensembles.

Supervisor:A. Albalate
Description:The aim of this project is to implement and evaluate several existing techniques for cluster ensemble, such as hiperedge paritioning or by constructing a new similarity matrix which reflects the consensus  between the different partitionings. A weighted modification of the techniques by considering the relative importance of the combined clusterings is to be introduced. 
Requirements:

The project can be implemented in R software/Java.

6. Evaluating the number of clusters in a data set via fuzzy clustering methods

Supervisor: A. Albalate
Description: The objective of this project consists of the implementation and test of formulations for assessing the quality of a cluster partition obtained from a clustering algorithm. Numerous schemes exist already for assessing hard clustering, but few of them have been applied to fuzzy clustering approaches. This do not only return a partition of the input objects but also a vector of memberships of each object into each cluster. The project consists of extending some of these traditional methods for assessing the output of fuzzy clustering algorithms.
Requirements: The software will be implemented for the use in the statistics R sotware (similar to matlab functions .m)

WS 2008/2009

1. Detection of Angry Callers in a Call Centre

Supervisor: A. Schmitt
Description: In this project, you will implement an emotion recognizer, that is able to determine angry caller utterances in call centre dialogues based on acoustic data. With that knowledge, we will be able to transfer such angry callers to human agents. The emotion recognizer will be based on RapidMiner, a machine learning and data mining tool implemented in Java. You will train classification models within RapidMiner based on several thousand utterances from a US-american call centre (1-2 Persons, can be combined with gender and age topic).
Requirements: Interest in Machine Learning, Emotion Recognition, Dialogue Systems. Knowledge in Java and Machine Learning/Neural Networks advantageous



2. Gender and Age Detection of Callers

Supervisor: A. Schmitt
Description: We as humans are basically able to determine gender and the rough age of persons that we hear over telephone. We also want to train computers to be able to determine age and gender of callers in order to change the dialogue strategy of voice computers. You will implement a gender and age recognizer based on acoustic data within the RapidMiner framework (1-2 Persons, can be combined with emotion recognition topic).
Requirements: Interest in Machine Learning, Emotion Recognition, Dialogue Systems. Knowledge in Java and Machine Learning/Neural Networks advantageous



3. Voice Control Simulator

Supervisor: T. Heinroth
Description: Aim is to implement a simulator for controlling lights, heater, and blinds by the use of speech. The simulator should view a room or house with several devices. It should provide an interface for a command and control dialogue running on a speech server (asynchronous communication) and several Java Servlets.
Requirements: Good Flash/Java Servlet/VoiceXML skills.



4. Evaluating the number of clusters in a data set via fuzzy clustering methods

Supervisor: A. Albalate
Description: The objective of this project consists of the implementation and test of formulations for assessing the quality of a cluster partition obtained from a clustering algorithm. Numerous schemes exist already for assessing hard clustering, but few of them have been applied to fuzzy clustering approaches. This do not only return a partition of the input objects but also a vector of memberships of each object into each cluster. The project consists of extending some of these traditional methods for assessing the output of fuzzy clustering algorithms.
Requirements: The software will be implemented for the use in the statistics R sotware (similar to matlab functions .m)



5. Using clustering and classification schemes for determining the optimum number of clusters

Supervisor: A. Albalate
Description: The objective of this project is similar to the previous one: determining the optimum number of groups in a data set without prior knowledge. For this reason, clustering algorithms will be used in combination with second classification methods (e.g. Neural networks, Bayes classifier, etc...). The result of the clustering scheme will be implicitely label. The optimum can be discerned as the result which provides, according to the implicit labels, the best classification performance.
Requirements: Tools will be implemented either in Java and/or R (similar to matlab functions .m)



6. Implementation of the Pole based overlapping clustering (POBOC) algorithm to R

Supervisor: A. Albalate
Description: The Pole Overlapping Clustering is a fuzzy clustering algorithm which provides, for each input object, a pattern of memberships into each output cluster. This software is, to the date, implemented in Java. The project consists of porting the software into the programming language of the R statistics package. The implemented material will be finally tested with some existing data-sets.
Requirements: Knowledge of Java and R (similar to matlab functions .m) programming languages are desired



7. Hierarchical Implementation of the Pole based overlapping clustering (POBOC)

Supervisor: A. Albalate
Description: The Pole Overlapping Clustering is a fuzzy clustering algorithm which provides, for each input object, a pattern of memberships into each output cluster. This method is also capable to detect the number of clusters and does not need that the user provides this information. However, the algorithm relies in an input parameter, the global average distance of all objects, and therefore may not provide accurate solutions if the input objects are organised in a hierarchy of distances. The student will be required to use the basis software in iterative calls using local distances, in order to detect the underlying hierarchy of clusters. This project can be implemented in java by one person, using the provided basis Java class, or may be also suitable for two students if selecting the block of this project with the the previous offered project.
Requirements: Knowledge of Java and R (similar to matlab functions .m) are necessary