Artificial Neural Networks (ANN)

People

• Prof. Dr. Günther Palm
  Stefan Faußer
  Michael GlodekDr. Hans Kestler
  Martin Schels
  Stefan SchererMiriam SchmidtDr. Friedhelm SchwenkerMohamed Farouk Abdel Hady
  

Alumni

• Zöhre Kara Kayikci
  Christian Thiel

Overview

• Pattern Recognition
• Radial Basis Function Networks
• Learning in Neural Networks
• Sensor Fusion
• Hierarchical Neural Networks
• Neural Associative Memory
• Sparse Coding
• Data Mining
• Cluster Analysis
• Signal Processing
• Statistical Learning Theory
• Automated Bioacoustics

Research Topics

Pattern Recognition

Pattern recognition includes a wide range of information processing problems, for example optical character recognition, face recognition speech recognition, medical diagnosis, etc. Almost all humans solve many of these problems very easily, however, computers have big difficulties to get good solutions. The field of pattern recognition has been developed over the last 40 years and is a still growing direction of research (e.g., data mining). The theoretical basis of pattern recognition is in statistics.

Research directions:

• artificial neural networks for pattern recognition
• hierarchical classifiers
• speech recognition; speaker identification
• audio-visual pattern recognition (lip reading)
• 3D object recognition
• classification of high resolution ECGs
• optical character recognition of mathematical formulas

Radial Basis Function (RBF) Networks

RBF networks were introduced into the neural network literature by Broomhead/Lowe and Poggio/Girosi in the late 1980s. The RBF network model is motivated by the locally tuned response observed in biologic neurons, e.g. in the visual or in the auditory system. RBFs have been studied in multivariate approximation theory, particularly in the field of function interpolation. The RBF neural network model is an alternative to multilayer perceptron which is perhaps the most often used neural network architecture.

Research directions:

• learning in RBF networks
• initalization techniques of RBF networks
• hierarchical RBF networks
• pattern recognition with RBF networks
• time series prediction with RBF networks

Learning in Neural Networks

One of the most interesting properties of a neural network is the ability to learn from its environment in order to improve its performance ( measured through a predefined performance measure) over time. Learning in an artificial neural network stands for an iterative process of adjusting the synaptic weights and threshold values. In artificial neural networks the following learning procedures can be distinguished:

• supervised learning (teacher or target is available)
• unsupervised learning (without teacher)
• reinforcement learning (without target, but with reward)

Learning in artificial and biological neural networks is a central field of research in our department.

Sensor Fusion

Sensor fusion is of interest in various fields of artificial neural networks and artificial intelligence, for example in pattern recognition and autonomous mobile robotics. Key issues are: How and where to combine the information from multiple sensors to improve the accuracy or confidence?

Research directions:

• neural architectures for sensor fusion
• audio-visual pattern recognition
• sensor fusion and self-supervised learning

Hierarchical Neural Networks

Hierarchical neural networks consist of multiple neural networks concreted in a form of an acyclic graph. Tree-structured neural architectures are a special type of hierarchical neural network. The networks within the graph can be single neurons or complexer neural architectures such as multilayer perceptrons or radial basis function networks. Decision trees, hierarchical self-organizing maps, hierarchies of experts, hierarchical or tree-based classifiers are typical applications for hierarchical neural networks.

Research directions:

• hierarchical classification (LVQ-, RBF-, and SVM networks)
• construction of classifier/class hierarchies
• applications of hierarchical classifiers: recognition of 3D objects, optical character recognition, gesture recognition

Neural Associative Memory

Sparse Coding

Sparse coding is the development of a distributed binary representation of arbitrary entities. It needs two requirements: (i) sparseness means that there are many more 0s than 1s in the codewords. (ii) Given a notion of similarity or dissimilarity on the set of entities, for example a distance or metric, the code should be similarity preserving, i.e., similar entities should set similar codewords (with respect to the Hamming distance). We are working on the development of such codes in various applications (e.g., words, images, speech). They are particularly useful for associative memories.

Data Mining

Most of the information this is stored on computer databases is in a rather raw form. There is a huge amount of information in such databases that may potentially be important, but has not yet been discovered. Data Mining is the extraction of implicit, previously not known, but perhaps useful information from raw data. The idea of data mining is to build some kind of intelligent computer programs searching for regularities or patterns. Data mining algorithms have to be robust in order to deal with noisy, fuzzy and incomplete data. The technical basis of data mining is machine learning including methods from clustering, classification, regression, visualization, decision trees, and artificial neural networks (ANN).

Research directions:

• visualization of high dimensional data
• cluster analysis with ANN and fuzzy clustering
• time series prediction with ANN

Cluster Analysis

Cluster analysis is a method for exploring the underlying structure of large and often high-dimensional sets of feature vectors that does not require further assumptions or a priori knowledge. Given a predefined distance or similarity measure, a cluster of feature vectors is defined as collection of vectors which are similar. Cluster analysis is based on statistical methods and has been developed over the past 40 years. Clustering methods are applied in data mining.

Research directions:

• visualization of high-dimensional data
• dynamic clustering
• hierarchical clustering
• selfsupervised learning
• fuzzy clustering
• clustering with artificial neural networks

Signal Processing

The area of signal processing is concerned with the representation and transformation of 1-D (e.g., time series) and 2-D (e.g., images) signals. This includes linear systems theory, filter design, and spectral transforms. Apart from being a subject on its own, signal processing methods are used as preprocessing and feature extraction tools.

Research directions:

• filter design for specialized applications (high resolution ECG)
• denoising
• feature extraction for hand gesture recognition and 3-D object recognition

Statistical Learning Theory

Statistical learning theory (also called Vapnik-Chervonenkis-Theory or VC-Theory) has been developed by Vladimir Vapnik over the past 30 years. Originally introduced for pattern recognition, it is developed for various statistical learning tasks including density estimation and function regression. One of the most powerful class of learning algorithm originating from statistical learning theory is the support vector machine. This universal constructive learning scheme can be used to train a variety of architectures such as artificial neural networks, radial basis functions, and polynomial estimators. Support vector learning can be used for classification and regression tasks.

Research directions:

• support vector learning in radial basis function networks
• multiclass classification with support vector machines
• hierarchies of support vector machines for multiclass pattern recognition

Automated Bioacoustics

In bioacoustic research a lot of signal parameters like the spectrogram, the sonogram, the periodogram, etc. can be analysed and used to classify the sounds of animals. The waveform contains a lot of information and characteristic features about the species. In automated classification of animal sounds different features are extracted from the time signal and artificial neural networks are utilized to classify these features. Some of these features can be used to classify the species, whereas others may be used to identify an individual animal. In context we study the problem of data fusion, decision fusion and temporal fusion.

Research directions:

• classification of crickets