Compressed Sensing

Important Information

The lecture is no longer offered from summer semester 2022!

This course will discuss the theoretical, numerical, and practical foundations of Compressed Sensing (CS) which has recently become a very important concept in information and signal processing. It allowed an alternative approach to conventional techniques for a simultaneous acquiring large amount of analog or digital sensor signals. By the advent of embedded sensor networks, the use of CS has become necessary in numerous applications, as in the industrial production (e.g. Industry 4.0), automotive (e.g. automated driving), logistics and energy supply (e.g. supply chain management and smart grid systems), medical technology (e.g. telemedicine), office equipment, consumer electronics, and so on up to the Internet of Things (IoT).

CS is also known as Compressive Sampling because it allows sampling of compressible analog signals with sampling rates well below the Nyquist rate. This analog setting of CS allows significant performance improvements of analog-to-digital converters for a broad class of time continuous signals. It is therefore possible to design universal CS based data acquisition systems with compressive sensors for analog and digital sensor signals, even if these signals are noisy.

The compressibility of most applied signal families can be exploited by developing novel CS methods which, in comparison to traditional approaches like the transform coding, involve far less processing effort for data compression. On the other hand, CS requires far more effort in order to reconstruct a sensor signal. Consequently, CS can help to resolve data deluge in complex sensing networks, where the number and resolution of the sensor signals grow to a point where the performance bottleneck moves to data processing in sensors. To avoid this raw data accumulation, new designs of data acquisition systems are proposed. They combine sensing and compression in one simple operation, replacing conventional sensors with compressive sensors. Instead of acquiring a massive amount of raw data and extracting the useful information afterwards, compressive sensors acquire this information directly.

Data compression using CS is performed by means of a simple linear superposition, while the decompression is based on optimization algorithms for finding the unique sparsest solution of an underdetermined system of linear equations. There are multiple approaches to solving this optimization problem, e.g. the generic Basis Pursuit algorithm. This fundamental CS decompression method is based on an optimization with respect to the L1-norm.

The principles of compressed sensing are difficult to comprehend from the available literature without prior special knowledge, since they encompass specific aspects and languages of many mathematical and engineering fields. The most relevant subjects to CS are high-dimensional geometries of Euclidean and Banach spaces, random matrices, information, coding and approximation theory, linear and convex programming, harmonic analysis, and combinatorics.

In these lectures, the usual highly abstract approach to compressed sensing will be replaced by a simpler, expressive and application oriented approach understandable for engineers. The underlying principle of this new research field will be systematically explained. The lectures and exercises will illuminate the basic principles of CS using the elementary language of signal processing, linear algebra and geometry only.

By attending this course, the participants are enabled to present a new, demanding and promising field of information processing, in an easy understandable way. At the end of the course the students will be able to analyze and discuss the underlying concept of compressed sensing based on solving underdetermined systems of linear equations. Likewise, they will be able to implement the main data recovery algorithms and to compare them according to various criteria. By the course, the participants are enabled to interpret complex optimization approaches using the geometry of higher dimensions. In this way they will be continuously motivated and supported to build their own approaches to the mater which may trigger new ideas for improvements. In addition, a variety of implementations of compressed sensing can be explained by the participants and new application areas can be identified and discussed. Students will be enabled to deal with the very subject-specific literature and to classify this in the proper context.

The main topics of the course include:

  • Importance of high-dimensional geometry in the modern information processing

  • Signal representation using bases and frames

  • Traditional and generalized sampling of analog signals

  • Overview of sparse recovery - discrete and analog setting

  • Necessary concepts from linear algebra

  • Sparsity and measurement basis and frames (dictionaries)

  • Sensing matrices and recovery equations

  • Geometric interpretation of linear systems of equations

  • Basics of multidimensional Euclidean geometry

  • Linear and affine subspaces, convex polytopes

  • Arrangements of hyperplanes

  • Configurations of sparse solutions

  • Linear optimization methods

  • L1-minimization

  • Basic pursuit

  • Orthogonal matching pursuit

  • Theoretical limits of CS

  • Sensing matrix design, deterministic and stochastic sensing matrices

  • Application of CS to A/D converters and RF systems

  • Application of CS to image processing and to medical imaging

  • Application of CS to channel coding and cryptography

  • Application of CS to radar technology

  • Application of CS to genetics: DNA-micro-arrays and DNA-sequencing

  • Perspectives of compressed sensing

To this day there is neither an introductory textbook, nor a detailed tutorial on compressed sensing suitable for beginners from engineering fields. Therefore, these lectures and exercises are completely accompanied with detailed presentation slides available as downloads on the website of the course.

References

  1. M. A. Davenport, M. F. Duarte, Y. C. Eldar, and G. Kutyniok, "Introduction to Compressed Sensing", Chapter 1 in the book Compressed Sensing: Theory and Applications, Cambridge University Press, 2012. Online available here.
  2. R. Baraniuk, M. A. Davenport, M. F. Duarte, C. Hegde: "An Introduction to Compressive Sensing". Online available here.
  3. A. M. Bruckstein, D. L. Donoho, and M. Elad (2009): From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images, SIAM Review, Vol. 51, No. 1, pp. 34-81. Online available here.
  4. Large parts of the exercises are based on the book "Understanding and using linear programming" written by Jiří Matoušek and Bernd Gärtner (Springer Verlag 2010)

Lecture Slides and Further Material

All lecture material can be downloaded from Moodle.

Language

English

Requirements

Basic knowledge in signal processing, linear algebra and probability

Exams

oral exam