Lecture: | Tuesday, 14:15 - 16:45, H45.2 |
Exercise: | Monday, 13:00 - 14:30, Room 43.2.101 |
Applied Information Theory
Exam
The review of the second Exam will be on Friday October 11, at 11 am in the institute's library (room 43.2.227).
Contents
Information theory is the basis of modern telecommunication systems. Main topics of information theory are source coding, channel coding, multi-user communication systems, and cryptology. These topics are based on Shannons work on information theory, which allows to describe information with measures like entropy and redundancy.
After a short overview of the whole area of information theory, we will consider concepts for statistic modeling of information sources and derive the source coding theorem. Afterwards, important source coding algorithms like Huffman, Tunstall, Lempel-Ziv and Elias-Willems will be described.
The second part of the lecture investigates channel coding. Important properties of codes and fundamental decoding strategies will be explained. Moreover, we will introduce possibilities for estimating the error probability and analyze the most important channel models according to the channel capacity introduced by Shannon.The Gaussian Channel is very important and therefore described extensively.
The third part deals with aspects of multi-user communication systems. We will introduce several models and investigate methods that can achieve the capacity regions.
Finally, we will give an introduction on data encryption and secure communication.
In the projects several information theoretic topics (e.g., Lempel-Ziv-coding) will be investigated by means of implementation tasks.
Basics:
- Uncertainty (entropy), mutual information
- Fano's lemma, data processing inequality
Source Coding:
- Shannon's source coding theorem
- Coding methods for memoryless sources: Shannon-Fano-, Huffman-, Tunstall, and arithmetic coding
- Coding for sources with memory
- Universal Source Coding
- Rate Distortion Theory
Channel Coding:
- Concepts of linear binary block codes
- Shannon's channel coding theorem
- Random coding and error exponent
- MAP and ML decoding
- Bounds
- Channels and capacities: Gaussian channel, fading channel
- Reed-Muller Codes
- Incremental Redundancy
- Channel Coding with Feedback
Multi-User Systems:
- Duplex transmission
- MAC channel
- BC channel
- MIMO channel
- Queueing Theory
- Random Access / ALOHA
Cryptography:
- Basics
References
- Thomas M. Cover and Joy A. Thomas, "Elements of Information Theory", Library ID: QAA 170/2006 C
- Rolf Johannesson, "Informationstheorie", Library ID: QAA 170/1992 J (in German, can also be bought in our secretariat)
- James L. Massey, Lecture Notes on "Applied Digital Information Theory I", ETH Zürich, external link to ETH Zürich (pdf)
- Former german lecture notes by Prof. Bossert (pdf)
Semesterapparat
"Semesterapparat" to this Lecture
Exercise Sheets
- Exercise Sheet 1 ( April 29 Solution 1 )
- Exercise Sheet 2 ( May 6 Solution 2 )
- Exercise Sheet 3 ( May 13 Solution 3 )
- Exercise Sheet 4 ( May 20 Solution 4 )
- Exercise Sheet 5 ( May 27 Solution 5 )
- Exercise Sheet 6 ( June 3 Solution 6 )
- Exercise Sheet 7 ( June 17 Solution 7 )
- Exercise Sheet 8 (June 24 Solution 8)
- Exercise Sheet 9 (July 1 Solution 9)
- Exercise Sheet 10 (July 15 Solution 10)
Script
Full Script (25.06.17)
Former German Lecture Notes
Former Lecture Slides
- First Lecture (in German)
- Introduction
- Introduction Stochastic
- Basics of Information Theory
- Source Coding Theorem
- Typical Sequences
- Source Coding
- Arithmetic Coding
- Sources with Memory
- Channel Coding
- Channel Coding Theorem
- Zero Error Capacity
- MAP- and ML-Decoding
- Gaussian Channels
- Gaussian Channels II
- Multi-User Communication
- Multiple-Access Channel
- Diversity
- IT Security
Labs
- Lab 1: Lempel-Ziv Coding ( Tasks | Source Files )
- Lab 2: Mutual Information of Finite Alphabets ( Tasks | Source Files )
The labs will not be discussed in the exercise.
English
Bachelor
Probability Theory
First exam: August 12, 10:00 - 11:30
Second exam: October 9, 10:00 - 11:30
Hours per Week: 3V + 2Ü + 1P
8 ECTS Credits