Markov Chains and Monte Carlo Simulation

Prof. Dr. Volker Schmidt

Dipl.-Math. oec. Florian Timmermann

Time and place

Tuesday, 10:00 - 12:00 (room E20 in He18)

Exercise session and tutorial
Friday, 10:00 - 12:00 Uhr (room E60 in He18)


2 hours lecture + 1 hour exercises + 1 hour tutorial

Credit points: 4

If desired, the lecture will be held in English.


Probability Calculus

Intended Audience

Bachelor/Master students in Mathematics, Business Mathematics, Mathematical Biometrics and Finance, Diploma students in Mathematics, Business Mathematics


This lecture broads and deepens methods and models discussed in Probability Calculus.

The main topics are:

  • Markov chains in discrete time with finite state space
  • Stationarity and ergodicity of Markov chains
  • Markov-Chain-Monte-Carlo (MCMC)
  • Reversibility and coupling algorithms

Requirements and Exam

In order to become accredited for the written exam, one has to earn 50% of all homework credits.

Exam: Thursday, 29th July, 2pm - 4pm in H15

Lecture notes

Lecture Notes (English version of 2010)


This list of textbooks contains merely a small selection of books, which in addition to the lecture notes can be recommended for further reading.

  • E. Behrends: Introduction to Markov Chains. Vieweg, 2000
  • P. Bremaud: Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, 2008
  • B. Chalmond: Modeling and Inverse Problems in Image Analysis. Springer, 2003
  • D. Gamerman, H. Lopes: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman & Hall, 2006
  • O. Häggström: Finite Markov Chains and Algorithmic Applications. Cambridge University Press, 2002
  • D.A. Levin, Y. Peres, E.L. Wilmer: Markov chains and mixing times. Publications of the AMS, 2009
  • S. I. Resnick: Adventures in Stochastic Processes. Birkhäuser, 1992
  • C. Robert, G. Casella: Introducing Monte Carlo Methods with R. Springer, 2009
  • T. Rolski, H. Schmidli, V. Schmidt, J. Teugels: Stochastic Processes for Insurance and Finance. Wiley, 1999
  • Y. Suhov, M. Kelbert: Probability and Statistics by Example. Volume 2. Markov Chains: A Primer in Random Processes and their Applications. Cambridge University Press, 2008
  • H. Thorisson: Coupling, Stationarity, and Regeneration. Springer, 2002
  • G. Winkler: Image Analysis, Random Fields and Dynamic Monte Carlo Methods. Springer, 2003



  • Office hours on appointment
  • Phone: +49 (0)731/50-23532
  • Homepage

Teaching Assistant