Markov Chains and Monte Carlo Simulation

Prof. Dr. Volker Schmidt

Dipl.-Math. Ole Stenzel

Time and place


Exercise session


2 hours lecture + 1 hour exercises

Credit points: 4

If desired, the lecture will be held in English.


Probability Calculus

Intended Audience

Bachelor/Master students in Mathematics, Business Mathematics, Mathematical Biometrics and Finance, Diploma students in Mathematics, Business Mathematics

The participation at an additional reading course is neccessary in order to get credits for a Master program. Details about the reading course will be announced soon.



This lecture broads and deepens methods and models discussed in Probability Calculus.

The main topics are:

  • Markov chains in discrete time with finite state space
  • Stationarity and ergodicity of Markov chains
  • Markov-Chain-Monte-Carlo (MCMC)
  • Reversibility and coupling algorithms


Requirements and Exam

In order to become accredited for the written exam, one has to earn 50% of all homework credits.

Lecture notes

Lecture Notes (English version of 2010)



This list of textbooks contains merely a small selection of books, which in addition to the lecture notes can be recommended for further reading.

  • E. Behrends: Introduction to Markov Chains. Vieweg, 2000
  • P. Bremaud: Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, 2008
  • B. Chalmond: Modeling and Inverse Problems in Image Analysis. Springer, 2003
  • D. Gamerman, H. Lopes: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman & Hall, 2006
  • O. Häggström: Finite Markov Chains and Algorithmic Applications. Cambridge University Press, 2002
  • D.P. Kroese, T. Taimre, Z.I. Botev: Handbook of Monte Carlo Methods. J. Wiley & Sons, 2011.
  • D.A. Levin, Y. Peres, E.L. Wilmer: Markov chains and mixing times. Publications of the AMS, 2009
  • S. I. Resnick: Adventures in Stochastic Processes. Birkhäuser, 1992
  • C. Robert, G. Casella: Introducing Monte Carlo Methods with R. Springer, 2009
  • T. Rolski, H. Schmidli, V. Schmidt, J. Teugels: Stochastic Processes for Insurance and Finance. Wiley, 1999
  • R.Y. Rubinstein, D.P. Kroese: Simulation and the Monte Carlo Method, 2nd Edition. John Wiley & Sons, 2007
  • Y. Suhov, M. Kelbert: Probability and Statistics by Example. Volume 2. Markov Chains: A Primer in Random Processes and their Applications. Cambridge University Press, 2008
  • H. Thorisson: Coupling, Stationarity, and Regeneration. Springer, 2002
  • G. Winkler: Image Analysis, Random Fields and Dynamic Monte Carlo Methods. Springer, 2003




  • Office hours on appointment
  • Phone: +49 (0)731/50-23532
  • Homepage

Teaching Assistant

  • Office hours on appointment
  • Phone: +49 (0)731/50-23590
  • Homepage


  • Lecture starts on 17/04/2012
  • No exercises on 17/04/2012