Markov Processes

Lecturer

Prof. Dr. Alexander Bulinski


Time and Place

Lecture (January 13th to February 7th, 2014)
Monday, 2-4 pm in O28/2002

Tuesday, 10-12 am in He220

Excercise session
Wednesday, 8-10 am in HeE60


Type

 4 hours lecture and 2 hours excercise

Credit points: 4


Prerequisites

Stochastics I, Stochastics II


Intended Audience

Master students in "Mathematik", "Wirtschaftsmathematik" and "Mathematische Biometrie"; Master students in "Finance"


Contents

  • Varous definitions of Markov processes (with discrete and continuous time).
  • Important examples of Markov processes (processes with independent increments, Brownian motion, Poisson process etc.).
  • Homogeneous Markov chains. Semigroup of transition matrices. Generator.
  • Some models involving Markov chains.
  • Competition theorem for Poisson processes. The Doob construction of Markov chain.
  • Limit theorems for Markov chains.
  • Markov processes and martingales.
  • The optimization problems. Simulated annealing.
  • Hidden Markov models, their applications.
  • Introduction to Markov random fields.

Requirements


Exam

tba


Problem Sheets

tba


Literature

  • B.A.Berg. Markov Chain, Monte Carlo Simulations and Their Statistical Analysis. World Scientific. 2004.
  • P.Brémaud. Markov chains. Gibbs Fields, Monte Carlo Simulation, and  Queues. Springer. 2005.
  • A.Bulinski, A.Shiryaev. Theory of Stochastic Processes. FIZMATLIT. 2005 (in Russian).
  • E.Pardoux. Markov Processes and Applications. Algorithms, Networks, Genome and Finance. J.Wiley. 2008.
  • D.W.Stroock. An Introduction to Markov Processes. Springer. 2005.

Contact

Lecturer

  • Alexander Bulinski
  • Office hours: on appointment
  • Phone: tba

News

  • The lecture on Tuesday, 14th of January, 2014, takes place in room N24/131.