Im Rahmen des Mathematischen Kolloquiums referiert Prof. Dr. Sergey Foss, Heriot-Watt University in Edinburgh, zu dem Thema: "On the instability of local learning algorithms: Q-learning can fail in infinite state spaces".

Datum: 15.05.2026

Zeit: 10:15 Uhr s.t.

Ort: Helmholtzstrasse 18, Raum 220

Ab 9:45 Uhr findet im Raum gegenüber ein Treffen mit Kaffee oder Tee statt. Nach dem Vortrag wird ein gemeinsames Mittagessen in einem Restaurant stattfinden. Bei Interesse schreiben Sie bitte eine E-Mail an Carolina Fernandez (carolina.fernandez(at)uni-ulm.de).

Abstract:

We investigate the challenges of applying model-free reinforcement learning
algorithms, such as online Q-learning, to Markov decision processes (MDPs) with
infinite state spaces. We first introduce the notion of Local Learning Processes
(LLPs), in which agents make decisions based solely on local information, and show
that Q-learning can be viewed as a specific instance of an LLP. Using renewal
techniques, we analyse LLPs and demonstrate their instability under certain drift and
initial conditions, revealing fundamental limitations in infinite state spaces. In
particular, we show that although Q-learning is asymptotically optimal in finite
settings, it can exhibit instability and strict suboptimality in infinite state spaces.