Sören
Christensen
Christian-Albrechts-Universität zu Kiel
How to Learn from Data in Stochastic Control Problems – An Approach Based on Statistics
Abstract.
While theoretical solutions to many stochastic control problems are well understood, their practicality often suffers from the assumption of known dynamics of the underlying stochastic process, which raises the statistical challenge of developing purely data-driven controls. In this talk, we discuss how stochastic control and statistics can be brought together, which we study for various classical control problems with underlying one- and multi-dimensional diffusions and jump processes. The dilemma between exploration and exploitation plays an essential role in the considerations. We find exact sublinear-order convergence rates for the regret and compare the results numerically with those of deep Q-learning algorithms. The talk is based on: - Nonparametric learning for impulse control problems (with C. Strauch), Annals of Applied Probability 33 (2), 1369 - 1387, 2023 - Learning to reflect: A unifying approach for data-driven stochastic control strategies (with C. Strauch, Lukas Trottner), Bernoulli 30 (3) 2074 - 2101, August 2024 - Data-driven rules for multidimensional reflection problems (with Asbjørn Holk Thomsen, Lukas Trottner), SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 4, 2024 - Data-driven optimal stopping: A pure exploration analysis (with Niklas Dexheimer, Claudia Strauch), 2024 (preprint on arXiv)