Mathematical Finance, Stochastic Analysis, and Machine Learning Seminar by Anthony Coache: Robust Reinforcement Learning with Dynamic Distortion Risk Measures

Speaker: Anthony Coache, research associate, Department of Mathematics, Imperial College London.

Title: Robust Reinforcement Learning with Dynamic Distortion Risk Measures

Abstract: In a reinforcement learning (RL) setting, the agent's optimal strategy heavily depends on her risk preferences and the underlying model dynamics of the training environment. These two aspects influence the agent's ability to make well-informed and time-consistent decisions when facing testing environments. In this talk, we present a framework to solve robust risk-aware RL problems where we simultaneously account for environmental uncertainty and risk with a class of dynamic robust distortion risk measures. Robustness is introduced by considering all models within a Wasserstein ball around a reference model. We show how to estimate such dynamic robust risk measures using neural networks by making use of strictly consistent scoring functions, derive policy gradient formulae using the quantile representation of distortion risk measures, and demonstrate the performance of our actor-critic algorithm on a portfolio allocation example. Joint work with Sebastian Jaimungal.

Mathematical Finance, Stochastic Analysis, and Machine Learning Seminar