Room 327

Markov decision processes: building and playing a plantation game

Teodor Knapik

IMSc & Univ of New Caledonia

We consider Markov decision processes (MDP) as discrete-time stochastic models of controlled dynamical systems. We view the problem of optimal control as the problem of finding optimal strategy in a two player game with a safety winning condition. To model biological phenomena with unknown internal dynamics, we use partially observable MDP (POMDP) which are a sort of combination of MDP and hidden Markov models. Although computing infinite-horizon optimal strategy is not recursive for POMDP, for a finite horizon, the problem is PSPACE-complete. POMDP also turn out to be useful for forecasting.

The talk will be illustrated by a concrete example of pest control in a fruit farm.