PhD Defence | A Model A Day Keeps The Doctor Away

Reinforcement learning has shown great potential in many applications in recent years. However, many of these applications rely on simulators such as games to obtain a significant amount of interactions at a relatively low cost. Especially in the health domain data suitable for sequential decision-making is hard to obtain in the health domain, while simulators are complicated to build. In this thesis, we investigate how we can use reinforcement learning for personalization in e-Health. We explored the state-of-the-art applications of reinforcement learning for personalization through a systematic literature review. Looking at the related work in the health domain, we found that reinforcement learning is an appropriate learning paradigm for personalization in general.

Furthermore, we observed a lack of simulator environments and standardized datasets for the health domain. To mitigate this limitation, we developed an open-source simulation environment for e-Health problems. Using this simulator as a testbed, we proposed a cluster-based reinforcement learning approach to personalize e-Health interventions. Our approach finds similarities in behaviors and learns personalized policies optimized for long-term health behaviors. To bridge the gap towards real-life applications, we employed generative models, such as generative adversarial models, to synthesize low-level sensor data we can obtain from mobile devices in real-life. To evaluate the generated data’s representativity for sequential decision-making problems, we proposed a framework consisting of two properties, structural and functional representativity. Using our realistic simulator, we used deep reinforcement learning to demonstrate that end-to-end learning on raw sensor data can improve performance. Finally, we utilized reinforcement learning for personalization for several real-world problems. We applied deep reinforcement learning for the hemodynamic optimization of patients with sepsis at the Intensive Care Unit. Finally, we applied our proposed cluster-based reinforcement learning approach for personalization to improve adherence to internet-based interventions tested in a real-life experiment using a mobile application.