PhD Defence | A Model A Day Keeps The Doctor Away

Vrije Universiteit Amsterdam PhD Candidate Ali El Hassouni will defend his PhD thesis titled: “A Model A Day Keeps The Doctor Away” on January 18th 2022. Ali’s research was supervised by Mark Hoogendoorn (VU) and Gusz Eiben (VU).

Reinforcement learning has shown great potential in many applications in recent years. However, many of these applications rely on simulators such as games to obtain a significant amount of interactions at a relatively low cost. Especially in the health domain data suitable for sequential decision-making is hard to obtain in the health domain, while simulators are complicated to build. In this thesis, we investigate how we can use reinforcement learning for personalization in e-Health. We explored the state-of-the-art applications of reinforcement learning for personalization through a systematic literature review. Looking at the related work in the health domain, we found that reinforcement learning is an appropriate learning paradigm for personalization in general.

Furthermore, we observed a lack of simulator environments and standardized datasets for the health domain. To mitigate this limitation, we developed an open-source simulation environment for e-Health problems. Using this simulator as a testbed, we proposed a cluster-based reinforcement learning approach to personalize e-Health interventions. Our approach finds similarities in behaviors and learns personalized policies optimized for long-term health behaviors. To bridge the gap towards real-life applications, we employed generative models, such as generative adversarial models, to synthesize low-level sensor data we can obtain from mobile devices in real-life. To evaluate the generated data’s representativity for sequential decision-making problems, we proposed a framework consisting of two properties, structural and functional representativity. Using our realistic simulator, we used deep reinforcement learning to demonstrate that end-to-end learning on raw sensor data can improve performance. Finally, we utilized reinforcement learning for personalization for several real-world problems. We applied deep reinforcement learning for the hemodynamic optimization of patients with sepsis at the Intensive Care Unit. Finally, we applied our proposed cluster-based reinforcement learning approach for personalization to improve adherence to internet-based interventions tested in a real-life experiment using a mobile application.