Abstract

Medical recommender systems are increasing in popularity within the digital health sector. Two main principles for personalised support are just-in-time interventions, and adaptiveness of treatment. Intervention concepts using these principals are called JITAIs, and they aid clients in self-management for health-related issues. In this contribution, the JITAI framework is introduced, and its advantages for recommender systems are discussed. Mathematically, the JITAI concept can be interpreted as a contextual or regular multi-armed bandit problem, which is solved via a bandit algorithm. After discussing several algorithmic strategies of bandit algorithms and elaborating on their differences, the Thompson Sampling strategy is identified as a practical solution for real-life applications using the JTIAI framework. Subsequently, existing recommender systems based on the (contextual) multi-armed bandit approach are reviewed, and the disruption of the algorithm’s learning process by instances of missing data is found to be a prevalent obstacle. An algorithm called Thompson Sampling with Restricted Context is put forward as a solution, where missing data is processed within the bandit setting.