Category Archive: Publications

Smartphone-based safety planning and self-monitoring for suicidal patients

Nuij, C., van Ballegooijen, W., Ruwaard, J., de Beurs, D., Mokkenstorm, J., van Duijn, E., … Kerkhof, A. (2018). Smartphone-based safety planning and self-monitoring for suicidal patients: Rationale and study protocol of the CASPAR (Continuous Assessment for Suicide Prevention And Research) study. Internet Interventions, 13, 16–23. doi: doi: 10.1016/j.invent.2018.04.005


It remains difficult to predict and prevent suicidal behaviour, despite growing understanding of the aetiology of suicidality. Clinical guidelines recommend that health care professionals develop a safety plan in collaboration with their high-risk patients, to lower the imminent risk of suicidal behaviour. Mobile health applications provide new opportunities for safety planning, and enable daily self-monitoring of suicide-related symptoms that may enhance safety planning. This paper presents the rationale and protocol of the Continuous Assessment for Suicide Prevention And Research (CASPAR) study. The aim of the study is two-fold: to evaluate the feasibility of mobile safety planning and daily mobile self-monitoring in routine care treatment for suicidal patients, and to conduct fundamental research on suicidal processes.

The study is an adaptive single cohort design among 80 adult outpatients or day-care patients, with the main diagnosis of major depressive disorder or dysthymia, who have an increased risk for suicidal behaviours. There are three measurement points, at baseline, at 1 and 3 months after baseline. Patients are instructed to use their mobile safety plan when necessary and monitor their suicidal symptoms daily. Both these apps will be used in treatment with their clinician.

The results from this study will provide insight into the feasibility of mobile safety planning and self-monitoring in treatment of suicidal patients. Furthermore, knowledge of the suicidal process will be enhanced, especially regarding the transition from suicidal ideation to behaviour. The study protocol is currently under revision for medical ethics approval by the medical ethics board of the Vrije Universiteit Medical centre Amsterdam (METc number 2017.512/NL62795.029.17).

Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis.

Karyotaki, E., Ebert, D. D., Donkin, L., Riper, H., Twisk, J., Burger, S., …, Ruwaard, J., …, Cuijpers, P. (2018). Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clinical Psychology Review, 63, 80–92. doi: 10.1016/J.CPR.2018.06.007

Little is known about clinically relevant changes in guided Internet-based interventions for depression. Moreover, methodological and power limitations preclude the identification of patients’ groups that may benefit more from these interventions. This study aimed to investigate response rates, remission rates, and their moderators in randomized controlled trials (RCTs) comparing the effect of guided Internet-based interventions for adult depression to control groups using an individual patient data meta-analysis approach. Literature searches in PubMed, Embase, PsycINFO and Cochrane Library resulted in 13,384 abstracts from database inception to January 1, 2016. Twenty-four RCTs (4889 participants) comparing a guided Internet-based intervention with a control group contributed data to the analysis. Missing data were multiply imputed. To examine treatment outcome on response and remission, mixed-effects models with participants nested within studies were used. Response and remission rates were calculated using the Reliable Change Index. The intervention group obtained significantly higher response rates (OR = 2.49, 95% CI 2.17-2.85) and remission rates compared to controls (OR = 2.41, 95% CI 2.07-2.79). The moderator analysis indicated that older participants (OR = 1.01) and native-born participants (1.66) were more likely to respond to treatment compared to younger participants and ethnic minorities respectively. Age (OR = 1.01) and ethnicity (1.73) also moderated the effects of treatment on remission.Moreover, adults with more severe depressive symptoms at baseline were more likely to remit after receiving internet-based treatment (OR = 1.19). Guided Internet-based interventions lead to substantial positive treatment effects on treatment response and remission at post-treatment. Thus, such interventions may complement existing services for depression and potentially reduce the gap between the need and provision of evidence-based treatments.

Mood Mirroring with an Embodied Virtual Agent: A Pilot Study on the Relationship Between Personalized Visual Feedback and Adherence

Provoost, S., Ruwaard, J., Neijenhuijs, K., Bosse, T. & Riper, H. (2018). Mood Mirroring with an Embodied Virtual Agent: A Pilot Study on the Relationship Between Personalized Visual Feedback and Adherence. In: Bajo J. et al. (eds) Highlights of Practical Applications of Agents, Multi-Agent Systems, and Complexity: The PAAMS Collection. PAAMS 2018. Communications in Computer and Information Science, vol 887, pp 24-35. Springer, Cham. Cham: Springer. doi: 10.1007/978-3-319-94779-2_3

Human support is thought to increase adherence to internet-based interventions for common mental health disorders, but can be costly and reduce treatment accessibility. Embodied virtual agents may be used to deliver automated support, but while many solutions have been shown to be feasible, there is still little controlled research that empirically validates their clinical effectiveness in this context. This study uses a controlled and randomized paradigm to investigate whether feedback from an embodied virtual agent can increase adherence. In a three-week ecological momentary assessment smartphone study, 68 participants were asked to report their mood three times a day. An embodied virtual agent could mirror participant-reported mood states when thanking them for their answers. A two-stage randomization into a text and personalized visual feedback group, versus a text-only control group, was applied to control for individual differences (study onset) and feedback history (after two weeks). Results indicate that while personalized visual feedback did not increase adherence, it did manage to keep adherence constant over a three-week period, whereas fluctuations in adherence could be observed in the text-only control group. Although this was a pilot study, and its results should be interpreted with some caution, this paper shows how virtual agent feedback may have a stabilizing effect on adherence, how controlled experiments on the relationship between virtual agent support and clinically relevant measures such as adherence can be conducted, and how results may be analyzed.

E-health interventies voor eetstoornissen (Handboek Eetstoornissen; Hoofdstuk 13)

Vos, R., Glashouwer, K.A. & Ruwaard, J. J. (2018). Hoofdstuk 13: E-health interventies voor eetstoornissen [e-Health interventions for eating disorders]. In: Annemarie van Elburg & Greta Noordenbos (red). Handboek Eetstoornissen (3e druk), Utrecht: de Tijdstroom. isbn: 9789058982506

E-health is een veelbelovende ontwikkeling voor de behandeling van eetstoornissen. Er bestaan online-interventies over het hele spectrum van preventie tot aan blended behandeling. In dit hoofdstuk worden een aantal voorbeelden van Nederlandse onlineproducten beschreven en geven we een actuele stand van zaken van onderzoek naar e-healthinterventies voor eetstoornissen. Kwalitatief goed onderzoek naar online-interventies voor eetstoornissen staat nog in de kinderschoenen en implementatie van online-interventies in de reguliere ggz gaat vooralsnog moeizaam. Verwacht wordt dat e-health in de toekomst een belangrijke plaats zal innemen in de behandelpraktijk.

Using multi-relational data mining to discriminate blended therapy efficiency on patients based on log data

Rocha, A., Camacho, R.,Ruwaard, J., & Riper, H. (2018). Using multi-relational data mining to discriminate blended therapy efficiency on patients based on log data. Internet Interventions, 12, 176–180. doi: 10.1016/j.invent.2018.03.003


Clinical trials of blended Internet-based treatments deliver a wealth of data from various sources, such as self-report questionnaires, diagnostic interviews, treatment platform log files and Ecological Momentary Assessments (EMA). Mining these complex data for clinically relevant patterns is a daunting task for which no definitive best method exists. In this paper, we explore the expressive power of the multi-relational Inductive Logic Programming (ILP) data mining approach, using combined trial data of the EU E-COMPARED depression trial.

We explored the capability of ILP to handle and combine (implicit) multiple relationships in the E-COMPARED data. This data set has the following features that favor ILP analysis: 1) Time reasoning is involved; 2) there is a reasonable amount of explicit useful relations to be analyzed; 3) ILP is capable of building comprehensible models that might be perceived as putative explanations by domain experts; 4) both numerical and statistical models may coexist within ILP models if necessary. In our analyses, we focused on scores of the PHQ-8 self-report questionnaire (which taps depressive symptom severity), and on EMA of mood and various other clinically relevant factors. Both measures were administered during treatment, which lasted between 9 to 16 weeks.

E-COMPARED trial data revealed different individual improvement patterns: PHQ-8 scores suggested that some individuals improved quickly during the first weeks of the treatment, while others improved at a (much) slower pace, or not at all. Combining self-reported Ecological Momentary Assessments (EMA), PHQ-8 scores and log data about the usage of the ICT4D platform in the context of blended care, we set out to unveil possible causes for these different trajectories.

This work complements other studies into alternative data mining approaches to E-COMPARED trial data analysis, which are all aimed to identify clinically meaningful predictors of system use and treatment outcome. Strengths and limitations of the ILP approach given this objective will be discussed.

Predicting short term mood developments among depressed patients using adherence and ecological momentary assessment data

Mikus, A., Hoogendoorn, M., Rocha, A., Gama, J., Ruwaard, J., & Riper, H. (2018). Predicting short term mood developments among depressed patients using adherence and ecological momentary assessment data. Internet Interventions, 12, 105–110. doi: 10.1016/j.invent.2017.10.001

Technology driven interventions provide us with an increasing amount of fine-grained data about the patient. This data includes regular ecological momentary assessments (EMA) but also response times to EMA questions by a user. When observing this data, we see a huge variation between the patterns exhibited by different patients. Some are more stable while others vary a lot over time. This poses a challenging problem for the domain of artificial intelligence and makes on wondering whether it is possible to predict the future mental state of a patient using the data that is available. In the end, these predictions could potentially contribute to interventions that tailor the feedback to the user on a daily basis, for example by warning a user that a fall-back might be expected during the next days, or by applying a strategy to prevent the fall-back from occurring in the first place.

In this work, we focus on short term mood prediction by considering the adherence and usage data as an additional predictor. We apply recurrent neural networks to handle the temporal aspects best and try to explore whether individual, group level, or one single predictive model provides the highest predictive performance (measured using the root mean squared error (RMSE)). We use data collected from patients from five countries who used the ICT4Depression/MoodBuster platform in the context of the EU E-COMPARED project. In total, we used the data from 143 patients (with between 9 and 425 days of EMA data) who were diagnosed with a major depressive disorder according to DSM-IV.

Results show that we can make predictions of short term mood change quite accurate (ranging between 0.065 and 0.11). The past EMA mood ratings proved to be the most influential while adherence and usage data did not improve prediction accuracy. In general, group level predictions proved to be the most promising, however differences were not significant.

Short term mood prediction remains a difficult task, but from this research we can conclude that sophisticated machine learning algorithms/setups can result in accurate performance. For future work, we want to use more data from the mobile phone to improve predictive performance of short term mood.