Author Archive: Jeroen Ruwaard

Ecological Momentary Assessment in Mental Health Research: A Practical Introduction, with Examples in R

Ruwaard, J., Kooistra, L. and Thong, M. (2018). Ecological Momentary Assessment in Mental Health Research: A Practical Introduction, with Examples in R. Amsterdam: APH Mental Health. free access

Given known limitations of retrospective self-report questionnaires, such as recall bias and poor generalizability of assessment results to real-life situations, mental health researchers increasingly adopt alternative assessment methods. One of the promising alternatives is Ecological Momentary Assessment (EMA), in which emotions and behaviors are repeatedly sampled in everyday life, through wearable electronic devices.

Conducting an EMA study, however, can be challenging. Researchers face a dazzling array of options related to the electronic wearables, outcomes selection, study design considerations, ethical and regulatory constraints, data management, statistical analysis, and study reporting. Although standards are emerging, clear guidelines for EMA research do not – at present – exist. EMA studies have unique characteristics that require specialist research skills, related to study design and statistical analysis. This manual was written to fill this gap.

The manual provides a practical introduction to EMA-research. It was written to aid beginning researchers of the Amsterdam School of Public Health (APH), who are looking for practical advice in conducting EMA studies. The manual provides an overview of EMA instruments, outcomes, methods and analytic techniques, guidelines for EMA-studies, and a catalogue of EMA research in the APH consortium.

This manual is available online at Sources are available at GitHub. Please post your comments and suggestions there, or via e-mail, via

Six-year healthcare trajectories of adults with anxiety and depressive disorders: Determinants of transition to specialised mental healthcare

Kooistra, L. C., Wiersma, J. E., Ruwaard, J. J., Riper, H., Penninx, B. W. J. H., & van Oppen, P. (2018). Six-year healthcare trajectories of adults with anxiety and depressive disorders: Determinants of transition to specialised mental healthcare. Journal of Affective Disorders.


To investigate potential facilitators and barriers for patients receiving specialised mental healthcare using a longitudinal design.

Longitudinal data on 701 adult participants with a depressive and/or anxiety disorder were derived from the Netherlands Study of Depression and Anxiety (NESDA). Demographic, clinical and treatment determinants at baseline were assessed with self-report questionnaires and the Composite International Diagnostic Interview (CIDI 2.1). Transition to specialised mental healthcare was assessed at one, two, four, and six-year follow-up with a self-report resource use questionnaire (TiC-P).

28.3% of patients with a depressive and/or anxiety disorder transitioned from receiving no care or primary mental healthcare to specialised mental health services during six-year follow-up. The multivariate Cox regression model identified suicidal ideation, younger age, higher education level, openness to experience, pharmacological treatment, prior treatment in primary mental healthcare and perceived unmet need for help as determinants of transition, explaining 8-18% of variance.

This study focused on baseline determinants of future transition to specialised mental healthcare. Recovery and remittance of depression and anxiety in relation to transition were not studied.

Not all key clinical guideline characteristics such as severity of symptoms and comorbidity were predictive of a transition to specialised mental healthcare, while non-clinical factors, such as age and perceived unmet need for help, did influence the process.

Smartphone-based safety planning and self-monitoring for suicidal patients

Nuij, C., van Ballegooijen, W., Ruwaard, J., de Beurs, D., Mokkenstorm, J., van Duijn, E., … Kerkhof, A. (2018). Smartphone-based safety planning and self-monitoring for suicidal patients: Rationale and study protocol of the CASPAR (Continuous Assessment for Suicide Prevention And Research) study. Internet Interventions, 13, 16–23. doi: 10.1016/j.invent.2018.04.005


It remains difficult to predict and prevent suicidal behaviour, despite growing understanding of the aetiology of suicidality. Clinical guidelines recommend that health care professionals develop a safety plan in collaboration with their high-risk patients, to lower the imminent risk of suicidal behaviour. Mobile health applications provide new opportunities for safety planning, and enable daily self-monitoring of suicide-related symptoms that may enhance safety planning. This paper presents the rationale and protocol of the Continuous Assessment for Suicide Prevention And Research (CASPAR) study. The aim of the study is two-fold: to evaluate the feasibility of mobile safety planning and daily mobile self-monitoring in routine care treatment for suicidal patients, and to conduct fundamental research on suicidal processes.

The study is an adaptive single cohort design among 80 adult outpatients or day-care patients, with the main diagnosis of major depressive disorder or dysthymia, who have an increased risk for suicidal behaviours. There are three measurement points, at baseline, at 1 and 3 months after baseline. Patients are instructed to use their mobile safety plan when necessary and monitor their suicidal symptoms daily. Both these apps will be used in treatment with their clinician.

The results from this study will provide insight into the feasibility of mobile safety planning and self-monitoring in treatment of suicidal patients. Furthermore, knowledge of the suicidal process will be enhanced, especially regarding the transition from suicidal ideation to behaviour. The study protocol is currently under revision for medical ethics approval by the medical ethics board of the Vrije Universiteit Medical centre Amsterdam (METc number 2017.512/NL62795.029.17).

Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis.

Karyotaki, E., Ebert, D. D., Donkin, L., Riper, H., Twisk, J., Burger, S., …, Ruwaard, J., …, Cuijpers, P. (2018). Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clinical Psychology Review, 63, 80–92. doi: 10.1016/J.CPR.2018.06.007

Little is known about clinically relevant changes in guided Internet-based interventions for depression. Moreover, methodological and power limitations preclude the identification of patients’ groups that may benefit more from these interventions. This study aimed to investigate response rates, remission rates, and their moderators in randomized controlled trials (RCTs) comparing the effect of guided Internet-based interventions for adult depression to control groups using an individual patient data meta-analysis approach. Literature searches in PubMed, Embase, PsycINFO and Cochrane Library resulted in 13,384 abstracts from database inception to January 1, 2016. Twenty-four RCTs (4889 participants) comparing a guided Internet-based intervention with a control group contributed data to the analysis. Missing data were multiply imputed. To examine treatment outcome on response and remission, mixed-effects models with participants nested within studies were used. Response and remission rates were calculated using the Reliable Change Index. The intervention group obtained significantly higher response rates (OR = 2.49, 95% CI 2.17-2.85) and remission rates compared to controls (OR = 2.41, 95% CI 2.07-2.79). The moderator analysis indicated that older participants (OR = 1.01) and native-born participants (1.66) were more likely to respond to treatment compared to younger participants and ethnic minorities respectively. Age (OR = 1.01) and ethnicity (1.73) also moderated the effects of treatment on remission.Moreover, adults with more severe depressive symptoms at baseline were more likely to remit after receiving internet-based treatment (OR = 1.19). Guided Internet-based interventions lead to substantial positive treatment effects on treatment response and remission at post-treatment. Thus, such interventions may complement existing services for depression and potentially reduce the gap between the need and provision of evidence-based treatments.

Mood Mirroring with an Embodied Virtual Agent: A Pilot Study on the Relationship Between Personalized Visual Feedback and Adherence

Provoost, S., Ruwaard, J., Neijenhuijs, K., Bosse, T. & Riper, H. (2018). Mood Mirroring with an Embodied Virtual Agent: A Pilot Study on the Relationship Between Personalized Visual Feedback and Adherence. In: Bajo J. et al. (eds) Highlights of Practical Applications of Agents, Multi-Agent Systems, and Complexity: The PAAMS Collection. PAAMS 2018. Communications in Computer and Information Science, vol 887, pp 24-35. Springer, Cham. Cham: Springer. doi: 10.1007/978-3-319-94779-2_3

Human support is thought to increase adherence to internet-based interventions for common mental health disorders, but can be costly and reduce treatment accessibility. Embodied virtual agents may be used to deliver automated support, but while many solutions have been shown to be feasible, there is still little controlled research that empirically validates their clinical effectiveness in this context. This study uses a controlled and randomized paradigm to investigate whether feedback from an embodied virtual agent can increase adherence. In a three-week ecological momentary assessment smartphone study, 68 participants were asked to report their mood three times a day. An embodied virtual agent could mirror participant-reported mood states when thanking them for their answers. A two-stage randomization into a text and personalized visual feedback group, versus a text-only control group, was applied to control for individual differences (study onset) and feedback history (after two weeks). Results indicate that while personalized visual feedback did not increase adherence, it did manage to keep adherence constant over a three-week period, whereas fluctuations in adherence could be observed in the text-only control group. Although this was a pilot study, and its results should be interpreted with some caution, this paper shows how virtual agent feedback may have a stabilizing effect on adherence, how controlled experiments on the relationship between virtual agent support and clinically relevant measures such as adherence can be conducted, and how results may be analyzed.

E-health interventies voor eetstoornissen (Handboek Eetstoornissen; Hoofdstuk 13)

Vos, R., Glashouwer, K.A. & Ruwaard, J. J. (2018). Hoofdstuk 13: E-health interventies voor eetstoornissen [e-Health interventions for eating disorders]. In: Annemarie van Elburg & Greta Noordenbos (red). Handboek Eetstoornissen (3e druk), Utrecht: de Tijdstroom. isbn: 9789058982506

E-health is een veelbelovende ontwikkeling voor de behandeling van eetstoornissen. Er bestaan online-interventies over het hele spectrum van preventie tot aan blended behandeling. In dit hoofdstuk worden een aantal voorbeelden van Nederlandse onlineproducten beschreven en geven we een actuele stand van zaken van onderzoek naar e-healthinterventies voor eetstoornissen. Kwalitatief goed onderzoek naar online-interventies voor eetstoornissen staat nog in de kinderschoenen en implementatie van online-interventies in de reguliere ggz gaat vooralsnog moeizaam. Verwacht wordt dat e-health in de toekomst een belangrijke plaats zal innemen in de behandelpraktijk.