Skip to main content
Download PDF

Working mechanisms of the use and acceptability of ecological momentary interventions: a realist evaluation of a guided self-help ecological momentary intervention targeting self-esteem

BMC Public Health volume 24, Article number: 1633 (2024) Cite this article

Abstract

Background

Technology improves accessibility of psychological interventions for youth. An ecological momentary intervention (EMI) is a digital intervention geared toward intervening in daily life to enhance the generalizability and ecological validity, and to be able to intervene in moments most needed. Identifying working mechanisms of the use of ecological momentary interventions might generate insights to improve interventions.

Methods

The present study investigates the working mechanisms of the use and acceptability of an ecological momentary intervention, named SELFIE, targeting self-esteem in youth exposed to childhood trauma, and evaluates under what circumstances these mechanisms of use and acceptability do or do not come into play. A realist evaluation approach was used for developing initial program theories (data: expert interviews and a stakeholders focus group), and subsequently testing (data: 15 interviews with participants, a focus group with therapists, debriefing questionnaire), and refining them.

Results

The SELFIE intervention is offered through a smartphone application enabling constant availability of the intervention and thereby increasing accessibility and feasibility. When the intervention was offered on their personal smartphone, this enhanced a sense of privacy and less hesitance in engaging with the app, leading to increased disclosure and active participation. Further, the smartphone application facilitates the practice of skills in daily life, supporting the repeated practice of exercises in different situations leading to the generalizability of the effect. Buffering against technical malfunction seemed important to decrease its possible negative effects.

Conclusions

This study enhanced our understanding of possible working mechanisms in EMIs, such as the constant availability supporting increased accessibility and feasibility, for which the use of the personal smartphone was experienced as a facilitating context. Hereby, the current study contributes to relatively limited research in this field. For the field to move forward, mechanisms of use, and acceptability of EMIs need to be understood. It is strongly recommended that alongside efficacy trials of an EMI on specific target mechanisms, a process evaluation is conducted investigating the working mechanisms of use.

Trial registration

The current paper reports on a realist evaluation within the SELFIE trial (Netherlands Trial Register NL7129 (NTR7475)).

Peer Review reports

Background

Youth, referring to a period of transitioning from childhood to adulthood spanning from 12 to 25 years of age, is a crucial time regarding the development of psychosocial capacities and the onset of mental disorders [1]. Specifically, 62.5% of individuals report an onset of a mental disorder before the age of 25 [2]. Thus, it is important to intervene in youth, however, a gap exists between their needs and modus operandi, and available care [3,4,5,6]. The importance of overcoming this gap has extensively been underscored in clinical work as well as research and has led to efforts for international reform of youth mental health [7]. In recent years, mHealth (i.e. mobile health, interventions making use of mobile devices) increasingly provides solutions to bridge this gap. Technology can make psychological interventions more accessible for youth, who are ‘digital natives’ [8]. An abundance of apps that promise to relieve psychological stress and enhance mental well-being are available in major app stores. But only a small number of applications are tested for efficacy [9] and most are not grounded in sound psychological theories, such as cognitive behavioral interventions [10].

Ecological momentary interventions (EMI) are typically digital interventions [11, 12]. However, the development of EMIs was not triggered by the digital transformation, it is rooted in ecological psychology, for which digital advances (e.g. with smartphones) provide new assessment methods and treatment delivery channels. Ecological psychology aims to study experience and behavior in the context of normal daily life. Therefore it uses Ecological Momentary Assessment (EMA) [11]. EMIs take this a step further and assume that experience and behavior are not only situated in, but also most amenable to change in momentary daily life situations [13]. EMIs are designed to intervene in daily life, herewith enhancing the generalizability and ecological validity of the learning situation, and focus interventions to target moments [14]. Moreover, EMIs can prompt behaviour, experiences, and assignments in real life, independant of contact with a therapist.

EMIs were developed for different mental health problems such as mood [15, 16], anxiety [17, 18], and substance use disorders [19,20,21]. In addition, EMIs can focus on transdiagnostic intervention elements [22, 23]. Unfortunately, still a limited number of studies rigorously research the effect of EMIs and several major (transdiagnostic) psychological domains remain unaddressed [24,25,26].

The term ‘just-in-time adaptive intervention’ (JITAI) has also been used in literature to describe an intervention delivered in daily life that is customized to an individual’s state aiming to provide the right type of support varying over time, using information assessed through e.g. EMA [27]. Even though there are similarities between EMI and JITAI (i.e. intervening in the moment when needed and prompted), JITAI emphasizes the specific element of adaptation over time (not inherent to EMI perse) and, hence, may be termed as a subclass of EMI [24]. However, from the outset, EMIs used adaptive strategies, e.g., referred to as interactive tasks in recent studies [28, 29], and as such adaptation of the intervention is inherent to EMIs. For this paper, with the EMI under study delivering interactive tasks, the term EMI will be used.

EMI research

To enhance the field of EMI research an important challenge is the need for high-quality trials [25, 30, 31]. The SELFIE trial targeting self-esteem, and the EMI compass trial targeting resilience through a compassion-focused intervention, are particular examples for targeting mechanisms that may confer the development of mental disorders at a later age [23, 29, 31]. Research moves from knowing that psychological interventions work to understanding how they work [32]. This requires understanding the ‘mechanism’, defined as the roots of the effect, i.e., the processes of events that are responsible for the change; the reasons why change occurred or how change came about” [33]. Thus, e.g. self-esteem may be coined as an ‘active ingredient’ for psychological interventions [34,35,36], and could be targeted as a mechanism of influence in the development and maintenance of psychological disorders [37]. Such a mechanism is hereafter referred to as ‘target mechanism’. Specifically in EMI research, the feasibility and effectiveness of intervening in target mechanisms in daily life (a core element of EMI) is researched. The testing of ecological interventionist causal models [12], offers an addition to existing research in testing whether an EMI can modify a target mechanism in daily life, and secondly test whether producing changes in the mechanism in daily life leads to changes in intended psychological outcomes. Nonetheless, how the target mechanism is being influenced remains unanswered. This influencing of the target mechanism may be seen as a mechanism in itself, defined as an explanatory theory of what elements of (the mode of delivery of) an intervention causes changes in individual behavior relating to the target mechanism [38], and will hereafter be referred to as ‘working mechanism’. Current research should pay attention to the target mechanisms as well as the working mechanisms of the intervention itself to improve the use of EMIs in clinical care [11].

Various working mechanisms have been pointed out in previous work. For example, EMIs are available at any given time and in every given context. Individuals may feel more equipped to apply new behaviours and skills in their actual experience in real life with the extra support of an EMI [14, 39]. Additionally, based on assessment responses, EMIs can tailor (the content of) the intervention to specific needs at specific times, based on, for example, need of additional support, motivation or readiness to change [14]. EMIs can also prompt contextual reminders to trigger specific behaviours in specific contexts [24, 40, 41]. Another established working mechanism is self-monitoring, individuals can track their behaviours and become more aware of them, which is necessary for change [41, 42]. In some EMIs individuals can connect with peers or friends for social support [14, 41]. Furthermore, personalized feedback on app progress can enhance self-awareness and motivation [15, 24, 40, 42], and goal-setting can increase motivation [14, 40]. Further, research has shown that guided self-help and face-to-face treatments may have comparable effects [43], and specifically for EMIs, guidance was proposed as an important working mechanism [10, 44]. Yet, limited research is available on how and for whom this mechanism works. Knowledge of working mechanisms is an important aspect to take into account when further implementing EMIs.

To understand the working mechanisms of an EMI, a realist evaluation method is a fitting approach because it develops a theory on how a program works, for whom, and under what circumstances [45]. Therefore, applying realist evaluation methodology within a EMI trial helps to advance current research. It also provides information to customize interventions [30]. Gaining insights into how and why an intervention works, would support future development and implementation of EMIs.

SELFIE intervention

The current paper reports on a realist evaluation within the SELFIE trial (Netherlands Trial Register NL7129 (NTR7475)) of which details are available elsewhere [29, 31]. The SELFIE trial aimed to investigate the efficacy and clinical feasibility of SELFIE, which is a smartphone-based guided self-help intervention for improving self-esteem in youth exposed to childhood trauma, in a multi-center, parallel-group, assessor-blind randomized controlled trial (RCT). Their findings show improvement in the primary outcome of self-esteem at postpost-intervention and 6-month follow-up, and small to moderate effect sizes point towards beneficial effects on some secondary outcomes such as general psychopathology and quality of life [31]. The EMI under study in the SELFIE trial aimed to enhance self-esteem in youth (12–25 years) who have experienced childhood adversity (i.e., abuse, neglect, bullying, and/or household discord). A 6-week manualized intervention is delivered by trained SELFIE therapists, consisting of three face-to-face sessions, three standardized e-mail contacts, and an EMI administered through a smartphone-based app (i.e., the PsyMate® app), supporting the adaptive real-time and real-world transfer of intervention components tailored to moment, person, and context. By providing an ecologically valid, accessible, and personalized intervention, SELFIE aimed to tailor interventions to the needs of youth [11, 12, 14]. The preventive content, principles and techniques of the transdiagnostic SELFIE intervention have been based on the CBT model and interventions [46] and the self-help manual by de Neef [47].

Aims

The aim of the present study was to investigate, within the SELFIE intervention, mechanisms of EMI and under what circumstances these mechanisms do or do not come into play.

Methods

Study design

A realist evaluation methodology was adopted to gain insight into EMI working mechanisms and consisted of three phases: (1) developing an initial programme theory (IPT), based on literature and two expert interviews and one focus group with stakeholders; (2) testing the IPT by comparing and explaining data offered by fifteen semi-structured interviews with youth who had received the SELFIE intervention, one focus-group with SELFIE therapists, and a debriefing questionnaire (n = 61); (3) refining the IPT. The guidelines offered by the RAMESES II reporting standards for realist evaluation [48] were followed, thereby adhering to the Standards for Reporting Qualitative Research (SRQR) [49].

Researchers undertaking this realist evaluation also partly played a role in developing (MP) and delivering the SELFIE intervention (MP, KS, SV). To minimize interpretation bias and influences due to a therapeutic relationship after delivering the intervention, researchers did not interview participants to whom they had previously delivered the SELFIE intervention. Neutrality in the process of data collection and analysis was ensured by reflexivity, i.e. documenting the progress, decisions, and motives of the researchers, aimed at supporting self-awareness concerning their role and impact on the research environment. In addition, frequent meetings with independent researchers (LD, MS, and a researcher conducting a realist evaluation in a different research trial) were held to discuss the researchers’ role.

The current realist evaluation was set in the context of the SELFIE trial [23, 29], and follows a previously undertaken realist evaluation of the SELFIE intervention [50]. This previously undertaken realist evaluation was a result of the known relevance of interventions aimed at low self-esteem in youth, and due to the complex nature of self-esteem and its targeting by an intervention, it was considered important to focus the analysis of the available qualitative data solely on characteristics and delivery of self-esteem interventions. The present realist evaluation analyses the same qualitative data, however, the analysis is aimed to investigate mechanisms of EMI, and under what circumstances these mechanisms do or do not come into play. The methods of the present realist evaluation, therefore, resemble the methods of the previous realist evaluation within the SELFIE trial but focus on the distinct aspect of EMI within the SELFIE intervention. A detailed description of the methods is shown in Table 1.

Table 1 Methods of recruitment, data collection, and analyses for phases 1 and 2
Full size table

Participants

Phase 1 (the development of the IPTs) was based on literature study, qualitative data from 2 expert interviews, and a focus group with stakeholders. Subsequently, for phase 2 (testing the IPTs) iterative data collection took place through 15 individual interviews with SELFIE participants (within six months after finishing the intervention), analyzing data from a debriefing questionnaire (n = 61), and a focus group with 4 SELFIE therapists. As stated, more details of the methods used in recruitment, data collection, and analyses are shown in Table 1.

Analyses

As stated above and shown in Table 1, data collection and analyses took place in an iterative process during the different phases of this realist evaluation leading to the development of IPTs after phase 1 and formulated context-mechanism-outcome configurations (CMOCs) after phase 2. In parallel, a debriefing questionnaire (participants filled this in directly after receiving the SELFIE intervention, and it inquired about participants’ experience with the app, the use of the exercises, satisfaction, and acceptance of the SELFIE intervention) was addressed in testing the IPTs. In phase 3 these CMOCs were synthesized back into the IPTs. MP conducted this analysis individually, and subsequently discussed it within the research team. The process of analysis was inspired by the approach described by Gilmore et al. [51]. Namely, IPTs were coded as nodes, to which quotations were assigned illustrative for IPT-specific CMOCs and their implications for the IPTs (refute, refine, or accept). To ensure transparency, the process of reasoning was explicitly described in research memos.

Results

Stakeholders and participants

For phase 1, two experts were contacted and found willing to participate. Further, four stakeholders took part in a focus group. For phase 2, firstly 23 participants in the SELFIE trial were contacted of whom 15 took part in semi-structured online (due to COVID-19 restrictions) interviews.

Table 2 Participant characteristics of the interview sample of this RE and the participant characteristics of the full sample of participants in the experimental condition of the SELFIE trial
Full size table

Table 2 shows the characteristics of this interview sample alongside the characteristics of the full sample of participants in the experimental condition of the SELFIE trial [31]. Not having sufficient time (n = 6) and mentally not feeling well enough to participate (n = 2) were reported as reasons not to participate. The average age of the participants was 21 years; 12 participants identified as women, and three as men. Secondly, six therapists who delivered the SELFIE intervention were contacted of whom four participated in a focus group. Non-participation was due to agenda restrictions (n = 2).

Main findings phase 1 (developing the IPTs)

Three pillars of the SELFIE intervention were defined based on its program architecture. First, it is delivered as an EMI, second, the intervention aims to target self-esteem, and last, it is offered as a guided self-help intervention. As described before, the working mechanisms that underpin EMIs are not fully understood, and this study seeks to obtain data that will help address this research gap by focusing the present analysis solely on the pillar regarding EMI. The other pillars are discussed in a separate paper [50].

Data collected during Phase 1 was described and interpreted in the context of pre-existing literature and theories. A detailed description of the process of identifying the IPTs can be found in Supplementary Material 6.

The three key IPTs related to EMI that were identified, tested against our data, and refined, were:

  • If participants experience the intervention as personalized, anonymous, and easily accessible, then they will be more comfortable with their input and can participate in the SELFIE intervention without feelings of shame.’ (IPT 1).

  • If a change in momentary self-esteem is established repeatedly and under different circumstances through the use of a smartphone application, then this will support the generalizability of this effect and support change in general self-esteem.’ (IPT 2).

  • If a technical malfunction is present or the reminder beeps are being perceived negatively, then the effect of the intervention will be limited and drop-out may occur.’ (IPT 3).

Table 3 shows the initial program theories, with supportive literature.

Table 3 Initial Program Theories on how the SELFIE intervention as an EMI may exert its effect on most users
Full size table

Main findings phase 2 and 3 (testing and refining IPTs)

The IPTs as shown in Table 3 were tested (phase 2) and refined (phase 3). Our main findings will be described below per IPT in the form of CMOCs as the analytical template of realist evaluation, followed by the refined IPT.

Testing IPT 1

A reported response to the intervention being offered repeatedly in daily life and being constantly available was to experience the intervention as accessible and feasible due to a lack of travel time to appointments, short exercises, reminders on the phone, and the ability to practice an exercise when there is time and need. The following quote illustrates the availability of the smartphone application being offered on their personal phone (important context vs. receiving a study phone).

“Ehm, well yes, it is nice that you can fill it in wherever, it does not matter where you are. I mean, yes ok, you could also bring a booklet anywhere but that would be something you might forget now and again. But you will not forget your phone as easily. You would go back home for it if you would forget it. So ehm, if you are on the train, at work, or you have a short break, then you can just work on it [the SELFIE intervention]. – Source: PI_3 (356–360)

Offering the intervention on their personal smartphone enhanced active participation, as illustrated in the following quote, supporting intervention outcomes.

“Yes, well it is just easy because you always carry your phone with you. So you don’t have to make a great effort (…) if I would have to do it [the exercises] on the computer, which I don’t carry with through the entire day, then you have to especially sit down to do it, making time for it, I feel. And with the app, you can note things down in between moments when you have some time left. (…) Yes, I think [that if I would have to use a computer] I would have been less engaged with the intervention. Because of the app, it just offers you easy access and so on, and that made me participate more actively than if I would have done it on a computer.” – Source: PI_6 (196–205)

In testing IPT 1, data supports the notion of ‘easily accessible’ as stated in IPT 1. Furthermore, it offers more detailed insight into why it is experienced as accessible. Having an intervention on your smartphone helps youngsters to stay committed due to the constant availability of the intervention, aiding flexible use of the exercises and making it more feasible to integrate it into their daily life since the expense in time is limited due to a lack of travel time to appointments for example. Supportive data was extracted from a debriefing questionnaire in that participants report to have experienced that the application helped them to practice the exercises in daily life, and that during the day they repetitively were aware of their context, feelings, thoughts, and behavior (as can be seen in Table 4).

Table 4 User experience and acceptance of the SELFIE intervention (n = 61)
Full size table

Regarding the sense of anonymity as phrased in IPT 1, it was found that the exercises were offered individually and privately, which offers a sense of privacy and anonymity without the interference of (possible) responses of others (response). The following quote from a participant illustrates this experience.

“By writing it down yourself in an app dares me to write down more that when I would have to say it (…) Yes, the idea that you are more anonymous, that people will not directly see or read everything or, well yes if you are in a conversation with someone else, I would experience that stronger. Now it was more the idea of writing it for yourself, rather than for someone else” - Source: PI_6 (182–195)

The sense of privacy and anonymity led to participants being less hesitant to fill in exercises in an honest and ‘unfiltered’ manner.

Reflecting on IPT 1, elements of ‘anonymity’ and ‘feeling more comfortable with their input’ are supported by our data. However, regarding the experience of participants that others will not interfere with their input, the word ‘privacy’ seems to be a better fit for this experience than ‘anonymity’ (as was used in IPT1). Participants were aware that their SELFIE therapist was able to partly read along on a reporting page, thus it wasn’t completely anonymous in that sense. In summary, a sense of privacy adds to the openness of participants when engaging in the intervention. They are less hesitant to fill in answers due to a lack of response by another person (for example, questioning their input further, or having an opinion about their input). It was inferred that open and active participation in the intervention is supportive for the intervention outcomes relating to self-esteem.

Refined IPT 1

The aforementioned findings in testing IPT 1 have been framed as CMOCs, as depicted in Table 5. These CMOCs have informed the following refinement of IPT 1:

  • The SELFIE intervention is offered through a smartphone application (C) enabling constant availability of the intervention and thereby increasing accessibility and feasibility (M), and, when offered on their personal smartphone (C) this enhanced a sense of privacy ensuring less hesitance in engaging with the app (M), leading to more open and active participation (O).

Table 5 CMOCs relating to IPT 1
Full size table

Testing IPT 2

Generalizability of effect is reported, however, emphasis is placed on acquiring a non-situation-specific skill (not dependent on situation-specific characteristics), and thereby able to lead to the generalizability of effect. The following quote refers to an exercise aimed at coping with receiving criticism, in which the participant previously mentioned that due to the exercise she gained a new perspective:

Interviewer: (…) What happens consequently when you look at things differently, does that change something in your thoughts or your feelings? What is its effect?

Participant: Mainly in my thoughts, that I think ‘ok’, well if I take the example again of that exercise on criticism, then, yes, how do you say that?… All of a sudden you are not so afraid of receiving feedback anymore or criticism or something like that, because I…, it has never been proven but I do think that I have a fear of failing. So I always found it very scary to make mistakes. And if it happens now I think ‘Yes, but actually, there is nothing wrong’. Because of that I then feel a lot more confident in my actions and experiences. Not just at work, but also in daily life. Well… my feeling related to that, that could make me very sad, that I felt that way, I could also be very anxious when I received criticism, which really frightened me. But, that is now a lot less.” - Source: PI_15

Participants rated the offering of prompting beeps for and the constant availability of the exercises as more than moderately effective in aiding change in self-esteem (also shown in Table 4). The sessions with the SELFIE therapist were rated slightly higher in supporting change in self-esteem, possibly implying the importance of guidance when delivering an EMI.

Synthesizing the above findings back into the IPT, we inferred that for the SELFIE intervention repetition in daily life is key in creating the circumstance of being more prone to cognitive restructuring, and since this surpasses situation-specific characteristics, with additionally the availability of certain tailored situation-specific exercises (e.g. criticism exercise), it could be argued that the effect is generalized over different situations in daily life. Thus generalizability is enhanced through the offering of the intervention in daily life as well as the availability of tailored interventions (as stated in IPT 2), however, this is not through the mechanism of changing momentary self-esteem in different situations, rather the mechanism seems to be that cognitive restructuring is more prominent and not dependent on situation-specific characteristics whereby effects are generalized.

Refined IPT 2

The above findings are phrased as a CMOC (presented in Table 6), and led to the following refined IPT 2:

  • Offering the intervention through the use of a smartphone application (C), facilitates the practice of skills that are not dependent on situation-specific characteristics in daily life, supporting repeated practice in different situations (M), leading to the generalizability of the effect (O).

Table 6 CMOC relating to IPT 2
Full size table

Testing IPT 3

Reminder beeps for the intervention exercises were offered through the smartphone application, popping up at random moments with an alerting sound. In some cases, as for the participant sharing the below-placed quote, this was experienced as loud and disruptive and/or for the participant at inconvenient timing.

“Ehm, yes I think the beeps are not so pleasant, I sometimes experienced them as rather annoying actually. (…) it beeped at inconvenient times, and the sound was also quite loud. Yes, it startled me sometimes .” - Source: PI_4 (317–321)

Thus, reminder beeps may sometimes be perceived as annoying or intrusive by sound or timing. Our data showed this could lead to negative sentiment and actions, such as turning off notifications and sound, with subsequently less active participation. It seems to be a balancing act between the intensity (repetition and duration) of the intervention being sufficient to reach positive outcomes (our data showed that in most cases, the reminder beeps were perceived as a positive element in supporting active participation), and the intensity being experienced as too much of a burden. Personal circumstances (well-being, energy, focus) seem to be an important context regarding assessing burden by capacity.

In the context of the intervention being delivered in the form of a smartphone application, technical malfunction may arise to which participants may respond by experiencing ‘hassle’ accompanied by irritation and a decrease in motivation. The following quote from a SELFIE participant who received the intervention illustrates discontinuity.

“It just made me postpone when it didn’t function to log in with the code, and yes, when I would receive a new code I would be in school for example, and with getting the email [with the new code] I would think ‘yes, I will do that this evening’, or something. (…) that made that actually one day or so was ‘lost” – Source: PI_1 (514–518)

In testing IPT 3, our data support the notion of a technical malfunction being perceived negatively and leading to less effective participation or drop-out. When, as part of the debriefing questionnaire, participants where asked about any problems they faced when using the SELFIE intervention, or whether they experienced it as annoying, rated as rarely (find participants’ ratings in Table 4), they mainly reported technical issues to explain their rating. In some cases, it was explained by their experience of not having enough space in the text boxes to write their answers. Technical malfunction is mainly experienced as a burden since it encompasses extra moments of contact or actions, both participants and SELFIE therapists have reported this. If needed, technical assistance was mentioned by participants to be readily available through, among more, WhatsApp, which was experienced as a low threshold, which is an important context for IPT 3. This buffered against the negative effects of technical malfunction. Further, as is shown in Table 4, ratings on the debriefing questionnaire prove very good satisfaction with and acceptance of the SELFIE intervention when also taking into account the mode of delivery.

Refined IPT 3

The aforementioned reflections have led to developing CMOCs relating to IPT 3, which can be found in Table 7. Subsequently, these CMOCs have informed the following refinement of IPT 3:

  • The use of a smartphone application to deliver the SELFIE intervention (C) may encompass technical malfunction and accompanied irritation and demotivation (M), leading to less active or delayed participation (O).

  • Furthermore, the reminder beeps (C) are activated at random moments and with an alerting sound which can be experienced as loud and disruptive (M), leading to decreased motivation and less active participation by e.g., turning off the notification or sound (O).

Table 7 CMOCs relating to IPT 3
Full size table

Discussion

Summary of findings

The aim of the present study was to investigate, within the SELFIE intervention, working mechanisms of EMI, and under what circumstances these working mechanisms do or do not come into play. The current research led to a revision of three IPTs. First, “The SELFIE intervention is a blended therapy using a smartphone application (C) enabling constant availability of the intervention and thereby increasing accessibility and feasibility (M), and, when offered on their personal smartphone (C) this enhanced a sense of privacy ensuring less hesitance in engaging with the app (M), leading to more open and active participation, thereby supporting intervention outcomes (O).” This refined programme theory illustrates that a smartphone application supports the mechanism of constant availability and consequently increases accessibility and feasibility. A supporting context for this mechanism is the use of their personal smartphone (rather than receiving a study phone), enhancing active participation and responding to the easily accessible intervention. Second, “Offering the intervention through the use of a smartphone application (C), facilitates the practice of skills (i.e. cognitive restructuring) that are not dependent on situation-specific characteristics in daily life, supporting repeated practice in different situations (M), leading to the generalizability of the effect (O).”. This refined programme theory stresses the importance of intervention exercises matching a variety of situations, meaning that if an intervention is offered repeatedly and in different situations, it will not be effective if the exercise offered is not applicable in a given situation. For the SELFIE study, it was found that practicing a relevant skill was not dependent on situation-specific characteristics and thereby effective to be offered repeatedly over a variety of situations. In addition, situation-specific exercises (such as how to deal with criticism) could be offered when permitting availability on demand. Third, “The use of a smartphone application to deliver the SELFIE intervention (C) may encompass technical malfunction and accompanied irritation which may induce demotivation (M), leading to less active or delayed participation (O).” Furthermore, “the reminder beeps (C) are activated at random moments and with a loud alerting sound which can be experienced as disruptive (M), leading to decreased motivation and less active participation by e.g., turning off the notification or sound (O).” These refined programme theories show that technical malfunction and disruptiveness of beeps do decrease motivation and engagement with the intervention. Since these ‘hurdles’ will not completely cease to exist, it is important to ‘buffer’ against the negative effects of this when delivering an EMI. We found that technical assistance being readily available and matching the participants’ needs (in this case youngsters who responded well to WhatsApp), may buffer against the negative effects of technical malfunction.

Comparison with existing literature

Delivering the SELFIE intervention through a personal smartphone app was found to be supportive of the outcome (refined IPT 1), which is in line with previous findings that the use of a smartphone seems to be associated with higher rates of adherence [52]. The use of one’s smartphone and not requiring multiple devices is often described as supporting the accessibility and usability of an EMI [53]. The current study sheds more light on the experience of youth in using their smartphone, not only supporting usability through convenience as previously reported but also revealing an underlying sense of privacy which seems to make participants less hesitant to engage with the app and thereby supporting open and active participation. Furthermore, the mentioning of a sense of privacy by participants links to research on therapeutic relationships in digital mental health delivery. A process proposed to enhance the openness of patients has been described as disinhibition, whereby clients were more prone to disclosure than in regular face-to-face contacts [54]. This is consistent with research on assessment methods, with computer-administered assessment methods obtaining more honest, open responses of personal information [55]. Furthermore, contact with a ‘virtual human’ controlled by a computer seemed to support participants’ openness and self-disclosure due to the experience that their responses were not being judged by another human, with the underlying assumption that experiencing fear of judgment will activate impression management resulting in withholding information that might threaten their reputation [55]. In contrast, even though quantitative research on the effect of coaching within an app-based intervention is limited and somewhat diffuse [56], evidence does seem to point towards human guidance within an app-based intervention to be positively associated with engagement, completion rates, and treatment outcomes [10, 57,58,59], and offering smartphone apps as standalone psychological interventions is not advisable due to the low level of current evidence on this [35]. These findings corroborate the results of the previous realist evaluation within the SELFIE trial, researching the element of guided self-help [50]. Thus, clinical judgment regarding the intention of an EMI could inform decision-making in EMI development on the degree of human interaction. The afore-described illustrates that an EMI with solely digital components is likely to differ substantially from an EMI with additional guidance by a therapist, and thus the findings from the current research should be placed within the context of guided EMIs.

The current results in the form of refined IPTs tentatively support the ecological interventionist causal model [12] in that participants report having practiced the intervention in their daily lives and that in their perception it leads to changes regarding the target mechanism of self-esteem. This is supportive of the main findings from the SELFIE study showing improvement in the primary outcome of self-esteem at post-intervention and 6-month follow-up [23]. As mentioned in the introduction, EMIs can prompt contextual reminders to trigger specific behaviours in specific contexts [24, 40, 41] aiding change in the aspired outcome of an intervention. In addition to this existing literature, our findings suggest exercises not only to be customized at the moment (and thus being context-specific) but also offer exercises to facilitate the practice of skills that are not dependent on situation-specific characteristics in daily life. By supporting repeated practice in different situations an EMI may broaden its relevance for the user.

In accordance with the current refined IPTs, previous studies have demonstrated that technical malfunction may be a negative influential factor regarding adherence and satisfaction when using app-based interventions [10, 60]. Furthermore, the finding that electronic prompts could be experienced as disturbing broadly supports the work of other studies where electronic prompts were experienced as a burden when they requested assessments of length, and a higher number of missing answers were reported when participants were prompted 8 times a day or more [52]. Thus, the findings replicate existing knowledge on prompts and technical issues with EMIs, however, the notion of a ‘buffer’ against possible negative effects by specifically adhering to the preferences of participants (in the current study the participants reported WhatsApp to be a low-threshold way of communicating, in contrast to email) is novel and further research may serve effective implementation and adherence to an EMI. User-experience outcomes could aid this type of research [61].

Strengths, limitations, and future directions

The method of realist evaluation made apparent possible working mechanisms of EMI which, even though researched within the context of the SELFIE intervention, seem to be relevant for other forms of (guided self-help) EMI. This is of crucial importance in the current state of research in the field of EMI, since developing EMIs in itself should be driven more by existing knowledge and theory of the working mechanisms to provide enhancement of the field. We would suggest that every trial that tests the efficacy of an EMI adds a process evaluation to interpret the main findings of the efficacy trial as well as to build knowledge on the working mechanisms of EMIs for further development and implementation. The current research offers insight into underlying processes that may exert an effect on the main outcome findings of the SELFIE trial. It should be noted, however, that the interactive element within the SELFIE intervention has not been addressed sufficiently concerning its importance within EMIs in the current paper. We would therefore suggest future research on this particular aspect. Further, synthesizing findings on efficacy in the SELFIE trial with findings from the realist evaluation would be of added value to deepen our understanding of interventions as well as target mechanisms as in this case self-esteem, and therefore, such a mixed-methods design would be strongly suggested for future research.

A future recommendation for EMI may be to allow participants to choose the notification sound and allow more flexibility in choosing time blocks. Studies using a pre-fixed sampling scheme (the prompts were programmed at certain times per day, and if multiple prompts were programmed, a fixed time interval was used) reported higher rates of adherence in (EMA) studies [52], however these studies offered very little prompts per day and did not aim to capture variability over the course of a day. The use of a pre-fixed sampling scheme is of course a trade-off with the effects due to the random occurrence of beeps and would not fit the aims of the SELFIE intervention since it would impair representative characterization of experience and, hence, limit EMIs substantially in tailoring/adapting to moments when help is most needed. Therefore, for the SELFIE intervention, we would recommend to offer flexibility in choosing the time blocks of a random sampling schedule within set blocks of time.

Furthermore, findings from feedback from users in an EMI for major depressive disorder reported three EMI features to have been highly appreciated: the possibility of receiving visual feedback about daily assessments, and consequently self-monitoring of daily patterns; the availability of psychoeducational material on depression and its mechanisms; and the opportunity to have continuous or periodic communication with a trained clinician [30]. Considering that the SELFIE intervention provides communication with a trained clinician, as well as psychoeducational material on self-esteem as session content, a new implication of the afore mentioned findings of Colombo, Fernández-Álvarez [30] is that the SELFIE intervention may benefit from offering participants visual feedback. In further development and implementation of the SELFIE intervention, the actual end users should continue to be involved in the design and evaluation of the technology for it to match their needs and to ensure that engagement is maximized [62].

Regarding methodological concerns, along with her role as researcher, MP was involved in developing and delivering the SELFIE intervention, which may lead to bias in carrying out the realist evaluation. To ensure reflexivity toward her role as a researcher, regular meetings with the research team were held, thereby focusing on the quality criteria of neutrality. Furthermore, the consensus in decision-making during the research process as well as the analysis was fostered through assistance by two other researchers (SV and KS) employing investigator triangulation over the study period. In addition, researchers not directly involved in the SELFIE trial (LD and MS) were regularly consulted. In contrast, the role of MP as being involved in both delivering and researching the SELFIE intervention may have supported the quality principle of applicability of evidence through transferability, i.e. the extent to which the current findings can be transferred or applied in different clinically relevant settings [63]. Second, despite applying a maximum variation sampling method, as described in Table 1 under ‘phase 2’, selection bias may have occurred in that individuals with particular characteristics might have agreed to participate in the study (e.g., participants experiencing enthusiasm regarding the SELFIE intervention may be more willing to share their experiences). For the focus group with SELFIE therapists, 4 out of 6 invited therapists were able to join the focus group. The two who were not able to attend reported this was due to agenda constraints and no particular characteristics of these two are thought to have had a bearing on the focus group data. Third, the timing of interviewing the participants was within six months after finishing the intervention to ensure a sufficient amount of interviews. It should be noted that the ability to recall the exact exercises and personal experiences is limited over such a period and should be taken into account when interpreting the findings. Future research should aim to interview participants closer to the date of finishing the intervention. Lastly, even though it is expected that certain findings are generalizable for EMIs not targeting self-esteem, it should be taken into account that self-esteem in itself may be a very important context for certain EMI working mechanisms to exert its effect. Illustrative of this is that our data support the notion that EMIs are particularly supportive of skill development. Furthermore, the context of the present study was a guided self-help intervention, this may limit the generalizability of our findings to EMIs in general. Therefore, the earlier-mentioned suggestion to promote research on the working mechanisms of EMI could aid in identifying certain combinations of context and working mechanisms to be effective.

Conclusions

The current research addressed working mechanisms of an EMI targeting self-esteem, such as the constant availability supporting increased accessibility and feasibility, for which the use of the personal smartphone was experienced as a facilitating context. Further, findings stressed the importance of delivering non situation-specific exercises (to support cognitive restructuring), or having situation-specific exercises available on demand (to aid in practicing adaptive copingstrategies in a given situation), to facilitate the mechanism of generalizability and ecological validity as an appreciated aspect of EMI. Lastly, technical malfunction and the burden of electronic prompts were experienced, reflecting previous literature, indicating the need for further research on ‘buffering’ possible negative effects of technical malfunction when delivering EMIs. Interestingly, within the SELFIE intervention, the context of accessible technical assistance showed indeed to be such a buffer against the negative effects.

The present findings can help us to understand possible working mechanisms and their contexts of EMIs, contributing to relatively limited research in this field. For the field to move forward, a better understanding of the working mechanisms of EMI needs to be developed, and it is strongly recommended that alongside efficacy trials of an EMI on specific target mechanisms, a process evaluation is conducted investigating the working mechanisms.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Abbreviations

CMOC:

Context-mechanism-outcome configuration

EMA:

Ecological Momentary Assessment

EMI:

Ecological Momentary Intervention

ESM:

Ecological Sampling Method

IPT:

Initial programme theory

JITAI:

Just-in-time adaptive intervention

References

  1. Coughlan H, Cannon M, Shiers D, Power P, Barry C, Bates T et al. Towards a new paradigm of care: the International Declaration on Youth Mental Health. 2013.

  2. Solmi M, Radua J, Olivola M, Croce E, Soardo L, Salazar de Pablo G, et al. Age at onset of mental disorders worldwide: large-scale meta-analysis of 192 epidemiological studies. Mol Psychiatry. 2022;27(1):281–95.

    Article  CAS  PubMed  Google Scholar 

  3. Hetrick SE, Bailey AP, Smith KE, Malla A, Mathias S, Singh SP, et al. Integrated (one-stop shop) youth health care: best available evidence and future directions. Med J Aust. 2017;207(10):S5–18.

    PubMed  Google Scholar 

  4. Malla A, Iyer S, McGorry P, Cannon M, Coughlan H, Singh S, et al. From early intervention in psychosis to youth mental health reform: a review of the evolution and transformation of mental health services for young people. Soc Psych Psych Epid. 2016;51:319–26.

    Article  Google Scholar 

  5. McGorry PD, Mei C. Early intervention in youth mental health: progress and future directions. Evid Based Ment Health. 2018;21(4):182–4.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Rickwood DJ, Deane FP, Wilson CJ. When and how do young people seek professional help for mental health problems? Med J Australia. 2007;187(7):S35–9.

    PubMed  Google Scholar 

  7. McGorry P, Bates T, Birchwood M. Designing youth mental health services for the 21st century: examples from Australia, Ireland and the UK. Br J Psychiatry. 2013;202(s54):s30–5.

    Article  Google Scholar 

  8. Dinesen B, Nonnecke B, Lindeman D, Toft E, Kidholm K, Jethwani K, et al. Personalized telehealth in the future: a global research agenda. J Med Internet Res. 2016;18(3):e53.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Baumel A, Torous J, Edan S, Kane JM. There is a non-evidence-based app for that: a systematic review and mixed methods analysis of depression-and anxiety-related apps that incorporate unrecognized techniques. J Affect Disorders. 2020;273:410–21.

    Article  PubMed  Google Scholar 

  10. Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T et al. What Works and What Doesn’t Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People. Front Psychiatry. 2019;10.

  11. Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatr. 2016;29(4):258–63.

    Article  Google Scholar 

  12. Reininghaus U, Depp CA, Myin-Germeys I. Ecological interventionist causal models in psychosis: Targeting Psychological mechanisms in Daily Life. Schizophr Bull. 2016;42(2):264–9.

    Article  PubMed  Google Scholar 

  13. Schulte-Strathaus JC, Rauschenberg C, Baumeister H, Reininghaus U. Ecological momentary interventions in public mental health provision. Digital phenotyping and Mobile sensing: New Developments in Psychoinformatics. Springer; 2022. pp. 427–39.

  14. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 2010;15(Pt 1):1–39.

    Article  PubMed  Google Scholar 

  15. Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, et al. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res. 2011;13(3):e1838.

    Article  Google Scholar 

  16. Wahle F, Kowatsch T, Fleisch E, Rufer M, Weidt S. Mobile sensing and support for people with depression: a pilot trial in the wild. JMIR mHealth uHealth. 2016;4(3):e5960.

    Article  Google Scholar 

  17. Pramana G, Parmanto B, Lomas J, Lindhiem O, Kendall PC, Silk J. Using mobile health gamification to facilitate cognitive behavioral therapy skills practice in child anxiety treatment: open clinical trial. JMIR Serious Games. 2018;6(2):e8902.

    Article  Google Scholar 

  18. Silk JS, Pramana G, Sequeira SL, Lindhiem O, Kendall PC, Rosen D, et al. Using a smartphone app and clinician portal to enhance brief cognitive behavioral therapy for childhood anxiety disorders. Behav Ther. 2020;51(1):69–84.

    Article  PubMed  Google Scholar 

  19. Blevins CE, Marsh EL, Stein MD, Schatten HT, Abrantes AM, Project CHOICE. Choosing healthy options in coping with emotions, an EMA/EMI plus in-person intervention for alcohol use. Substance Abuse. 2021;42(4):569–76.

    Article  PubMed  Google Scholar 

  20. Businelle MS, Walters ST, Mun E-Y, Kirchner TR, Hébert ET, Li X. Reducing drinking among people experiencing homelessness: protocol for the development and testing of a just-in-time adaptive intervention. JMIR Res Protocols. 2020;9(4):e15610.

    Article  Google Scholar 

  21. Shrier LA, Burke PJ, Kells M, Scherer EA, Sarda V, Jonestrask C et al. Pilot randomized trial of MOMENT, a motivational counseling-plus-ecological momentary intervention to reduce marijuana use in youth. Mhealth. 2018;4.

  22. Myin-Germeys I, van Aubel E, Vaessen T, Steinhart H, Klippel A, Lafit G et al. Efficacy of Acceptance and Commitment Therapy in Daily Life (ACT-DL) in early psychosis: results from the multi-center INTERACT randomized controlled trial. medRxiv. 2021:2021.05. 28.21257986.

  23. Reininghaus U, Paetzold I, Rauschenberg C, Hirjak D, Banaschewski T, Meyer-Lindenberg A et al. Effects of a Novel, Transdiagnostic Ecological Momentary Intervention for Prevention, and Early Intervention of Severe Mental Disorder in Youth (EMIcompass): findings from an exploratory Randomized Controlled Trial. Schizophrenia Bull. 2023:sbac212.

  24. Balaskas A, Schueller SM, Cox AL, Doherty G. Ecological momentary interventions for mental health: a scoping review. PLoS ONE. 2021;16(3):e0248152.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Goldberg SB, Lam SU, Simonsson O, Torous J, Sun S. Mobile phone-based interventions for mental health: a systematic meta-review of 14 meta-analyses of randomized controlled trials. PLOS Digit Health. 2022;1(1):e0000002.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Smith KE, Juarascio A. From ecological momentary assessment (EMA) to ecological momentary intervention (EMI): past and future directions for ambulatory assessment and interventions in eating disorders. Curr Psychiatry Rep. 2019;21:1–8.

    Article  Google Scholar 

  27. Doherty K, Balaskas A, Doherty G. The design of ecological momentary assessment technologies. Interact Comput. 2020;32(1):257–78.

    Article  Google Scholar 

  28. Schick A, Paetzold I, Rauschenberg C, Hirjak D, Banaschewski T, Meyer-Lindenberg A, et al. Effects of a novel, transdiagnostic, hybrid ecological momentary intervention for improving resilience in youth (EMIcompass): protocol for an exploratory randomized controlled trial. JMIR Res Protocols. 2021;10(12):e27462.

    Article  Google Scholar 

  29. Daemen M, Postma MR, Lindauer R, Hoes-van der Meulen I, Nieman D, Delespaul P, et al. Efficacy of a transdiagnostic ecological momentary intervention for improving self-esteem (SELFIE) in youth exposed to childhood adversity: study protocol for a multi-center randomized controlled trial. Trials. 2021;22(1):1–15.

    Article  Google Scholar 

  30. Colombo D, Fernández-Álvarez J, Patané A, Semonella M, Kwiatkowska M, García-Palacios A, et al. Current state and future directions of technology-based ecological momentary assessment and intervention for major depressive disorder: a systematic review. J Clin Med. 2019;8(4):465.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Reininghaus U, Daemen M, Postma MR, Schick A, Hoes-van der Meulen I, Volbragt N et al. Transdiagnostic Ecological Momentary Intervention for Improving Self-Esteem in Youth exposed to Childhood Adversity: the SELFIE Randomized Clinical Trial. JAMA Psychiatry. 2023.

  32. Holmes EA, Ghaderi A, Harmer CJ, Ramchandani PG, Cuijpers P, Morrison AP, et al. The Lancet Psychiatry Commission on psychological treatments research in tomorrow’s science. Lancet Psychiatry. 2018;5(3):237–86.

    Article  PubMed  Google Scholar 

  33. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol. 2007;3:1–27.

    Article  PubMed  Google Scholar 

  34. Wellcome’s Mental Health challenge area https://wellcome.org/what-we-do/mental-health 2020.

  35. Wellcome. What science has shown can help young people with anxiety and depression. Identifying and reviewing the ‘active ingredients’ of effective interventions: Part 1. Zenodo. 2021.

  36. Wellcome. What science has shown can help young people with anxiety and depression. Identifying and reviewing the ‘active ingredients’ of effective interventions: Part 2. Zenodo. 2022.

  37. Zeigler-Hill V. The connections between self-esteem and psychopathology. J Contemp Psychother. 2011;41(3):157–64.

    Article  Google Scholar 

  38. Shaw J, Gray CS, Baker GR, Denis JL, Breton M, Gutberg J, et al. Mechanisms, contexts and points of contention: operationalizing realist-informed research for complex health interventions. BMC Med Res Methodol. 2018;18(1):178.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry. 2013;35(4):332–8.

    Article  PubMed  Google Scholar 

  40. King AC, Hekler EB, Grieco LA, Winter SJ, Sheats JL, Buman MP, et al. Harnessing different motivational frames via mobile phones to promote daily physical activity and reduce sedentary behavior in aging adults. PLoS ONE. 2013;8(4):e62613.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Dao KP, De Cocker K, Tong HL, Kocaballi AB, Chow C, Laranjo L. Smartphone-delivered ecological momentary interventions based on ecological momentary assessments to promote health behaviors: systematic review and adapted checklist for reporting ecological momentary assessment and intervention studies. JMIR mHealth uHealth. 2021;9(11):e22890.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: are our theories up to the task? Translational Behav Med. 2011;1(1):53–71.

    Article  Google Scholar 

  43. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40(12):1943–57.

    Article  CAS  PubMed  Google Scholar 

  44. Lecomte T, Potvin S, Corbière M, Guay S, Samson C, Cloutier B, et al. Mobile apps for mental health issues: meta-review of meta-analyses. JMIR mHealth uHealth. 2020;8(5):e17458.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Pawson R, Tilley N. An introduction to scientific realist evaluation. Evaluation 21st Century: Handb. 1997;1997:405–18.

    Article  Google Scholar 

  46. Beck AT. Cognitive therapy of depression. Guilford Press; 1979.

  47. De Neef M. Negatief zelfbeeld: Boom uitgevers; 2010.

  48. Wong G, Westhorp G, Manzano A, Greenhalgh J, Jagosh J, Greenhalgh T. RAMESES II reporting standards for realist evaluations. BMC Med. 2016;14(1):1–18.

    Article  CAS  Google Scholar 

  49. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

    Article  PubMed  Google Scholar 

  50. Postma MR, Vrancken S, Daemen M, Hoes-van der Meulen I, Volbragt N, Nieman D, Delespaul P, De Haag L, Van der Pluijm M, Breedvelt JJF, Van der Gaag M, Lindauer R, Van den Berg D, Bockting C, Van Amelsvoort TAMJ, Schwannauer M, Doi L. Reininghaus, U. Investigating mechanisms in enhancing self-esteem in youth through an ecological momentary intervention: A realist evaluation within the SELFIE trial. In preparation. 2023.

  51. Gilmore B, McAuliffe E, Power J, Vallieres F. Data Analysis and Synthesis within a Realist evaluation: toward more transparent methodological approaches. Int J Qual Meth. 2019;18.

  52. Colombo D, Cipresso P, Fernández-Álvarez J, Díaz-García A, Riva G, Botella C. An overview of factors associated with adherence and dropout to ecological momentary assessments in depression. 2018.

  53. Opie JEVA, Welsh ET, Esler TB, Khan UR, Khalil H. Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young Adults with Emerging Symptoms: Part II. A Systematic Review of User Experience Outcomes. Clin Child Fam Psychol Rev. 2024 Apr 18. 2024.

  54. Hanley T, Reynolds D. Counselling psychology and the internet: a review of the quantitative research into online outcomes and alliances within text-based therapy. Counselling Psychol Rev. 2009;24(2):4–13.

    Article  Google Scholar 

  55. Lucas GM, Gratch J, King A, Morency L-P. It’s only a computer: virtual humans increase willingness to disclose. Comput Hum Behav. 2014;37:94–100.

    Article  Google Scholar 

  56. Bernstein EE, Weingarden H, Wolfe EC, Hall MD, Snorrason I, Wilhelm S. Human support in app-based cognitive behavioral therapies for emotional disorders: scoping review. J Med Internet Res. 2022;24(4):e33307.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app‐supported smartphone interventions for mental health problems: a meta‐analysis of randomized controlled trials. World Psychiatry. 2019;18(3):325–36.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis. J Affect Disorders. 2020;263:413–9.

    Article  PubMed  Google Scholar 

  59. Wu A, Scult MA, Barnes ED, Betancourt JA, Falk A, Gunning FM. Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement. NPJ Digit Med. 2021;4(1):20.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res. 2013;15(11):e247.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Opie JE, Vuong A, Welsh ET, Gray R, Pearce N, Marchionda S et al. Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young adults with emerging symptoms: part I. A systematic review of Socioemotional outcomes and recommendations. Clin Child Fam Psychol Rev. 2024:1–52.

  62. Berry N, Machin M, Ainsworth J, Berry K, Edge D, Haddock G, et al. Developing a theory-informed smartphone app for early psychosis: learning points from a multidisciplinary collaboration. Front Psychiatry. 2020;11:602861.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Frambach JM, van der Vleuten CP, Durning SJ. AM last page: quality criteria in qualitative and quantitative research. Acad Med. 2013;88(4):552.

    PubMed  Google Scholar 

  64. Punton M, Isabel V, Leavy J, Michaelis C, Boydell E. Reality bites: Making realist evaluation useful in the real world. 2020.

  65. Friese S. Qualitative data analysis with ATLAS. ti: Sage; 2019.

    Google Scholar 

  66. Fink L, Bernstein D. Childhood Trauma Questionnaire. A retrospective self-report. Manual. San Antonio: Harcourt Brace & Co; 1998.

    Google Scholar 

  67. Schäfer M, Korn S, Smith PK, Hunter SC, Mora-Merchán JA, Singer MM, et al. Lonely in the crowd: recollections of bullying. Br J Dev Psychol. 2004;22(3):379–94.

    Article  Google Scholar 

  68. Bifulco A, Brown GW, Harris TO. Childhood experience of care and abuse (CECA): a retrospective interview measure. J Child Psychol Psychiatry. 1994;35(8):1419–35.

    Article  CAS  PubMed  Google Scholar 

  69. Rosenberg M. Society and the adolescent self-image. Princeton, NJ: Princeton University Press; 1965.

    Book  Google Scholar 

  70. Manzano A. The craft of interviewing in realist evaluation. Evaluation-Us. 2016;22(3):342–60.

    Article  Google Scholar 

  71. Paay J, Nielsen H, Larsen H, Kjeldskov J, editors. Happy bits: interactive technologies helping young adults with low self-esteem. Proceedings of the 10th Nordic Conference on Human-Computer Interaction; 2018.

  72. Newman MG, Consoli A, Taylor CB. Computers in assessment and cognitive behavioral treatment of clinical disorders: anxiety as a case in point. Behav Ther. 1997;28(2):211–35.

    Article  Google Scholar 

  73. Van Ameringen M, Turna J, Khalesi Z, Pullia K, Patterson B. There is an app for that! The current state of mobile applications (apps) for DSM-5 obsessive-compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders. Depress Anxiety. 2017;34(6):526–39.

    Article  PubMed  Google Scholar 

  74. Myin-Germeys I, Birchwood M, Kwapil T. From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophr Bull. 2011;37(2):244–7.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Bell IH, Rossell SL, Farhall J, Hayward M, Lim MH, Fielding-Smith SF, et al. Pilot randomised controlled trial of a brief coping-focused intervention for hearing voices blended with smartphone-based ecological momentary assessment and intervention (SAVVy): feasibility, acceptability and preliminary clinical outcomes. Schizophr Res. 2020;216:479–87.

    Article  PubMed  Google Scholar 

  76. Parmar A, Sharma P. Ecological momentary interventions delivered by smartphone apps: applications in substance use treatment in Indian scenario. Indian J Psychol Med. 2017;39(1):102–3.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Wichers M, Simons C, Kramer I, Hartmann JA, Lothmann C, Myin-Germeys I, et al. Momentary assessment technology as a tool to help patients with depression help themselves. Acta Psychiatrica Scandinavica. 2011;124(4):262–72.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We are indebted to all individuals who participated in the SELFIE study, particularly the participants who offered their time and insights by participating in the interviews. Further, we thank all SELFIE therapists who offered the SELFIE intervention and in particular the SELFIE therapists who joined the focus group. We thank M. de Neef and prof. dr. I. Myin-Germeys for their expertise, all stakeholders, researchers, and interns. In memory of K. Swartebroeckx.

Funding

This work was supported by a ZonMw research program grant for the SELFIE trial (grant no. 636310006); a Heisenberg professorship from the German Research Foundation (grant no. 389624707 to U.R.).

Author information

Authors and Affiliations

  1. Department of Psychiatry and Neuropychology, School for Mental Health and Neuroscience, Maastricht University, Vijverdalseweg 1, Maastricht, 6226 NB, the Netherlands

    Mary Rose Postma, Maud Daemen, Nele Volbragt, Philippe Delespaul & Therese van Amelsvoort

  2. Mondriaan Mental Health Center, Heerlen, The Netherlands

    Mary Rose Postma, Suzanne Vrancken, Philippe Delespaul & Therese van Amelsvoort

  3. Prodeba Mental Health Center, Leiden, The Netherlands

    Iris Hoes-van der Meulen

  4. Department of Psychiatry, Amsterdam Public Health, Amsterdam University Medical Centers (location AMC), Amsterdam, The Netherlands

    Lieuwe de Haan, Marieke van der Pluijm & Claudi Bockting

  5. Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, London, UK

    Josefien Johanna Froukje Breedvelt

  6. Centre for Urban Mental Health, University of Amsterdam, Amsterdam, The Netherlands

    Claudi Bockting

  7. Department of Clinical Psychology, VU University, Amsterdam, The Netherlands

    Mark van der Gaag & David van den Berg

  8. Department of Child and Adolescent Psychiatry, Amsterdam UMC, location Academic Medical Center, Amsterdam, The Netherlands

    Ramon Lindauer

  9. Levvel, Academic Centre for Child and Adolescent Psychiatry, Amsterdam, The Netherlands

    Ramon Lindauer

  10. Parnassia Academy, The Hague, The Netherlands

    David van den Berg

  11. School of Health in Social Science, University of Edinburgh, Edinburgh, UK

    Matthias Schwannauer & Lawrence Doi

  12. Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Mannheim, Germany

    Ulrich Reininghaus

  13. ESRC Centre for Society and Mental Health and Social Epidemiology Research Group, King’s College London, London, UK

    Ulrich Reininghaus

  14. Health Service and Population Research Department, Centre for Epidemiology and Public Health, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK

    Ulrich Reininghaus

Authors
  1. Mary Rose Postma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suzanne Vrancken
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maud Daemen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Iris Hoes-van der Meulen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nele Volbragt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Philippe Delespaul
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lieuwe de Haan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Marieke van der Pluijm
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Josefien Johanna Froukje Breedvelt
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mark van der Gaag
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Ramon Lindauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. David van den Berg
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Claudi Bockting
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Therese van Amelsvoort
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Matthias Schwannauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Lawrence Doi
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Ulrich Reininghaus
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

UR is the principal investigator of the SELFIE trial. MP, SV, MD, NV, IHvdM, DN, MvdP, JB, MvdG, RL, DvdB, CB, and TvA contributed in some form to recruitment of participants in the SELFIE trial.For the present paper MP and SV collected interview data, and MD and NV collaborated in collecting the debriefing questionnaire data. MP, and to an extent SV, contributed to the analysis of this realist evaluation. MP wrote the paper. LD and MS were consulted as experts regarding methodology. The remaining authors edited the final iteration of the paper. All authors approved for this paper to be published.

Corresponding author

Correspondence to Mary Rose Postma.

Ethics declarations

Ethics approval and consent to participate

The SELFIE trial received ethical approval from the Medical Ethics Review Committee (MERC) at Maastricht University Medical Centre (MUMC), the Netherlands (reference: NL64393.068.17) on August 1st, 2018. An amendment has been approved in which the current aims were described. Participants were asked for written informed consent before being interviewed, consented to the audio recording of the interview, and could consent or decline that quotes may be used for publications.

Consent for publication

In the consent form, participants indicated whether or not they permitted quotes from the transcripts of their interview to be used in publications.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Postma, M.R., Vrancken, S., Daemen, M. et al. Working mechanisms of the use and acceptability of ecological momentary interventions: a realist evaluation of a guided self-help ecological momentary intervention targeting self-esteem. BMC Public Health 24, 1633 (2024). https://doi.org/10.1186/s12889-024-19143-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-024-19143-z

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us