UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

How notifications affect engagement with a behaviour change app: Results from a micro-randomised trial

Bell, Lauren; Garnett, C; Bao, Y; Cheng, Z; Qian, T; Perski, Olga; Potts, Henry; (2023) How notifications affect engagement with a behaviour change app: Results from a micro-randomised trial. JMIR mHealth and uHealth , 11 , Article e38342. 10.2196/38342. Green open access

[thumbnail of PDF (3).pdf]
Preview
PDF
PDF (3).pdf - Published Version

Download (195kB) | Preview

Abstract

Background: Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to “Please complete your drinks and mood diary,” yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users’ reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement. Objective: Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy. Methods: We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement. Results: Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different. Conclusions: We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase “in-the-moment” engagement. Further optimization is required to improve the long-term engagement. International Registered Report Identifier (IRRID): RR2-10.2196/18690

Type: Article
Title: How notifications affect engagement with a behaviour change app: Results from a micro-randomised trial
Open access status: An open access version is available from UCL Discovery
DOI: 10.2196/38342
Publisher version: https://doi.org/10.2196/38342
Language: English
Additional information: This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
Keywords: Mobile health; mHealth; digital health; behavior change; behavior change; digital behavior change; engagement; micro-randomized trial; randomized trial; randomization; just-in-time adaptive intervention; adaptive intervention; push notification; notification; excessive alcohol consumption; smartphone app; alcohol; drinking; drinker; mobile phone
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health > Behavioural Science and Health
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics > CHIME
URI: https://discovery.ucl.ac.uk/id/eprint/10171904
Downloads since deposit
10Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item