User Experience of a Large-Scale Smartphone-Based Observational Study in Multiple Sclerosis:Global, Open-Access, Digital-Only Study

Background: The Floodlight Open app is a digital health technology tool (DHTT) that comprises remote, smartphone sensor-based tests (daily activities) for assessing symptoms of multiple sclerosis (MS). User acquisition, engagement, and retention remain a barrier to successfully deploying such tools. Objective: This study aims to quantitatively and qualitatively investigate key user experience (UX) factors associated with the Floodlight Open app. Methods: Floodlight Open is a global, open-access, digital-only study designed to understand the drivers and barriers in deploying a DHTT in a naturalistic setting without supervision and onboarding by a clinician. Daily activities included tests assessing cognition (Information Processing Speed and Information Processing Speed Digit-Digit), hand-motor function (Pinching Test and Draw a Shape Test), and postural stability and gait (Static Balance Test, U-Turn Test, and Two-Minute Walk Test [2MWT]). All daily activities except the 2MWT were taken in a fixed sequence. Qualitative UX was studied through semistructured interviews in a substudy of US participants with MS. The quantitative UX analysis investigated the impact of new UX design features on user engagement and retention in US participants for 3 separate test series: all daily activities included in the fixed sequence (DA), all daily activities included in the fixed sequence except the Static Balance Test and U-Turn Test (DA(x)), and the 2MWT. Results: The qualitative UX substudy (N=22) revealed the need for 2 new UX design features: a more seamless user journey during the activation process that eliminates the requirement of switching back and forth between the app and the email that the participants received upon registration, and configurable reminders and push notifications to help plan and remind the participants to complete their daily activities. Both UX design features were assessed in the quantitative UX analysis. Introducing the more seamless user journey (original user journey: n=608; more seamless user journey: n=481) improved the conversion rate of participants who enrolled in the study and proceeded to successfully activate the app from 53.9% (328/608) to 74.6% (359/481). Introducing reminders and push notifications (with reminders and notifications: n=350; without reminders and notifications: n=172) improved continuous usage time (proportion of participants with >= 3 consecutive days of usage: DA and DA(x): similar to 30% vs similar to 12%; 2MWT: similar to 30% vs similar to 20%); test completion rates (maximum number of test series completed: DA: 279 vs 64; DAx: 283 vs 126; 2MWT: 302 vs 76); and user retention rates (at day 30: DA: 53/172, 30.8% vs 34/350, 9.7%; DA(x): 53/172, 30.8% vs 60/350, 17.1%; 2MWT: 39/172, 22.6% vs 22/350, 6.2%). Inactivity times remained comparable. Conclusions: The remote assessment of MS with DHTTs is a relatively nascent but growing field of research. The continued assessment and improvement of UX design features can play a crucial role in the successful long-term adoption of new DHTTs.