Primer and Vision for Generative AI-Powered Interactive Self Reflection Interventions

This presentation will introduce a vision for using generative AI, specifically Large Language Models (LLMs), as dynamic “sensors” to capture subjective states and contexts in mental health research. It will address the limitations of traditional Experience Sampling Methods (EMAs), which are often burdensome and lack nuance, and will propose leveraging LLMs to generate raw conversational data streams that can be processed into perception time series—enabling more flexible and personalized insights into individual internal experiences. Emphasis will be placed on establishing reliability and validity comparable to physiological sensors, while exploring applications that will range from passive data collection to interactive self-reflection interventions. By positioning LLMs as adaptive, language-based tools for understanding and supporting mental health, the presentation will outline a future pathway toward richer, more responsive approaches to assessing and enhancing human well-being.

Self-Reflection to Promote Stress Management by Individuals and Couples in the Pre-Generative AI Era

Self-reflection can have profound impact on people to motivate them to initiate and sustain healthier behaviors. In the context of stress management, where the source of stress are so varied, providing them with the right kind of information is critical. This talk will showcase two successful applications of self-reflection. In the 100-day MOODS study, 122 participants were provided with visualization of their stress, stressors, and associated context. They reported self-initiating 14 different types of behavior changes and a significant reduction in both self-reported intensity and frequency of stressful events. In the couples study, 19 couples reviewed their stress and stressor data with each other and discovered many new life lessons about how they can better care for each other. The two studies show that self-reflection can help people self-initiate changes in their behaviors that can reduce stress for them, and for others that they closely interact with.

Using Gen AI to provide visual guidance in completing complex tasks – A case study in using Gen AI to produce multimedia intervention content

There is growing interest in creating AI agents that could support individuals in performing complex tasks throughout their daily life. For example, a person wearing a pair of smart glasses who is facing a repair task for the first time, or assembling a new piece of furniture, could benefit from personalized instructions, delivered via the glasses, which are tailored to their particular context. This talk will describe an approach to generating an instructional image, meant to easily inform a participant on how they should perform a particular task. We assume we are given an image captured from an egocentric perspective, a first person view of the scene captured via smart glasses, along with a question about how to perform a task. The output is an automatically synthesized image in the same egocentric reference frame that illustrates the process of performing the action. I will describe our solution to the two technical challenges: 1) generating datasets with detailed action descriptions in a scalable way, and 2) adapting existing diffusion-based image generation models to produce images in the egocentric data domain.

Toward Autonomous mHealth Agents: Harnessing Multimodal LLMs and Foundation Models

Mobile health (mHealth) systems are rapidly evolving from smartphones and wearable based passive data collection platforms into autonomous agents that sense, reason, and act to support health and well-being. Large Language Models (LLMs) and related foundation models, including multimodal LLMs and vision-language models (VLMs), provide a powerful substrate for building such agents. Their ability to integrate world knowledge with sensor inputs enables new forms of reasoning over noisy, heterogeneous, unstructured data, and their generative capabilities open opportunities for naturalistic, human-centered actions such as adaptive feedback, guidance, and conversation. In this talk, drawing upon recent work from our group, I will outline how generative AI can power autonomous mHealth agents across the sensing–reasoning–action pipeline. I will conclude with key opportunities and open challenges in making these agents efficient and trustworthy.

Towards Holistically Intelligent Adaptive Interventions

The use of reinforcement learning (RL) methods to support health behavior change via personalized and just-in-time adaptive interventions is of significant interest to health and behavioral science researchers focused on problems such as smoking cessation support and physical activity promotion. However, RL methods are often applied to these domains using a small collection of context variables to mitigate the significant data scarcity issues that arise from practical limitations on the design of adaptive intervention trials. In this talk, I will describe an approach to significantly expanding the state space of an adaptive intervention using a combination of participant-provided state descriptions and natural inference with pre-trained large language models. This approach is a first step towards the development of adaptive intervention agents that have a more holistic understanding of participants while retaining the ability of intervention designers to completely control intervention content.

Democratizing Adaptive mHealth Intervention Research with the JustIn Framework and the pJITAI

This presentation introduces the Justin Framework, an open-source platform designed to democratize adaptive mobile health (mHealth) intervention research by streamlining the development of just-in-time adaptive interventions (JITAIs). Current approaches to building JITAIs are often costly, time-intensive, and technically demanding, requiring researchers to recreate core infrastructure for each new project. The Justin Framework addresses these challenges through a modular, layered architecture that supports three levels of interaction: a programming layer for developers, a configuration layer for technical researchers, and a graphical interface for behavioral scientists. This design enables users with varying technical expertise to efficiently build and deploy adaptive interventions. Key goals include reducing development costs and complexity, fostering collaboration, and supporting rapid, flexible intervention design. Envisioned as a consortium-based model similar to REDCap, the framework promotes shared innovation and resource exchange across institutions. Ultimately, the Justin Framework—together with the pJITAI Toolbox—advances a standardized, accessible foundation for accelerating behavioral science and adaptive mHealth intervention research.

Toolbox

Abstract Text