Rethinking Mental Health Interventions: How Crowd-Powered Therapy Can Help Everyone Help Everyone

By on

We all have dark thoughts sometimes. And if you’ve ever been a graduate student, perhaps thoughts like the following feel familiar:

The thoughts in this image are real data points collected during deployment of a prototype called Flip*Doubt, an app in which negative thoughts are entered and then sent to three random crowd workers to be “positively reframed” and sent back to the user. (The full paper title is “Effective Strategies for Crowd-Powered Cognitive Reappraisal Systems: A Field Deployment of the Flip*Doubt Web Application for Mental Health” and you can read it here.) 

Rates of mental illness continue to rise every year. Yet there are nowhere near enough trained mental health professionals available to meet the need–and Covid-19 has only worsened the state of affairs. In short–we urgently need to rethink how we design mental health interventions so that they are more scalable, accessible, affordable, effective, and safe. 

So, how can technology create new ways to expand models of delivery for clinically validated therapeutic techniques? In Flip*Doubt, we focus on “cognitive reappraisal”–a well-researched technique for changing one’s thoughts about a situation in order to improve emotional wellbeing. This skill is often taught by trained therapists (e.g., in Cognitive and Dialectical Behavioral Therapy), and it has been shown to be highly effective at reducing symptoms of anxiety and depression. The problem is, it’s really hard to learn, and even harder to apply in one’s own mind on an ongoing basis. 

We envision that people could learn the skill through practice by reframing thoughts for each other–since research shows that it’s easier to learn by objectively sizing up others’ thoughts, rather than immediately trying to challenge your own entrenched ways of thinking. Thus, Flip*Doubt relies on crowd workers to create reframes, and the major driving questions of our study were: What makes reframes good or bad? And how can we design systems that effectively help people to nail the skill?

Our deployment yielded some fascinating results about how people use cognitive reappraisal systems in the wild, the types of negative thought patterns that weigh grad students down, and what types of strategies are most effective at flipping dark thoughts. For instance, the example below shows how a participant rated three reframes from Flip*Doubt:

Represented here are three different reappraisal tactics for transforming the original thought that we identified through our data analysis. “Direct negation” isn’t effective at all–it’s just invalidating and frustrating for someone to suggest the opposite of what you’re struggling with. “Agency” rings more true–yet can feel a bit simplistic. “Silver Lining” wins the gold for this thought–it provides fresh perspective by emphasizing an important positive that wouldn’t be possible without all the struggle. ​Our paper provides additional analyses, culminating in six hypotheses for what makes an effective reframe.

Our work suggests several important design implications. First, systems should consider prompting for structured reflection rather than prompting for negative thoughts. People aren’t always thinking negatively, and only allowing negative thoughts for input can reinforce those thoughts, or drive people away. Second, systems should consider tailoring user experiences to focus on a few core issues, since the best gains may come if meaningful progress can be made to address vicious and repetitive thoughts, rather than any old negative thought. Finally, crowd-powered systems can be safer and more effective if we design AI/ML-based mechanisms to help peers shape their responses through effective reappraisal strategies and behaviors–there’s a lot more on this in the paper, so we hope you’ll read about it there.

We’re honored that this paper won an Honorable Mention at #CSCW2021, since the world truly needs new interventions like Flip*Doubt to help us all to help each other. You can find the full paper at or watch the virtual presentation on YouTube

Thanks for reading, and we hope to discuss this work with you at CSCW and beyond.

Social Computing Researchers Need to Pay Attention to Religion and Spirituality in Design (Especially in Matters of Life & Death)

By on

Everything changes in a heartbeat when you or someone you love receives a life-threatening health diagnosis. 

Research from the medical and nursing fields repeatedly shows that people turn toward religion or spirituality to cope, even if they didn’t necessarily see themselves as “spiritual” during their lives. Many people wish they could go back and apply lessons learned earlier in their lives, so that they could live more fully and be better people. What if technology could help with that? To embrace the aspects of our experiences that most provide us with a sense of meaning, hope, and fulfillment–however we each individually define that? is a nonprofit health journaling platform that offers a free service similar to a blog, but with specialized tools and privacy controls to facilitate social support during serious or life-threatening illness. Our prior research showed that prayer support is more important to CaringBridge users than any other form of support [1]. Although HCI research has largely ignored religion and spirituality for decades [2-5], our #CSCW2021 paper follows up on this finding to ask, beyond prayer, “What is Spiritual Support and How Might It Impact the Design of Online Communities?” (Full paper here.)

Through participatory design focus groups with CaringBridge stakeholders, we derived the following definition:

Spiritual support is an integral dimension that underlies and can be expressed through every category of social support, including informational, emotional, instrumental, network, esteem, and prayer support. This dimension creates a triadic relationship between a recipient, a provider, and the sacred or significant, with the purpose of helping recipients and providers experience a mutually positive presence with each other, and with the sacred or significant.

The point is, when our aim is truly to support someone who is struggling, a fundamental underlying element of love and connection needs to transcend specific beliefs. Take prayer, for example. If you’re Christian, prayer might just be the most meaningful way someone can help you. If you’re an atheist, though, prayer could be quite an offensive way of expressing support. One implication is that in sensitive health contexts, designers might consider ways to help people represent their beliefs, so that supporters can craft expressions of care and support that respect them.

Building on this concept of expression, our results also highlight that even when spiritually supportive intentions are there, it’s difficult to respond to devastating news—so, participants wanted technological assistance with writing helpful comments. A second implication–which could span many types of online communities–is that commenting interfaces could embed mechanisms such as training resources, tips, or possibly automatic text recommendations. Future research will need to investigate how to design such features without damaging the meaningfulness and authenticity of comments.

Stakeholders also envisioned future systems that could create more immersive sensory experiences–e.g., by visualizing spiritual support networks and all the specific types of support they can provide (ps. check out this awesome viz project by Avleen Kaur on the topic!)–or that could even help people come to terms with their mortality and plan for a time beyond their final days–e.g., by designing mechanisms that aid users to configure advance planning directives and to mindfully sculpt the digital legacies they will leave behind. Read the full paper or watch our video presentation to learn more about these fascinating implications.

I’ll close on the note that, for a topic like this, a scientific paper truly cannot convey the depth and richness of participants’ experiences. So, I worked with artist Laura Clapper ( to illustrate a few special quotes from our data that highlight what spiritual support means to people–both online and offline. I’ll let these stories speak for themselves, and I hope to see you at our session at #CSCW2021.

“An older woman had just been admitted, and she had a kind of a rough night and didn’t feel great. She was just near tears. An aid was in the room, and they were talking about religion. They were the same religion. And the aid got down on her knees and held the woman’s hand and she said, “Can I pray for you when I go home?” It was towards the end of the shift, and the woman, I thought she was going to cry. She just changed her whole tone. It just gave her an extra bit of hope, and I think it was a kind thing to do.”
“My husband was in the hospital, having had a massive motorcycle accident. He was one of those, “Will he make it for the first 24 hours? We’ll see…” And I had a friend start a CaringBridge site. He was in the ICU for almost two weeks. This is the description I came away with–it’s like riding the wave of love. That’s what it felt like. Both of us could feel this support, that was in the writing. Later, when people stop writing because you’ve gotten better, you can feel that diminishing. That was very, very tangible.”
“For us, receiving meals was spiritual support because the people who would come to deliver food, it wouldn’t be an expectation of sitting in our house and us entertaining them. But they would just kind of give her a hug or something. And it was quick. And it was loving. And to us, it wasn’t even about the food. It was just kind of, them doing something out of love, taking time out of their day, showing that they care.”
“When I was an oncology nurse, I had different experiences with patients, right before they’re dying. I had one moment when somebody had cancer, and she had been lying there, kind of unresponsive. But then this morning, she woke up, and I was like, “Hey, do you want to stand up? Let’s brush your teeth. Let’s get you cleaned up.” I got her back in bed, then her husband came, and I was like, awesome, he’s gonna see her awake, and she just kept smiling. And I was like, “What are you looking at? Do you see something?” And she’s like, “I see three beautiful beings.” And I said, “You look so peaceful,” and she goes, “I’m so peaceful.” I said, “You look happy, are you happy?” And she said, “I’m so happy.” And she ended up dying later that day, and I was like, the husband got to hear her say, “I’m happy, I’m at peace.”
“​​Even though I have a lot of experience providing spiritual support, one of the experiences that most sticks out to me was that my father had ALS and he was 84. The doctors said it will be relatively quick. I had been out about six weeks before he passed away, and had just started my chaplain internship. Somebody came and found me on oncology and pulled me out from a patient and said, I’m saddened to tell you this, but your dad died. And it was a… I just broke down and I said, “I thought I was ready.” This oncologist, who I didn’t think really knew who I was, or you know, was all business, stopped in her tracks and just put her arms around me and said, “Don’t worry about anything. Just go take care of yourself and your family, we’ll take care of everything else.” The night I got back from my dad’s funeral, I was on call, and I got a call at midnight, 96 year old woman, she had keeled over the family dinner. After a couple hours in the ER, doctor said, “We gotta call it, there’s not much we can do.” So we gathered the family together. So I went from caring for, to being cared for, to caring for–so, giving and receiving, all in a 72-hour period.”

Citation: Smith, C. Estelle, Avleen Kaur, Katie Z. Gach, Loren Terveen, Mary Jo Kreitzer, and Susan O’Conner-Von. “What is Spiritual Support and How Might It Impact the Design of Online Communities?” Proceedings of the ACM on Human-Computer Interaction 5, no. CSCW1 (2021): 1-42.


[1] Smith, C. Estelle, Zachary Levonian, Haiwei Ma, Robert Giaquinto, Gemma Lein-Mcdonough, Zixuan Li, Susan O’Conner-Von, and Svetlana Yarosh. “” I Cannot Do All of This Alone” Exploring Instrumental and Prayer Support in Online Health Communities.” ACM Transactions on Computer-Human Interaction (TOCHI) 27, no. 5 (2020): 1-41.

[2] Wyche, Susan P., Gillian R. Hayes, Lonnie D. Harvel, and Rebecca E. Grinter. “Technology in spiritual formation: an exploratory study of computer mediated religious communications.” In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp. 199-208. 2006.

[3] Bell, Genevieve. “No more SMS from Jesus: Ubicomp, religion and techno-spiritual practices.” In International Conference on Ubiquitous Computing, pp. 141-158. Springer, Berlin, Heidelberg, 2006.

[4] Bell, Genevieve. “Messy Futures: culture, technology and research.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2012.

[5] Buie, Elizabeth, and Mark Blythe. “Spirituality: there’s an app for that! (but not a lot of research).” In CHI’13 Extended Abstracts on Human Factors in Computing Systems, pp. 2315-2324. 2013.

13 Rules for Successful Hybrid Meetings: Lessons from 100 Articles

By on

By Lana Yarosh, Peter Genatempo, Georgie Jin, Charles Zhang, Joseph Konstan, and Loren Terveen

Three people at a meeting table and one person on a video screen joining in remotely
We have to be prepared to run meetings that include both co-present and remote participants. How do we do these hybrid meetings well? 

In-person meetings as we know them are a thing of the past. The pandemic has changed the way we work and the assumption that all the relevant people will gather together in the same room no longer holds true. We have to be prepared to run meetings that include both co-present and remote participants. How do we do these kinds of “hybrid meetings” well, given that the prior wisdom on this topic was to go all virtual or go all in-person? 

We read 100 advice articles on hybrid meetings1! Collectively, these articles identified 474 separate pieces of advice, which we’ve distilled down to a lucky 13 practices that you can implement today. As we clustered and selected among the tips, our vision was to focus on supporting those small (4-20 people) quotidian meetings where anyone may chime in and contribute (so, we omitted tips aimed more at running large events, talks, or conferences). 

Many of the insights in the articles we reviewed could be described as best practices for ANY good meeting; however these may be particularly important to orient and guide hybrid meetings. The most common piece of advice was to (1) minimize the number, length, and people at any meeting. This means to only have meetings if they are actually necessary, only include the people who actually need to be there (and break up larger meetings with multiple agenda items into smaller ones), and only focus on topics that need synchronous review and moving other content to asynchronous distribution. Second, organizers need to (2) share the pre- and post-meeting materials, including an agenda, outcome goals, presentation materials, notes, recordings, opportunities to provide feedback, and next action steps. These are things that most of us know are a good idea to do when we organize meetings, but we may not always do them for daily in-person meetings. Finally, (3) communicate to the attendees the expectations of the meeting. This may include practices like raising your hand vs. jumping in, no side conversations, no multitasking, etc.

We identified another set of tips that could be described as best practices for virtual meetings, but that are important for hybrid meetings as well. (4) Support all participants in using the communication technologies, including having a meeting participant take on a tech support role during the meeting and making sure participants have a good internet connection, a headset with a microphone, and knowledge of behaviors such as muting when not speaking and authentication/security practices for the meeting. (5) Use video when you can, but prioritize good audio above all else. For virtual meetings, you’ll need to (6) plan and communicate how participation will be moderated, including having a meeting participant take on the role of a moderator and sharing a set of practices for interruptions, asking questions, etc. Finally, if appropriate for the particular meeting, provide some (7) opportunities for informal social interaction before and/or after the meeting either in the same virtual room or using a different tool (e.g.,

Finally, here’s the new stuff that is specific to hybrid meetings! We’ll borrow some pithy terms from one of the articles, referring to in-person participants as “roomies” and virtual participants as “zoomies.” First of all, (8) consider if the meeting should actually be hybrid or if there are enough zoomies to just have it be a fully virtual meeting. If fewer than half of the people are planning to be roomies, maybe just go all virtual? If you do decide to go hybrid, you should plan to include the zoomies in all the activities, including (9) enabling the chat feature encouraging both roomies and zoomies to make use of it, (10) including zoomies in your interactive activities (e.g., brainstorming, polling), and (11) mixing roomies and zoomies in breakout groups. Hybrid meetings also require special considerations for the room where the in-person portion is held. (12) Give each virtual participant an in-room “avatar,” such as displaying them all on a large screen or having each one on a separate device “in” a seat at the table (maybe even assign an in-person partner for each virtual participant to help them move around the room). Mindfully set up the cameras and microphones in the room, understanding what others will see and hear and (13) ensuring proper positioning of microphones and cameras for the planned activities.

Reflecting on these 13 best practices, there were three things that were clear to us. First, hybrid meetings will take more time to plan well and they’re more exhausting for everybody. More than ever, it’s important to think through whether such a meeting is necessary and how to make the synchronous portions shorter, more focused, and with fewer participants. DO NOT just do a 1-to-1 transfer of all of your group’s in-person meetings to hybrid meetings. Second, in addition to the usual roles of meeting lead and scribe, hybrid meetings need an assigned person providing technical support and another facilitating interaction in an inclusive way. Typically, these will be volunteer meeting participants, but these roles need to be assigned, recognized, and supported! Finally, the in-person experience WILL be different. You can’t just tack on a virtual component to an in-person meeting. Roomies should come prepared to participate on their devices in activities like polls, adding questions to the queue, backchannel and public chat, digital versions of activities like brainstorming, and even digital elements to pre-meeting informal interaction and social activities. Will this sap some of the benefits from meeting in-person and slowly move all meetings to fully virtual? Or will there always be some committed “roomies”? Only time and further research will tell.

These points represent the “folk” wisdom on the topic of hybrid meetings. There is a lot of work to be done to empirically confirm or disconfirm these best practices. If hybrid meetings are here to stay, we will also need to be developing new technologies and environments that make these successful. There is so much to do from architectural design of meeting rooms, to better mics/cameras, to fun collaboration and interaction tools! We think this is really a compelling space for future research!

Do you have some practices that have worked well for you that we missed here? Post below!

Please cite this post as follows: Yarosh, S., Genatempo, P., Jin, Q., Zhang, C., Konstan, J., and Terveen, L. 2021. “13 Rules for Successful Hybrid Meetings: Lessons from 100 Articles.” GroupLens Blog.

1 We used private browsing mode and to Google the phrase “hybrid meetings best practices.” We reviewed the top 100 articles returned (as sorted by relevance).

Tabletop Games in the Age of Remote Collaboration

By on

(Guest written by Jan Cao, cross-posted from Jan’s website)

Now that you are (almost) vaccinated, do you remember this time last year when you had been stuck at home for two months and was aching for some human interaction? How long have you been fantasizing your next game night? Did you wonder, how other board-game enthusiasts coped with COVID-19?


Human-Aided Information Retrieval to Create a Peer Support Group Meeting List

By on

Can you think of someone familiar who has been affected by alcoholism in some way? For many of you probably the answer is yes, since about 6% of US adults ages 18 and older suffers from Alcohol Use Disorder. For many of these affected people, the Alcoholics Anonymous (AA) program has been providing a venue where they can get social support. They can share any problems they experience along the way as well as get inspired from other individuals who have built a successful recovery. Many people continue going to the meetings even though they have been sober for many years.