AI and Mental Health – Ep. 3

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email with any questions.

Nick Jacobson: In terms of guardrails that get invoked. We have made a decision with their bot to not shut down a conversation when it happens. And actually, a lot of our training data is specifically developing to try to actually deliver crisis intervention when folks are in moments of crisis. So that is something that we’ve thought a lot about and wanted to do, in large part because if folks are essentially willing to use their body as a system, they may not be willing to actually have levels of escalated care or crisis that they are willing to engage with, and I’d rather not withhold a resource that they are willing to engage with. I think it lowers the likelihood of clinical outcomes.

Stephanie Hepburn: This is CrisisTalk. I’m your host, Stephanie Hepburn. This episode is more technical, but bear with us in today’s conversation. I’m speaking with Nick Jacobson, an associate professor at Dartmouth and lead developer of Therabot, a generative AI chatbot in beta testing that’s designed to provide mental health interventions. We cover the clinical outcomes of therapy. What happens when a person develops an emotional attachment to AI? The difference between rule based and generative AI, and the layers of safeguards needed to provide mental health support. A quick conflict of interest disclaimer I discovered my books are part of a pirated ebook database called Libgen. Used to train AI chatbots I am part of a class action lawsuit against anthropic, the creator of the AI chatbot. Claude. Let’s jump in.

Nick Jacobson: My name is Nick Jacobson. I’m an associate professor of biomedical data science, psychiatry, and computer science here at Dartmouth. I am a clinical psychologist by training, and I and my lab focused on using technology to enhance the assessment and treatment of mental health. A lot of our work in the last six years has been in using generative AI to try to provide evidence based treatment for mental health.

Stephanie Hepburn: So, Nick, can you tell me a little bit about Therabot? It’s a generative AI therapist that’s in beta. It’s not accessible to the public yet. What does it do that’s different?

Nick Jacobson: In large part, it’s really specifically trained to deliver evidence based treatments. So a lot of what folks will do in this space, the most popular approach to folks that would be getting into this kind of area, and a lot of how folks are using AI for mental health is just by prompting generative AI with a system prompt to act as a therapist. So that would be like a prompt engineering style way of trying to get this. So you might say something like act as a psychotherapist. You could potentially even mention techniques or things like that that you’d want the AI to be using. That’s a popular way that folks use it right now.

Stephanie Hepburn: You mean just using general ChatGPT, the large language models that exist? Exactly. And okay, I see what you’re saying.

Nick Jacobson: Yes. That’s like the most common approach that folks are accessing AI for their mental health. About half the mental health population is really using it in that way. Based on some emerging evidence. That’s the most common way that folks are accessing it. It’s probably the the least effective in producing something that is evidence based responses. Another technique that folks will use is something that is similar in style. You have that, but then you add a document or a series of documents that is a repository that AI could have some additional context to refer to. The most popular of these techniques is called retrieval-augmented generation. It sounds really fancy, but it’s actually really simple as a technique. Essentially what would happen is you’d upload a document and then the AI would look for something that is semantically similar and try to inject that into the system prompt as some context before it responds. And what we are doing is essentially adding training data and changing the model weights themselves, meaning we’re changing how the model behaves like its internal knowledge, how it acts. And so we are engaging in a process called expert supervised fine-tuning, meaning we are not just doing fine-tuning data, but actually making sure that it is actually done based on expert data. And so this essentially affects the core internal working knowledge of these models. So rather than relying on a prompt or something like that that might be on the surface, this essentially steers the model exactly in the ways that they’re trying to emulate the way that they’re trained to actually work.

Stephanie Hepburn: Are you building off of an existing LLM? Are you building off of ChatGPT? Claude Llama and then building this infrastructure on layering it on top, or what does that look like in order to infuse it with the training model?

Nick Jacobson: So we work with a series of models. So not just one model but multiple. The models themselves change over time. So new models have continued to be put out on a very regular pace. And fortunately we can adapt and improve over time. So it’s something that we continue to work on. The models that we tested within the randomized controlled trial where we tested this over the past summer, there’s an open source model called Falcon, and then another model was Llama 2, which at this point has gone through major revisions from the foundation model. But these are essentially for folks that are not in this space, open source versions of some kinds of variants that are used similar in style to ChatGPT, and that they’re large open domain areas that we’re then adapting to our space.

Stephanie Hepburn: So when I tested out the different models ChatGPT or Claude, for example, what I’ve seen is over time that there are different internal guardrails. Right now, this space is not federally regulated. The internal guardrails shift. So even when I’ve tested it out, for example, months apart, there’s some changes that happen. Sometimes they halt dialogue. They halt further engagement with the person who’s using language, and even the language that they use has some variation. What triggers the guardrail shifts as well. So there have been times where I typed in something to the effect of I don’t want to be here anymore. That triggered the guardrail and others. It has to be really direct. It has to use trigger keywords. If you’re shifting from model to model, how do you ensure that it’s broader than that, not just trigger words?

Nick Jacobson: Yeah. So we have different ways of essentially developing these models and different ways of overseeing it. Essentially, the data themselves are consistent across models, and that allows a lot of their behavior to actually be far more similar over time. A lot of the things that are changing with the models themselves most directly has been their intellectual capacity, as opposed to the major ways that they behave or are steered.

Stephanie Hepburn: If we could take a step back in terms of rule based versus generative AI. It sounds like in the mental health space, historically it has been rule based. What are the strengths and flaws? I mean, I would assume one is with generative AI, you have more interaction. It can feel more natural, it can be more layered, for example, but also more inconsistent. So you might get varying responses. Can you just explain the differences and why you went in the direction of generative AI?

Nick Jacobson: Yeah, sure. So I’ve been working with rule based chatbots for about the past decade. So before we did work on generative AI, we worked in that for listeners that maybe have not heard these terms. I imagine every listener has actually interacted with a rule based chatbot or conversational agent, which is just an auditory version, potentially. So you might have called, say, your cable provider, and they’re very common in customer service settings. If you call your customer service provider and you’ve got these like menu of routing options to try to search through, you’re navigating a rule based chatbot. That’s a rule based chatbot just interacted with through voice. Essentially, I think you get a good feel for how well that works and how well it doesn’t. The system won’t say something that you don’t intend, but that’s inherent to both. It’s safety, but it’s also fundamentally limited. And the experience that you can have, whoever develops this has to essentially anticipate everything that the chatbot will say, and all of the potential routing options and conversations that could be had. And because of that, it can be a frustrating experience for a lot of users, in large part because they might want their responses to actually be adapted and more real ways than might be provided through routing options. It might be literally trying to predict whether you said yes or no, just to try to navigate you through that system. So compared to something where it’s like actually changing what the response is. It’s pretty different technology. The challenge is that they’re pretty limited in terms of what you can do. And ultimately kind of frustrating.

Stephanie Hepburn: So when we talked about what their pod can deliver, what does that look like to help the user, whether they’re in crisis or just navigating a challenging moment? What is there about able to do?

Nick Jacobson: Yeah. So it’s been trained on cognitive behavioral therapy and what’s sometimes called third wave variants of that. So a lot of modern cognitive behavioral therapies, the exact strategy that you’ll get will depend exactly on what you’re going through. So we try to actually make sure that what the user is expressing. So a problem that they might be experiencing is directly matched to the the type of intervention that therapy would be providing. So for example, if you’ve got social anxiety, high arousal, discomfort surrounding social experiences paired with avoidance, it will try to go into exposure, and so really try to walk you through the concept of essentially trying to go and actually meet and experience those fears, learning that the distress is inherently something that will pass over time. And the more that you approach this discomfort, the more you realize that you can tolerate it. And starting with low levels and walking folks up, what is sometimes called an exposure hierarchy is just an example where folks try to take bigger and bigger steps over time. So yeah, that’s an example of an intervention that folks might experience within the context of using it, but it really depends exactly on what they’re experiencing and what they’re going through. And so the experience can be really different from one user to the next based on what they’re actually going through.

Stephanie Hepburn: So it’s tailored towards the individual by nature that you have this generative aspect to it.

Nick Jacobson: Yeah, absolutely.

Stephanie Hepburn: So OpenAI last year released some of the social impacts of using ChatGPT. And one of them was this emotional reliance on AI models. How do you address that? Especially when people are going to it with some level of dependence, using it as an intervention, how do you ensure that they’re reminded that they’re not actually fostering an emotional connection with the AI?

Nick Jacobson: So that is an interesting one, actually. We are finding that folks are forming an attachment to their bot, and I actually view that as fundamentally the only way that they could benefit from it. So some degree of attachment and reliance and trust ultimately that therapy could help them. And that’s essentially that is the most well-studied construct in how psychotherapy works. And so we actually in our trial, for example, measured that and found folks had really high levels of that with their bot. And we need more work in this area. But based on how we see the evidence of psychotherapy working in other settings, my suspicion is that is one of the things that underlies why therapy was so helpful. So we don’t want to take that away. That type of reliance between essentially the therapeutic interactions. We don’t want to foster a dependency in the way that folks might be with a companion. And I think a lot of that is in really clear roles. So I think a lot of the way that we try to actually keep roles is really similar to how psychologists actually work. And so one of these professional decorum is really a role of technical neutrality, meaning Therabot doesn’t bring an express aspects of itself. And so it’s trained not to, for example, develop reciprocal companionship, affection. So the user asks or says something like, I love you Therabot won’t respond with, oh, I love you too, and then try to develop this kind of style of intervention. If you do this to a psychologist, a lot of how this is studied systematically is in a process called transference, where folks actually have a lot of these experiences in bonds that often happen in these types of settings.

Nick Jacobson: And so really trying to manage that relationship in ways that don’t involve a change in the functional role that you want to have. So somebody says, I love you asking, oh, it’s interesting to hear you say that. Can you tell me more about what makes you feel that way? So really diving more into the person as opposed to bringing something or changing the function or the role of that plane of blurring lines. So a lot of how we’ve approached this is trying to keep that consistent, to try to keep that relationship ultimately professional, but also something that really the focus is on that individual as opposed to bringing something about the relationship from the AI. But I think the other piece of things is a lot of this is really intense. We’re often engaging in things that are not easy for the user and systematically actually hard. So in the trial, we didn’t see folks that were seeming to foster some level of problematic engagement or things like that. And in large ways, if you have a good therapy session when you walk out of it kind of feels exhausting. So a lot of these interactions, I think with therapy, although they can be really helpful, they’re intense and they’re often engaging folks and things that are actually really hard for them. So in that way, I think you don’t see as much of this style of problematic use in terms of Stephanie, around your question, around like particular guardrails that get invoked. We have made a decision with therapy to not shut down a conversation when it happens.

Nick Jacobson: And actually, a lot of our training data is specifically developed to try to actually deliver crisis intervention when folks are in moments of crisis. So that is something that we’ve thought a lot about and wanted to do, in large part because if folks are essentially willing to use their body as a system, they may not be willing to actually have levels of escalated care or crisis that they are willing to engage with, and I’d rather not withhold a resource that they are willing to engage with. I think it lowers the likelihood of clinical outcomes. But when somebody is messaging our application about this, we have a lot of risk-classification models that are also trying to predict the level of risk that somebody might be in at a given time. And a lot of those give us a sense of the different type of risk. So something that is very acute and clear. So something like I want to end my life in the near future, but also common experiences that happen across psychiatric populations, like thoughts of death with no intent to act on it. And so essentially we’ve got different types of classification that happens based on the level of clear intent around these topics. And so we do this with things like suicide, self-harm, aggression and homicidal behavior, but also substance use and eating disorder behavior that might put folks at imminent clinical risk. And so these are just some examples of those types of interventions that we’re looking at. And when that gets triggered the UI changes. So there is a crisis button that starts to flash. And the UI and.

Stephanie Hepburn: UI being user interface.

Nick Jacobson: Yeah. The app, there’s all of a sudden a red button that starts to flash within the application. And the user is then directed to call 911, call 988 or text crisis text line. And it tries to make it easier for the user to actually do that. We’re trying to allow for escalation of resources in those moments. But on top of that, we also, as a team will be paged when those go off. So we’ve got systems of review to make sure that we are seeing those types of messages in real time. And then the humans would actually then from the research team would actually reach out to those participants and probably very traditional ways that they might do in research settings when you might be exposed to risk.

Stephanie Hepburn: So, Nick, are you using an algorithm to determine keywords language? And then you have the human element of your actual research team being able to see pop ups make determinations from there. Also intervene if necessary. What does that look like? Does it start with an algorithm? What is underlying it?

Nick Jacobson: Yeah. So I guess within the context of the trial, we monitored every message that was sent to or from their bot in near real time, regardless of any algorithm that was implemented, in large part because this was the first trial of generative AI that’s really designed for mental health. And we want to make sure that even though we’ve been developing it for six years, with over 100 people spending 100,000 human hours getting into this, that it actually acts safely. And so that was really where we saw the state of the evidence was to try to really have heavy, heavy oversight of everything it said. And future trials will increasingly focus on these algorithms that have been working exceptionally well and actually classifying risk. We’ll continue to have oversight of these models in human ways, but increasingly rely on the real time aspects of it for this type of oversight to make sure the system actually scales. Like a lot of why we’re interested in getting into this is ultimately to provide evidence based care that could be personalized and dynamic, which is great, but also scalable. A lot of why this is interesting is really to try to make sure that it actually could increase access to care. And so I think a big part of that is actually trying to figure out ways that we can essentially effectively scale it without increasing the rate of harm. And so a lot of this we need to continue to do and evaluate based on the evidence. But we had higher levels of oversight in the initial trial. But in future trials based on how both the system itself is performing, but also how all of our risk classification was working, will have still oversight of these models, but more heavily reliant on some of the other methods we have and safeguards.

Stephanie Hepburn: Can you tell me a little bit about the outcomes, what successes you’re experiencing, where you’d like to take it next?

Nick Jacobson: Yeah. So we put it to test in a randomized controlled trial last year. We tested it in folks with depression, clinical anxiety, folks that were high risk for feeding and eating disorders and looked at essentially the changes in symptoms that happened within the theropod arm versus a weightless control arm. So they were randomized to essentially receive access to the robot or not. And then we looked at the degree of change that happened and the folks that had access to therapy versus those that didn’t. And we saw some really large symptom reductions in depression, anxiety, and eating disorder symptoms. And the magnitudes were really large in terms of how the effects look within psychotherapy research and really medication treatment work and psychiatry, we’re seeing the types of effects that are about as large as you would see, and kind of our gold best standard care that we would try to provide in these types of settings. So really promising in a lot of ways. The thing that I think is so interesting about generative AI for mental health is we have trained their bot to not only treat depression, anxiety, and eating disorders, but all of the things that are comorbid with that, which is just about everything.

Nick Jacobson: And so one of the things that I’m really excited about is looking at how the robot does in other populations. So we have evidence that it seems to work well for depression, anxiety and eating disorder symptoms. But this is a starting point, and I’m really interested in seeing how well it does in other comorbidities. So we’ve got trials ongoing in other areas. So substance use disorders for example. And then in other populations why I’m interested in this is it could potentially scale a lot of different treatment to provide something that’s readily available almost regardless of what folks are actually experiencing in ways that are evidence based, essentially trying to train their body to be and test their basic competency and the range of what your average psychotherapist might be able to treat. And when you would go into a clinic in that way, that’s ultimately a lot of the goals of the work that we’re trying to do with therapy, but also in my lab more broadly, is actually just trying to make care more available, easier and more scalable. So that’s a lot of our interest and excitement there.

Stephanie Hepburn: That was Nick Jacobson, the lead developer of Therabot, a generative AI chatbot designed to provide evidence based mental health treatment. If you enjoyed this episode, please subscribe and leave us a review. Wherever you listen to the podcast, it helps others find the show. Thanks for listening. I’m your host and producer. Our associate producer is Rin Koenig. Audio engineering by Chris Mann. Music is vinyl couch by Blue Dot Sessions.