Dr. Daniel Knoepflmacher (Host): Welcome to On the Mind, the official podcast of the Weill Cornell Medicine Department of Psychiatry. I'm your host, Dr. Daniel Knoepflmacher. In each episode, I speak with experts in various aspects of psychiatry, psychotherapy, research, and other important topics on the mind.

Today's episode focuses on artificial intelligence and mental health care, a topic I'll admit is not entirely abstract for me. I periodically turn to AI myself, including a specialized chatbot that draws from peer-reviewed literature and finds primary sources that have helped me prepare for this podcast in the past. Not today, I'll admit, but I have used it before to back up the data that I present. That kind of tool represents just one corner of a rapidly expanding landscape. AI is now being used to streamline clinical documentation, assist with diagnostic reasoning, and even deliver CBT interventions directly to patients. There's tremendous energy and tremendous money flowing into this space.

And with it comes a real tension between two competing impulses: the genuine promise of expanding access to care during a mental health crisis and entrepreneurial hype that doesn't always put patients first. Can medicine's oldest obligation—first, do no harm—survive contact with an industry that champions rapid change and disruption as virtues? Those who need help are already voting with their thumbs.

An article published last week in JAMA Pediatrics found that a fifth of adolescents and young adults in the US now use chatbots for mental health advice. That's a 50% increase from the year before. Most of them are turning to tools like ChatGPT that are built to help make profits for their companies, not to provide safe, evidence-based treatment. This is a complex, rapidly evolving topic that demands our attention.

I'm very excited to welcome Dr. Dhruv Khullar to explore AI's promise and peril with me today. Dr. Khullar is a hospitalist here at Weill Cornell, where he is an Associate Professor of Population Health Policy and Economics, and directs the Center for the Study of Physician Practice and Leadership. He's also a contributing writer for The New Yorker, where he's written with remarkable clarity and depth about AI's potential impact on medicine and mental health care. He is, simply put, one of the best physician writers out there today. Nice to have you on the podcast, Dhruv.

Dr. Dhruv Khullar: It's great to be here. Thank you for having me.

Dr. Daniel Knoepflmacher: So, you wear multiple hats, and these are just a few, but you're a clinician, you're a researcher, you're obviously a journalist. How did your career develop into what it is today? And I'm curious what parts were intentional? Were they, you know, maybe filling aspirations that were long held for you or part of plans, or were some of these things accidental or unexpected?

Dr. Dhruv Khullar: Well, my dad is a doctor. And so, as we all know, that's a risk factor for going into medicine yourself. And so, I always kind of had the sense that I wanted to go into medicine. I saw the way that he cared for patients, the way that he was able to enter people's lives and the impact that he could have. And I knew very early on that I wanted to do something similar.

I wanted to be someone who could be there for people in those moments when they really needed you. But I also had this sense, particularly during college and afterwards, that I wanted to understand the system around healthcare in more detail. You know, I was in medical school when the Affordable Care Act was being debated and passed, and it became very clear both that the healthcare system was troubled in a lot of ways, it was expensive, it's inaccessible, it was inconvenient, but also that it was going to change pretty dramatically, whether because of that legislation or other economic and social forces.

And so, I ended up taking a year and a half away from medical school. I did three years of medical school. I went to the Kennedy School, which is a public policy school at Harvard, and then came back and finished up my medical school. And so, I had a really deep sense that I wanted public health and public policy to be part of my career. I wanted to be able to see patients, but also try to transform the healthcare system or at least contribute to that transformation in positive ways. And then, all the while, I was reading physician writers who I really admired, people like Oliver Sacks and Atul Gawande and Siddhartha Mukherjee.

And what was so impressive about them was that they were able to take these incredibly complex topics and distill them in a way that was not just informative, but was even entertaining that you could grip people and make them want to read and learn about sometimes really challenging subjects. And that became kind of the third pillar of what I wanted to do. So, I always knew that being a doctor and caring for patients would be the bedrock of what I did. But understanding the system and the incentives in a way through research became the second pillar. And then, communicating about both of those things, what patients are going through. But also, how the system is contributing to what we can and can't do for patients, to a broader audience, became that third leg that I wanted to stand up.

Dr. Daniel Knoepflmacher: Well, telling the story of what's happening in medicine is really important. I feel like it's even more important now with all of the changes and potential threats that exist. I'm curious, when you decided to write, let's say, these two stories that I referred to about AI, how did that come about? Was that an assignment from The New Yorker in either case, or were you just observing things and said, "I got to write about this"?

Dr. Dhruv Khullar: Well, most of what I write for The New Yorker is really a conversation between myself and the editors and trying to figure out what is timely, what is important, where do we have something new to contribute to what's already out there. And the first piece I wrote in the early months of 2023 was actually assigned before ChatGPT dropped at the end of 2022. And so, it kind of had to really rework that piece in a pretty substantial way. And like everything in AI, you know, something that's three years old now feels like it's from a totally different era.

And then, I wrote another piece more recently a few months ago, which still somehow feels dated already. And that was focused on diagnosis can these tools perform diagnostic reasoning in the way that clinicians do? Can they help us arrive at more correct diagnoses and avoid misdiagnoses? And both of those pieces, I think, really informed my understanding of both the real potential of these technologies, but also some of the drawbacks and risks.

Dr. Daniel Knoepflmacher: Yeah. And you manage really well to talk about the nuance because there's so much in this that I think it actually elicits polarization. And I think, with AI, there's this divide often that we see between techno-optimists who favor this rapid, unfettered adoption. And then, there's apocalyptic doomsayers who really are predicting existential peril sometimes just around the corner.

And as I talked about in the introduction, the fact is AI is already a part of our lives, and it's developing incredibly rapidly. There's benefits and risks. You're writing about this and you're thinking about this from your vantage point as a hospitalist. You're a health policy researcher, which I think is a really important part of this and, obviously, a journalist. And I'm wondering, given the work that you've done, you clearly did a lot of interviews and research, when you were writing these pieces. Where do you land? Do you lean towards one of these two poles a little more? Are you strictly down the middle? How have you come to understand this?

Dr. Dhruv Khullar: It's a great question. And what's so interesting about the question is that even the people that are closest to the technology probably end up on both sides of that spectrum. And so, it gives you a sense of how uncertain, in a way, the future is. The place where I start is recognizing that these technologies are here, they're powerful, and they're going to be a larger part of both healthcare but also society more generally. And so, we're not going to stop that transition. What we can do and what we hope to do, I think, is to try to shape how the technology is used and try to make the most of the upsides and mitigate some of the downsides that are clearly also very present. And so, I've been thinking about a couple of things.

One is that just because the technology can do something doesn't mean that it necessarily will do that thing. And so, part of what we need to be thinking about in healthcare is understanding how do we integrate these technologies in a way that enter the clinical workflow that really help both doctors, nurses, and patients. And so, part of making use of a powerful technology is integrating in the workflow in a way that is positive.

I think the other thing that is really important to note is that technologies exist in a cultural, social, and economic milieu. Let's take the example of AI scribes or other types of AI documentation. Let's say that that makes doctors twice as efficient or it takes a lot off their plates. One potential way that that could go is that doctors now have a lot more time to talk to their patients, to look them in the eye, to make those connections that many of us went into medicine for. Another way that it could go is that the expectations are to see twice or more as many patients in the same unit of time. And that has little to do with the technology itself and more about the decisions that we make about the use of the technology.

And the third thing I'll say is that there are so many claims being made about AI and what it can do. And there's still relatively little evidence, certainly in the real world, about the application of these technologies. I don't think that they should be necessarily regulated or held to a standard that we might hold a new chemotherapeutic agent to. But I do think that there needs to be real efforts to understand the downstream implications of integrating some of these technologies into clinical practice. And right now, a lot of the discussion is speculative, and it is based on the claims that companies and others are making. And we need to have a really robust infrastructure of researching what the actual impact is for patients.

Dr. Daniel Knoepflmacher: I want to get into that more as we continue to discuss this, the two Rs, which aren't just research, but also regulation, which I think would be important part of this.

.

Dr. Daniel Knoepflmacher: That said, I mean, thinking about patients using it, we use it—I admit, I've used it. Can you share about how AI is used in your life, either personally and/or professionally?

Dr. Dhruv Khullar: Sure. There's a few ways in which I use it in each of those domains. I mean, when I'm in the hospital and when I'm caring for patients, the way that I found it to be most helpful is as kind of a second opinion. And so, seeing a patient, understanding what's going on with them, thinking about that, I think, for myself. But also then, if it's a complex case or a difficult situation, asking are there things that I might be missing or what is the latest evidence on X, Y, or Z?

I think, what we should be wary of and something that I think about in the medical education space is not using it as a second opinion, but as a substitute for your own critical thinking and clinical reasoning. And I think there's a real danger of that because these things are so powerful and so facile and so fluent that the temptation is large to allow it to substitute for thinking as opposed to using it as a curbside consult in a way.

In my writing and my research life, I haven't used it for writing anything myself, partly because I don't think it's a very good writer. But it is I think a fantastic research tool and allowing you to unearth things in a much more efficient way than, let's say, a Google search or even a PubMed search. It can point you very quickly to the very specific thing that you are trying to understand. And the caveat here is it is incredibly important to verify that information because, particularly with things like citations or hyperlinks, many times it gets it wrong.

And certainly, for high stakes types of things, you want to verify the information after it is unearthed too. And then, at home, you know, I use it in all sorts of ways. Often for things like my coffee machine broke a couple weeks ago and need some help figuring out how to fix it or concocting bedtime stories for the kids. Things of that nature, I found it to be quite interesting and helpful to use.

Dr. Daniel Knoepflmacher: So even in the time that you've been working as a journalist, I imagine there's a before and after AI then for you that it's impacted—obviously, there has to be that verification and making sure that those sources are accurate. But it's altered your workflow

Dr. Dhruv Khullar: yeah, I wouldn't say enormously, but I think 5% or 10%, you know, if I had to put a number on it. For instance, if I'm starting to research a space that I don't know a whole lot about, I might ask it: who are five people in this space that might be interesting to talk to or have different perspectives? That might be an example.

Another might be something that I previously would've just gone to PubMed for or immediately gone to Google and tried to sift through which papers are potentially most relevant for what I'm trying to understand. Now, I might run that through an AI model, and it will unearth interesting things that I might have missed.

I think that I've been taken by this saying, I think, it was by a computer scientist. But he said, "Maybe it's a good idea to use AI to offload things that are boring, but not things that are hard." And so, when I think about just trying to find something or plow through something, those are things that are often quite boring. But when I think about reading a paper and synthesizing in your mind and making the connections to other ideas that you have, I'm pretty cautious about allowing AI to do those types of things. Because, I think, you know, I don't want to let that muscle atrophy, but I also think that there's a real danger of having a more homogenized way that we do science or do journalism if everything is AI-informed, AI-generated.

Dr. Daniel Knoepflmacher: And that relates to what you were saying before about it as a diagnostic tool, that using it as a second opinion versus being your primary source for that, I mean, I think it gets down to the values that we all have to have and the understanding of the psychological tendency to go to the easiest path when you're stressed and you're a resident or a physician working in a hospital. We're going to need to create an ethos really, I think, which is a whole 'nother topic.

But I want to go back to AI's role in medicine. And really kind of tying this to mental health specifically, what do you think are going to be the most salient near-term impacts that we're going to be seeing? And I think I'm asking you this in terms of both benefits and risks.

Dr. Dhruv Khullar: Yeah. Well, I think the most immediate way in which AI will be used and already is being used on the administrative side and the regulatory burden side. And so, thinking about things like AI scribes, billing, coding, risk adjustment, all those types of things, I think, are increasingly being automated and done by AI. And that is true in the mental healthcare space, but also more generally in healthcare.

The second kind of low-hanging fruit is patient navigation as you mentioned. And I think we sometimes underestimate how complex and difficult it is to navigate the system, particularly for people with chronic medical conditions or certainly serious mental illnesses. And to the extent that these tools can help people understand and maybe even prepare for their next doctor's visit, understand their lab results or imaging studies. Increasingly, we may have AI agents that are capable of performing multi-step type of processes to schedule appointments and coordinate care and so on. I think that's another huge unlock.

We've talked about drug discovery certainly in the mental health space. That's as important as it is in other parts of healthcare. But also, drug repurposing. And so, this idea that there are all these medications that we have that are approved, that are safe and effective for a particular indication, but do they have other use cases that we're not currently using?

And a good example of that is GLP-1s. I've been following some of the literature. First, the anecdotes and then the observational studies, and now the randomized control trials that GLP-1s seem to have potential to treat things like alcohol use disorder and opioid use disorder. And these medications in some form have been around for 20 years now. And so, are there any number of other medications that we could get to before a decade or two after they are approved by the FDA, and they could potentially be helping people right now? So, I'm pretty optimistic about that.

And then, the last space, I've talked about diagnosis, but I think there's room for AI to help beyond just diagnosis to things like understanding which treatments are going to potentially be helpful for which patients. We've talked about personalized medicine for many years now, but maybe this is how things actually end up coming to fruition in a way that we're able to much better match particular medications to people beyond the use cases that we do it now, which is, you know, hematology and oncology, let's say, where you see a particular mutation and you target the tumor in that way. But maybe that could hold true for any number of other parts of medicine as well. And so, in a way, sometimes it's hard to talk about AI in medicine because it does and is so many different things, but sometimes it's helpful to break down into different buckets.

Dr. Daniel Knoepflmacher: Definitely. I mean, some of those buckets I think are particularly important in mental health care. The patient navigation piece, as you know, there's not just a shortage of clinicians, but there's just tremendous wait times. And if there were things that could allow some kind of connection and not to mention streamlining to care, but also even providing some kind of temporizing care until the more comprehensive care was available, that could be really an area for growth.

You turn to diagnosis, in the piece that you wrote in September of 2025 on AI and diagnosis, you talked about how an LLM, which I think sounds like it was quite surprising to you, really could take clinical data that was quite complex and arrive in an accurate diagnosis for the most part in a fraction of the time that would be needed for the top human expert.

So, that makes us all a little uncomfortable, I think. The prospect of artificial intelligence doing integral aspects of the work we do that kind of is our pride and our identity is, I think, threatening. And you asked a question, which was the title of your piece, "If AI Can Diagnose Patients, What Are Doctors For?" I'm wondering, what's the answer to that question, do you think?

Dr. Dhruv Khullar: Well, the title itself was, I think, in some ways provocative. But there's a lot of things that doctors and other clinicians do that AI is nowhere near being able to do, and I think won't be able to do for the foreseeable future. Just a point on the premise of the question.

So, this specific AI that I went to see, it was developed at Harvard. And it was incredibly impressive in being able to crack some of the most complex cases that are written down in the New England Journal of Medicine. And a couple of the things that I came to think, beyond the impressiveness was that one of the reasons that it was so effective is because the cases were curated in a particular way and presented to the AI.

And it's not clear to me that, if the cases were not curated, if the labs weren't given, the right ones weren't given, the imaging tests weren't provided the right sequences and so on, that it would perform just as well as it did. And I think sometimes we underestimate how much of what we do as doctors is not just trying to nail the diagnosis, but curating the story, understanding which facts are important, dismissing ones that are not important. A lot of what we're doing is often fact-gathering and trying to understand the story and put it into a coherent narrative. And so, I think that's one thing that I think just is not clear to me that AI in real-world cases is going to be as good as it does in, let's say, board exams.

The second thing I would say is that AIs often have what are called kind of jagged intelligence. And so, they might be very spiky, extremely good, certainly superhuman in some capacities, but also very dull in other places where very common sense things that we just take for granted that any child would understand or be able to do, they end up failing at. You know, for a while there was meme about them not being able to count the number of Rs in strawberry, that type of thing. And that's been patched over, but there will be other things similar to that.

And then I think, doctors, there will always be a need to adjudicate the output that comes from an LLM, let's say. So, it may give you a series of differential diagnoses, or it may give you what its opinion is or pull together the medical evidence and point you towards a particular path of action. But someone needs to be able to take all that in and use their judgment to make the use of a very powerful tool. And I think doctors are going to continue to be those people.

And I'll just mention a couple other things. There will be a role for someone to manage the uncertainty with any diagnosis or any treatment plan. So, we have to manage the uncertainty. We have to elicit patient's values and preferences, try to understand what is important to them. And then, at the end of the day, someone needs to take responsibility for all that care. And people want, I think, a human, a person, who they can look at, who they understand, has their best interest in mind to take responsibility for the care that is being delivered. So, I guess that's a long-winded way of saying there's still a lot of things that we need doctors for and that we will need doctors for.

Dr. Daniel Knoepflmacher: You're speaking about humanism ultimately, which I think is really an important thing that we need to preserve as central to all medical care to all mental health care right now, because I do think there's some threats to that.

I want to stick to diagnosis for a moment longer. In psychiatry, diagnosis relies on interviewing skills, on reviewing history, performing a mental status exam, and all of this is something which takes a lot of time to master. I mean, I'm a residency training director. This is something that we start teaching residents , and in medical students too, from the very beginning.

That's very different than work in radiology, which is very important, but takes a different kind of pattern recognition, but maybe is something which image analysis through AI could do in a different way. And I'm just wondering, maybe this is just me being biased from a psychiatric standpoint, but I'm thinking about our incredibly human-driven process of psychiatric diagnosis, in some ways because we're dealing with a lot more gray than in other medical specialties. If it's going to be a more difficult target for AI integration, and I realize maybe it's the opposite, maybe because there's so much grayness, it's actually a great target, because maybe there's new tools like imaging data or voice and facial expression analysis that could be done with AI that we can't really do well as we're listening to somebody's story.

So, I'm just curious, based on what you've seen with AI adoption in general, do you have an idea of how that could play out, let's say, in psychiatry?

Dr. Dhruv Khullar: A few years ago, before ChatGPT was released, I think the obvious thing to say would be that radiology and pathology would be the places that would be most disrupted. But so much of AI, particularly the way that the public experiences it, is through chatbots, which are fundamentally about language.

And psychiatry is people who are skilled in the use of language and directing people's thoughts through language. And so, in a way, it seems like parts of psychiatry or parts of mental health care are potentially amenable to these types of things. I think there are many reasons why it's certainly not a substitute. We should talk about sycophancy and we should talk about the relational aspects of care and knowing when to push and when to support and all these things. And so, I don't think that it's, in any way, a substitute and it won't be. But I think that parts of psychiatry and other parts of, let's say, non-procedural specialties as well, because a lot of it is language-based, it is ripe for AI.

One thing that I think is interesting that I came across recently just goes to show how unpredictable the future can be. You know, if we were talking 10 years ago, we might say that radiology—in fact, Geoffrey Hinton famously predicted that we should stop training radiologists in 2015 or '16 because we won't need them, and I think Elon Musk around the same time said that we won't need cars. All the cars will be self-driving cars. And those were predictions made a decade ago. And I saw a meme recently that just showed radiologists driving themselves to work, sitting in traffic. And as I understand it, there are more radiologists practicing in the US today, and they are busier than ever, and their incomes are higher. And so, we don't know how these things will play out.

And I think in the short term, what we can do is, from a medical education standpoint, take a kind of a conservative approach to training. By which I mean, it's clear that we should familiarize people with these tools and help them understand the potential of them, but it's not clear what we should stop teaching people or what we should take off people's plate in terms of the types of knowledge or critical reasoning that they need to themselves have, as they're moving through the medical training process.

Dr. Daniel Knoepflmacher: Well, that's the risk of cognitive de-skilling that you've talked about. Any thoughts on if and when AI is really doing more work in diagnosis, how can we create a culture of learning that prevents that? What do you think as we're training this future generation of clinicians, how can we be sure that they'll be able to think for themselves?

Dr. Dhruv Khullar: Yeah. We've talked about cognitive de-skilling where you had skills and then they maybe atrophied over time. I'm also worried about cognitive foreclosure where you never even develop the skills in the first place. Some of this can sound like an older generation kind of shaking its fist at a younger generation, but I don't think that's entirely true. I mean, there are certainly some things that technology has made less important over time. You know, we were probably better at listening to the heart in the past or palpating the liver. And now, we have echocardiograms and MRIs and CT scans. And I think people don't see that as a huge loss. But I do think that there's something different about the critical reasoning skills that are required to make a diagnosis or to walk someone through, let's say, a CBT or a complex medical procedure that we really want to preserve in clinicians.

And again, I think, what we should be focused on right now is making sure that people have a sense of the power of these tools but also the limitations of these tools, that they're integrating them in ways that are useful, and maybe take away some of the routine tasks that people have been burdened by. I'm thinking about discharge summaries and clinical documentation and chart summarization. But be extremely cautious about displacing the critical thinking that goes into understanding what's going on with the patient and what that patient needs.

And I should note, as another complicating factor, sometimes we think documentation is totally superfluous and distinct from the job of medicine. But there's a lot of doctors, myself included, that really think through the case when you're writing the note, because you realize that things aren't stacking up the way that you initially thought they were, that you missed this lab finding or you recall something that the patient said. And when you're trying to write it down in a coherent story, you see some of the parts are not fitting quite right. And so, there is some critical thinking that goes in there. And so, I don't think it's quite as easy as just saying, "Thank goodness we no longer have to write notes, but we'll preserve every bit of the cognitive work that we were doing before."

Dr. Daniel Knoepflmacher: Not to mention the people who are reading your notes want to know your thinking.

Dr. Dhruv Khullar: What you thought. Exactly. Exactly.

Dr. Daniel Knoepflmacher: Well, I want to turn now to the article that you wrote in 2023, that seems like a lifetime ago when it comes to AI. We've talked about all of the rapid changes. In it, you actually talked about the surprising sense of connection that people can establish with a chatbot. Not just socially, but even people expressing that they felt a therapeutic benefit.

Again, now that was written when ChatGPT had just come out. So in this time, I am not sure what you've been tracking in your work since then, but can you elaborate, I guess, on what you learned from writing that article and how things look to you now after over three years of rapid AI development?

Dr. Dhruv Khullar: One of the things that became really apparent was how long we've been trying to do this. So, there were chatbots back in the 1960s where people were trying to deliver some version of therapy via an automated chatbot. So, this has been a half-century-long, journey. When I wrote that piece, starting in '22, '23, I was focused on a different chatbot, which was called Woebot, which actually no longer even has a patient-oriented side of it anymore because it's been kind of displaced by these newer models. And it used some natural language processing, but was fundamentally based on scripts that were developed by psychologists and writers to respond to a patient and help them work through, let's say, a course of CBT.

But even in that primitive—relatively primitive now—chatbot, what was so interesting as I started using it was that, when we are interacting with something that uses pronouns like I or we. And when it is even simulating empathy, how easy, how natural it is for humans to fall into a relationship with something like that.

I mean, already so much of our interaction with other people, real people, is digital. We're always texting and we're always looking at social media and we're doing things that are not face-to-face. And so, it's even more perhaps easy than in the past to develop a relationship with relatively primitive chatbots. And now, these chatbots are so much more fluent and so much more effective and so much more flexible than they ever were in the past. And I think that creates a real opportunity. You can develop a connection. They're much more engaging, and perhaps certain types of therapy can be delivered. Certainly, it can be delivered at hours and days that a human can't be there with you.

But there's also a lot of cause for concern because there's risks of, let's say, over-relying on chatbots, of being manipulated by them or manipulating them. We don't know that it's healthy to have, you know, long-term digital relationships with these types of models, and do they displace other forms of real human interaction? And so, I'm actually quite worried that there's going to be these types of parasocial relationships. Which maybe in narrow cases can be quite helpful for people. But we are running kind of a mass experiment in a way where people are now having the types of relationships they might have had with humans with machines.

Dr. Daniel Knoepflmacher: I share your worries. I think that we are connecting beings, and there is the attachment that develops in development and during development. And we already are seeing, in fact, there's just some legislation coming before the New York State Legislature, trying to regulate the use of chatbots in toys for young children, which if you can imagine that level of interactivity, what that could sap from the ability to go out and meet actual human beings that you can develop attachments with, it's quite concerning.

Dr. Dhruv Khullar: Yeah. And I think it can even reset your expectations for the ways that other humans should be interacting with you. You know, you can imagine if a child or even adult is interacting with something that's always giving them what they want and a response to them all the time and doesn't present any friction in those interactions. And then, you talk to a real human and they have their own ideas and own preferences, that can make the latter less attractive to people and actually lead to, I think, a world in which people just end up preferring things that are frictionless and easy, but ultimately, much shallower

Dr. Daniel Knoepflmacher: So, that's in the toy, but we're seeing that actually in the psychotherapy space too, that sycophancy that you're describing. And certainly, that type of approach can be reassuring when someone is in a moment of distress and they're using a chatbot. And actually in that data I mentioned in the beginning from the JAMA article, most of those young adults and adolescents felt positively about those interactions or most of those interactions with chatbots.

But that doesn't mean the chatbot is providing therapy or is even frankly therapeutic. because therapy, as you know and as you said, like if we just have these perfect great relationships from toys, what about the uncomfortable messy relationships that exist out in the world and that exist in our own psychology? And really, therapy is often uncomfortable. It's part of a delicate process of recovery, self-understanding, and doing behavioral change, which is really difficult, but is powered by this therapeutic alliance that exists between a therapist and a patient. And you have written about—you've alluded to this already—the crucial humanistic elements of medicine, that special encounter that's central to healing.

So, how can we square that fundamentally human aspect of mental health care with the adoption of AI chatbots for therapy? It's different than a computer that's interpreting an EKG because if we're starting to hack these inherently human parts of our work, what's left behind? And what impact is that going to have on our patients, as human beings?

Dr. Dhruv Khullar: Yeah. One thing that it calls to mind is the idea of sycophancy that you raised. And the fact that sometimes what's really important is kind of a form of tough love actually, and pointing out where there's a problem that someone may not be realizing that they need to address.

I'm remembering a case that a patient I spoke to shared. She was struggling with alcohol use disorder. She'd struggled for many decades, and it was getting out of control. And about a year ago, her primary care doctor, she called her out. She said, "This is really a problem, and it's something that needs to be addressed." And that patient ended up stopped drinking, quite abruptly in fact, and ended up having alcohol withdrawal. She messaged the doctor. The doctor responded immediately, said, "You know, I'm here for you. If you want to come into the emergency room, I will meet you in the emergency room." And she ended up staying home. But she, you know, checked in with her doctor. Her doctor checked in all evening. She got through the withdrawal. And the patient was ultimately very grateful that, number one, that she'd been called out; number two, that someone had been there for her during the most challenging moments. And now, her life was better for it. And it's hard to see how a chatbot, or at least the chatbots the way that they're constructed today, could offer that type of care to a person.

So, I think, your point around sometimes what's needed is not certainly not sycophancy. It's something quite different, I also think that we should be quite skeptical of things that try to displace or replace relationships in our lives, whether those are clinical relationships or whether those are familial relationships or social relationships. And to the extent that these tools can be used as just that, as tools, as opposed to substitutes for relationships.

In psychiatry or in medicine, if they are being used in a, purposeful way to accomplish a particular goal to implement some type of behavior change or to get people to, recognize or make things easier for them in certain ways, that's all great. But when we say these things are now going to be a substitute for the doctor that you would have had, or to be a companion in some sort of way, or you should have a direct relationship that's not based on, "Let's manage your heart failure" or "Let's make sure that you adhere to this particular medication," that's where I think things get really tricky, and we should be quite wary of that.

Dr. Daniel Knoepflmacher: And I think when the producers of these products, because they are products, with profit being a motivator, it does not necessarily champion that human element. And I want to ask you another piece about this related to it being profit-driven, which is that it's difficult, I would say, to make a distinction amongst all these products that are out there, that they're being marketed as mental health tools blurs into wellness. And some of this is built on solid clinical evidence. Others are wellness products that are dressed up in clinical language.

And in the age of MAHA and with the erosion of trust and authority placed in the medical establishment, I don't know that people necessarily are that good at understanding a meaningful distinction between an evidence-based mental healthcare app, let's say, on one hand, versus other things that might be affirmations or journaling prompts or maybe even meditation, which is healthful, but is not necessarily applied in the same way as an evidence-based treatment. So, what can we do to help people understand this? Because it's complicated.

Dr. Dhruv Khullar: Yeah. I think one thing that is really important to point out is what a company or product's business model is really can show you what the incentive is. If, for instance, a lot of mental health startups, the business model or the way that they've made money, at least some of them, is by writing prescriptions. The more pills that they prescribe, the more money they make. And writing prescriptions is always going to be more scalable than building relationships.

And so, just understanding something about the business model of a company, I think, it's not that necessarily all these types of companies are bad or that they're not fulfilling a need that actually needs to be met. But where things can go awry is when the business model is pointing you in one direction and the care for the patient is pointing you in a different direction. So, that's the first thing that I would say is that we need to understand what the incentives are. And that's true of AI companies, and that's true of other companies as well.

The other thing that I think would be really helpful is just trying to help people understand the distinctions between levels of evidence when recommendations or products are being proffered. And so, it is not to say that every product on the market is going to have a double-blinded, placebo-controlled, randomized controlled trial, and that's fine. But there's a huge difference between something has interesting animal studies, something has particular theoretical reasons that it might work. We have lots of anecdotes. We have some observational studies. We have other types of evidence, I think, that this might help or work. But helping people understand the level of evidence behind a particular recommendation, a particular product, I think, that can potentially go a long way as well.

And the level of evidence that we require depends on how risky the intervention is. So if someone's telling you to, let's say, drink more water or take time to exercise more or something like this, those are not particularly risky interventions. And so, we might look at them differently than someone telling you to take this peptide or take this supplement.

Dr. Daniel Knoepflmacher: So, coming up with some kind of structure and standards really that we can communicate clearly. Well, do you have ideas, since you've worked in public policy and research, about really concrete things we can do to help enact some of what you just described? And maybe that's through regulatory efforts, policy. I mean, I'm just curious what you think might be ways for us to navigate through this uncertain future.

Dr. Dhruv Khullar: I think the first thing I would say is that we should really emphasize that we want to generate a strong evidence base behind AI-driven interventions. And so, to the extent that claims are being made, and things are being piloted in hospitals and clinics, there should be some type of evidence that's collected on them and then presented in the way that we've done for a long time.

The other thing that I would say is that AI algorithms are different than drugs and typical devices, in part because they can make certain types of decisions by themselves in the way that your pacemaker can't, and also because they continue to evolve over time. And so, there may need to be a more of a regulatory regime that is focused not just on pre-market clearance before something enters the healthcare setting, but post-market surveillance. We want to see what's happening after these models are deployed. How do they change over time? Are there biases that crop up that need to be dealt with? Should some of them be rolled back? Should other ones be promulgated more widely?

And so, I think the other big part—one is evidence generation, second is post-market surveillance. And then, I think, the third thing that I would say is we should start with places where we feel we can get kind of the biggest bang for our buck. And so, there are many parts of the healthcare system that are tremendously inefficient, that are costly both for patients and for clinicians. Of course, everyone would love a future in which diagnoses and treatments are done in a way that's informed by AI and we get the right answer every time.

But in the short term, I think there's a lot of waste in the system that we could take out. There's a lot of inconvenience that we could take out, and those are places in the short term that I think AI has a really large potential to start changing things in a positive direction.

Dr. Daniel Knoepflmacher: Well, Dhruv, thank you so much for coming on today and talking about this. This has really been instructive for me. And just thinking about what you just proposed there, I hope to see some of this enacted, and it's really something which, as a field, in medicine and in psychiatry specifically related to mental health, this is something we're really going to have to organize and try to do in a thoughtful way because, as we're discussing, it's already happening and there's a lot of questions that remain. But thank you, as always, for your cogent analysis, your clear thinking on all of this. It was really wonderful to speak to you.

Dr. Dhruv Khullar: Oh, well, thanks so much for having me. This was a real pleasure.

Dr. Daniel Knoepflmacher: Thank you very much. And thank you to all who listened to this episode of On the Mind, the official podcast of the Weill Cornell Medicine Department of Psychiatry. Our podcast is available on all major audio streaming platforms. If you like what you heard today, please give us a rating and subscribe so you can stay up-to-date with all of our latest episodes. And tell your friends. We'll be back again next month with another episode. And until then, wishing you good health in body and mind.

kids health promo: Every parent wants what's best for their children. But in the age of the internet, it can be difficult to navigate what is actually fact-based or pure speculation. Cut through the noise with Kids Health Cast, featuring Weill Cornell Medicine's expert physicians and researchers discussing a wide range of health topics, providing information on the latest medical science.

Finally, a podcast to help you make informed choices for your family's health and wellness. Subscribe wherever you listen to podcasts. Also, don't forget to rate us five stars.

disclaimer: All information contained in this podcast is intended for informational and educational purposes. The information is not intended nor suited to be a replacement or substitute for professional medical treatment or for professional medical advice relative to a specific medical question or condition. We urge you to always seek the advice of your physician or medical professional with respect to your medical condition or questions. Weill Cornell Medicine makes no warranty, guarantee, or representation as to the accuracy or sufficiency of the information featured in this podcast, and any reliance on such information is done at your own risk.

Participants may have consulting, equity, board membership, or other relationships with pharmaceutical, biotech, or device companies unrelated to their role in this podcast. No payments have been made by any company to endorse any treatments, devices, or procedures. And Weill Cornell Medicine does not endorse, approve, or recommend any product, service, or entity mentioned in this podcast.

Opinions expressed in this podcast are those of the speaker and do not represent the perspectives of Weill Cornell Medicine as an institution.