Carl Maronich (Host): Welcome to the Healthcare Executive podcast, providing insightful commentary and developments in the world of healthcare leadership. To learn more, visit ache.org. I'm Carl Maronich. And with me is Dr. Matthew Montoya from Johns Hopkins University, where he leads graduate programs in Systems Engineering and Healthcare Systems Engineering.

In addition, we're joined by Dr. David Rastall, also from Johns Hopkins University, where he is an AI leader and founder of the Max Lab. His work examines how AI systems behave in real-world clinical settings and focus on safety, implementation risk, and performance. This esteemed pair joins us today for a conversation on AI-enabled health systems and what it takes to design, implement, and scale AI safely and effectively across modern healthcare organizations. Doctors, welcome to the podcast.

Matthew Montoya, DrEng: Glad to be here.

David P. Rastall, DO, PhD: It's an honor to be here.

Host: Well, we appreciate that. And David, I think I'm going to start with you. You know, healthcare leaders are being pulled in every direction when it comes to AI right now—vendors, boards, regulators, clinicians. From your experience, what is the single biggest misconception leaders bring into that conversation and why does it matter so much?

David P. Rastall, DO, PhD: I think a lot of leaders are finding themself in a really difficult situation right now, because if they act too quickly without knowing how to successfully deploy AI systems, they can put their health system at risk. But if they act too slow, they're losing the competitive advantage to their competitors.

And so, it's really important at this moment to understand artificial intelligence, and I think the single biggest misconception is that treating this like any other technology. If you treat AI like an MRI machine, you can spec it, install it, and it performs the same way over time, and how it performed in someone else's healthcare setting is roughly how it will perform in yours. But AI is not like that. AI behaves differently depending on the workflow, the data that's going into it, and even the incentives and the people around it, and it changes those things as well. So, it needs to be treated differently than other technologies.

Host: And Matt, both of you come from Johns Hopkins, one from systems engineering, one from patient safety. How did these two different disciplines end up looking at healthcare AI and arriving at the same conclusion?

Matthew Montoya, DrEng: We actually have a third partner who looks at the financials and ROIs as well. And you know, all three of us came to the similar conclusions. And the way that I like to approach things, and I think David and Mike as well, is we look at first principles. And the first principles is not what tool do you need? The first principles is what problem are you trying to solve? And when you approach things from a problem-solving perspective, not reaching for solutions, not trying to grab quick win, that's going to bring you to the same type of solutions. You're going to be looking at problem-solving. You're going to be looking at what do we need to do. You're going to be looking at scenarios. You're going to be looking at the top level, and you're going to be looking and asking the basic questions of what problem are we trying to solve? When you approach it that way, as opposed to what tool do I need, most times you will end up at the same place.

And so for me, from a systems perspective, from David, from a patient safety, and Mike, from an ROI and a financial perspective, you tend to converge if you look at it from that level.

Host: Matt, staying with you, can you walk us through what an AI deployment failure actually looks like inside a health system, and what typically goes wrong, and when does it typically go wrong?

Matthew Montoya, DrEng: Yes. Again, I will start at first principles here because, unfortunately, we end up going down this path of failures and trying to resolve failures. And so, I think first thing to recognize is that this is a tough problem. And as you start looking at healthcare, there are a lot of swim lanes, there's lots of things that need to be done, lots of different incentives. As well as, again, first principles perspective, anybody's ever heard of the term entropy. Basically, entropy means things go to a state of disorder. And so, what we're working against is that we have these different swim lanes, we have these different areas we're trying to pursue, and then there's this natural tendency to go into disorder. And so, those that are listening recognize that this is a very tough problem. And so, it's not like people jump into solutions and then magically things happen. So, recognize that failures can occur. And so, what we're trying to do is create systems and infrastructures so that we allow ourselves to work this.

And so, the things that I like to think about, number one, is making sure you understand what your context is. Making sure, again, what is your problem? What are you trying to solve? Look at scenarios and use cases, because what we're trying to do is get on the same page. And so, if you can look at scenarios and use cases and see where everybody's role is, what they're trying to do, what they're trying to accomplish, what functions they need to perform, and then also who are those stakeholders that need to be involved.

So, that's kind of the general principles that I follow. And so, let's just say, for example, and again, I'll just do a quick one here. For example, let's just say, there's a sepsis detection that you may want to implement. You know, that's something that I think a lot of people are trying to pursue. There's measures and metrics that you can look at it. But if you don't look at it in terms of who are the stakeholders, how they're involved, what metrics they're looking at, what scenarios they actually are trying to figure out, then you're going to have a whole lot of failures because you end up having gaps. The executives may not know what the clinicians may want. And then, those that are actually working on other levels may not understand what the others want.

And so, when a tool is introduced, you go to to make sure that you are all on the same page. The stakeholders are understanding the context, they understand the scenarios. And so when you start evolving this, you evolve it together. And so, like I said in my earlier comment, it's very common for things to devolve because there's so many competing forces as well as the idea of entropy. And so, that's a specific example. But again, you could walk down many swim lanes and give those examples.

Host: What do executives most consistently underestimate, and what are the consequences when they do?

Matthew Montoya, DrEng: There's many swim lanes that one can go down. And so, one of the things I do want to comment on is David and I and Mike, we look at these things from different lenses. And so as we kind of dive down into my systems lens, I think of things, in terms of time, capacity, performance, value, risk, and culture.

And so, a lot of those have—I'll call them hard metrics that you can look at. But one of the areas that is the one that really impacts things is the idea of culture. And so, that's difficult to quantify. And so, when you start thinking about culture, there's alignment. And then, when you have alignment, there's alignment with different levels of the organization. And so if there's differences in terms of philosophies, differences in terms of the way it wants to approach at different levels and an understanding of where people want to go with this certain technology, there can create issues. And so, making sure that culturally people are aligned.

And then, another angle with that is the idea, and I'll call it the adoption spectrum. When we introduce new tools and new capabilities, there is always the spectrum within the culture where someone who's very into technology, they're very excited about technology, and they want to introduce it quickly. There's others on the other side of the spectrum who may not be as, you know, enthusiastic about introducing the new technology. And so, the culture and making sure that people are aligned in that way is important. When you think about it from that perspective, you can get a sense, again, of my foundational areas in terms of stakeholders, context, and scenarios.

If you can find that culture, find that alignment, and see where everybody can work together. Those are the kind of things we need to look at. So if one can understand the culture, again, part of, the system is the culture. You can get a sense of how things can move forward and how quickly and in which direction.

Host: Speaking of culture, one of the things that seems to have crept into the culture all around is speed. And when it comes healthcare, executives are under extreme pressure to move fast on AI. How would you counsel executives who feel they can't afford to slow down long enough to do systems analysis that you're describing?

Matthew Montoya, DrEng: That is an all too common incident. I've worked on this for years. Quite literally, I've worked on in systems in different areas for four decades at least. And so, it doesn't matter if it's healthcare or if it's aerospace, it doesn't matter what it is, there's that pressure to be able to introduce new technologies.

And I always reckon back to an old commercial back in the 1970s. There was a commercial that said, "You pay me now or you pay me later." And I don't remember what they were advertising, but that stuck with me. And so, as I've worked systems for, you know, these several decades here, I've seen that in all the areas here.

And so, part of, I think, what comes to mind when people want to do systems analysis, they may view it as some version of a bureaucratic gate or some type of friction that won't let us get there in time. But really, what it is, it allows you to further synthesize and understand what problem you're trying to generate, what kind of stakeholders you need, what kind of scenarios you need.

And by doing that, you're actually creating an environment so where you will be successful. You may have heard the term rushing to failure. And so, part of what you want to avoid is rushing to failure. And so, if you want to move systematically, it allows you to build that analytical infrastructure to understand your workflows, to understand the stakeholders, to understand who actually needs to be involved. If you don't, there will be, I guarantee it, rework. And so, I've worked in many industries. And many times, I've had people come back and say, "Yes, you were right. I should have taken the time early on to do the systems analysis because we are, whatever, three years down the road and we're nowhere. So, let's go back and do it right."

So, the pay me now, pay me later, it has to be done. You have to do the systems analysis. You have to understand the organization. You have to understand the workflows and the construction and the stakeholders. Because if you don't do that, you won't map it out well, and then the tool or capabilities you bring forward is just not going to work.

Host: You draw a distinction between deploying AI, and engineering AI into a system. What does that difference look like in practice?

Matthew Montoya, DrEng: Great question. I appreciate that. Because there's nuances here, and we often use language, and there's nuances in terms of these terms like deployment and engineering. And so, first of all, first principles again here is making sure, language-wise we're talking the same thing.

So, sometimes when people think in terms of systems, they think of maybe a device, and so maybe an echocardiogram and that kind of thing. But when I talk about systems, there is something called a system, a system where you're looking at multiple complex devices working together. You can think of something like an ER and there's multiple complex devices. And then, you're looking at an enterprise system. And then, within an enterprise systems, there are people, processes, and sometimes we refer to them as ecosystems.

And so, when I talk about deployment versus engineering, recognize, first of all, I'm talking about the larger ecosystem and an enterprise system and putting something in there, which includes these basics that I was talking about the scenarios, the stakeholders, the technology, and making sure we understand the context. So, thinking about deployment, the way that I think about deployment is the ideas that you're putting a capability, you're putting a technology into a swim lane and moving on. In a sense, you're just deploying in a certain area. Whereas if we really want to engineer it, we need to pull everybody in that is the stakeholder that needs to think through how they actually design, how they actually do their workflows, what kind of areas they need to worry about, what kind of incentives, what kind of activities they have.

So when we actually work on the enterprise, the ecosystem, we've included all those factors, as opposed to just taking a tool, put it in one thread, and then seeing how it goes. And so when that happens, oftentimes it is just either people will develop it, put it in that swim lane. And then, in a sense, people will be asked to use it. And over time, people will develop workarounds. They may not want to work with the tool anymore. And so, part of what we're trying to do is engineer, which is engineer the ecosystem, not just a device, engineer the ecosystem so that the ecosystem and the workflows work together so that, when you have a deployment, it's more comprehensive and holistic and all the stakeholders can embrace it through the process, through this systems analysis process.

Host: Yeah, makes good sense. And David, we want to bring you back into the conversation. Many leaders think of AI as having a fixed level of performance once it's deployed. Why isn't that true? And what makes healthcare AI different from previous technologies, as you mentioned earlier?

David P. Rastall, DO, PhD: This is a fantastic point. So when we think about— I'm going to use the example of diagnostic AI as an example, but it will go to everything. So, for instance, in a traditional diagnosis like a lab test or something that you're doing, you can measure how well it performs over a certain population, and that is how well it will perform for you.

But what we found working with some of the first approved AIs is that's not the case. So in 2018, the very first, AI received FDA approval that was able to diagnose diabetic retinopathy without a doctor in the loop. It was a completely autonomous AI system, and it performed phenomenally well, and it was tested here and it was tested other places.

And the remarkable performance of that kind of made us assume that this was going to become like the way this was done across the United States because it worked so well. But we're now, you know, five years out, and even though the market share for other imaging technologies has grown substantially, it's actually decreased for this technology. And what we found out was that was because unlike traditional technologies like a blood assay or any other test where it's just a chemical reaction, the performance of an AI depends on the data coming into it.

And so, what had happened, and the reason that ophthalmology even was the first, is in different health systems, there's different cultures, like Matt was talking about. And ophthalmologists are very precise people because they have to fit glasses. So, they're people that, you know, if you're working-- I work with ophthalmologists, I work with ED doctors. If you're with an ED doctor and the picture is good enough to make the diagnosis, you're done. If you're working with an ophthalmologist and they take that retinal photo and it's a little smudgy, even if you could see everything perfectly well as a doctor, retake that photo. That photo is low quality. So, that culture, that culture of perfection that had been developed in ophthalmology was inherited by both where it was tested and the data that it was trained on, and that's what translated to such a high early performance.

As soon as you took that from either an ophthalmology clinic or a primary care clinic where there was a study on how good this AI is working going on, and put it in a regular primary care clinic without a dedicated revenue stream to pay for a big staff to make sure this is being done just correctly, but loading this onto busy nurse who's doing everything else already, what you found was human behavior changed. The nurse's priority was to get the patient through. The quality of the images dropped off. And therefore, the diagnostic quality of the AI dropped off.

So, the core concept is very similar to what Matt's saying, is that AI not just a single technology that performs a single way. AI is shaped by the data that goes in, AI reshapes human behavior, and human behavior determines what data goes in. So, executives should think of AI less like an isolated technology and more like AI plus the environment you're putting it in as a unit.

Host: Kind of to that point, David, iif AI performance and value evolve over time, what does it mean to govern AI like a clinical process rather than manage it like it were a software purchase?

David P. Rastall, DO, PhD: Yeah. This is one of the biggest takeaways that should be easy for leaders to understand, but is critical. You can't look at AI like it's software, and you're just going to do your go-live checklist, you're going to do your implementation push, and then you're going to forget about it. The AI is going to change clinical behavior after it's deployed, and that clinical behavior is going to have an influence on your organization and is also going to change the AI's performance.

And so, you need to treat it more like if you were expanding and hiring a brand new specialist, If you're opening an oncology center and you were hiring a bunch of oncologists. Obviously, you know the oncologists are going to be influenced by your healthcare organization. And your healthcare organization is going to influence those oncologists. And we should think of AI more like that. And so, if nothing else, you need to take measurements over time, like Matt was talking about, and think about what are the key things that our vision of our organization cares about. And now that we've deployed this AI, are those improving or are there problems?

Host: You're making the point that clinician behavior isn't just affected by AI, it actually becomes part of the AI system. What do you mean by that, and why should executives care about that?

David P. Rastall, DO, PhD: Yeah, this is critical. People like to toss out a lot of somewhat confusing buzzwords. You hear about data shift and data drift, and you hear about automation bias and various other things. But what it really means, again, is that more than any prior technology, AI can reshape human behavior. And that can be good or that can be bad. But if you don't consider it, as Matt pointed out, it will be bad.

And so, you can design around this, you can consider around this. And this is one of the really interesting things we've found, because we study a lot with the Armstrong Institute, we study a lot of how this impacts people. So, one of the measurements that you might think to take is you might say, okay, well, let's say it's helping doctors. "I'm going to measure doctor hours. I'm going to measure the amount of hours a doctor spends a day. And then, we'll turn on the technology and see if that number goes down." One of the really interesting things we found is, again, that does change over time because people change their behavior gradually, so you can't just measure it, you know, the first month and assume that's going to be at the end of the first year.

But even more interestingly, sometimes the amount of hours a doctor was doing something would go up, but they would feel like the system made things easier. And sometimes, the system would actually decrease the number of hours they were doing something, but it would increase the stress and they felt like they were doing it more, it would increase burnout.

So, this is the complexity of these types of systems. But the good news is, if you pick the right one and it's designed correctly, it can do all of the above. It can decrease the hours the employees are working, and it can decrease their burnout and make them happier, and make them more likely to do behaviors that are aligned with what your organization needs.

Host: David, let's shift a little bit to regula-regulatory issues and regulatory pressure around healthcare AI is clearly intensifying. How should health system leaders be positioning themselves now before the requirements fully land?

David P. Rastall, DO, PhD: That's a really good question, and this is really critical. I think this is something that you can't ignore. Because right now, essentially, there's not many regulations right now. We're kind of at the frontier of this. But if you just ignore it and you don't take any action whatsoever and you say, "Well, there's no regulation right now for how to do this, so we're just going to do it," you will really end up in the exact situation Matt was just warning about. You know, you'll pay for this now or you'll pay for this later.

What I would really encourage is for health executives to get some education on this, and then, use that and do current best practices even if they're not necessarily required because that future-proofs everything. And then, when regulations hit, they're the leaders in this area and also they've done, you know, the best practices. So I think, before all these regulations hit and formalize exactly what's being required, the healthcare systems that take the initiative to do this correctly, both getting education and then implementing it, will be the ones that are caught flat-footed when the regulations hit or, even worse, that have a patient safety event which ends up really damaging their brand permanently.

Host: Healthcare AI investments often arrive at the board with ROI projections built on time savings and vendor benchmarks. Your colleague Michael McShay, who was referenced a little earlier, has argued that time saved is not the same as money captured, and that AI is fundamentally different from prior technologies because it adjusts to the clinical environment it enters, meaning ROI from one institution may not be transferable to another, as was mentioned earlier. What should C-suite leaders and boards be demanding differently when they evaluate an AI business case?

David P. Rastall, DO, PhD: That's a really good point. So just like how we were talking about the clinical behavior, the safety and so forth, and the diagnostic accuracy of an AI isn't fixed, it's a product of the AI and its environment. The same is true for ROI. So, unlike previously when you're analyzing ROI, and you can think about that as a sort of fixed amount, this is essentially a dynamic ROI that will be very different, depending on your healthcare system.

Like, I consulted with a country who has an island health system, and their difficulties and needs are very different, than when I consult with other major health systems about how to deploy AI. The biggest thing is that previously we've already learned from things like the EHR, and Matt McShay pointed this out.

In the era of the EHR, there was an idea of, well, look, this study shows that doctors work three less hours a day, so you can multiply that by the amount you're paying your physicians, and then that's your savings. But when the EHR was deployed, the fact that it saved that time did not immediately go over into ROI because that time was distributed elsewhere.

So, that concept remains, but the unique concept to AI, again, is that the ROI will be genuinely a product of both the healthcare system it's deployed in and the AI, and it will not generalize for every AI system and every AI model and every AI system. It will not generalize. It will be very specific to your needs, what your organization needs. And again, the example being that an island health system where there's one doctor, there's one neurologist on one island, on another island there's no neurologist, et cetera, had very different needs, than most of the health systems I was addressing. And finding the right AI that addressed those was critical and fantastic for them.

Host: Well, David, II'll address this one to you as well, but Matt, hang on because I'm going to come back to you in a second. David, beyond financial returns, durable AI value has to reach patients, clinicians, and the institute's mission as well. How do you think about what the broader value actually looks like, and how do you know when you've achieved it?

David P. Rastall, DO, PhD: This is actually the most essential thing of all of this, is we need to look at this from everyone's perspective and how the AI is going to impact them and what the value it's bringing to the organization is. And just like we talk about ROI, which is one form of value, we can talk about, employee burnout, which is another form of value. We can talk about the quality of the medical care we're giving, which is another form of value. And what we really want to do is we want to assemble a diverse team of stakeholders, everyone who might be impacted by that technology, and ensure that this particular AI system is likely to improve all of that and stay aligned with the corporation's vision.

Host: Well, Matt, I promise we're going to come back to you and we're going to give you the wrap-up slot here. And if a healthcare executive is listening to this and takes one thing away to their next leadership meeting about AI, what would you want that to be?

Matthew Montoya, DrEng: I will go back to my earlier comment that use first principles. What happens in AI and any technology insertion is that there's lots of buzzwords, there's lots of different, you know, use cases. There's very different ways that people look at it. There's people with different incentives, different vendors. All these things are going to be thrown at you. And so, what you need to do is go back to first principles, which as I said earlier, what am I trying to solve?

What's the most basic problem I'm trying to solve? I referenced, for example, a little bit earlier, the idea of sepsis. You know, maybe I want early detection of sepsis in certain areas. And so, what do you do? First of all, let's baseline this. And so, baselining, it means what do I do right now? And so, if you don't baseline these things, first principles, if you don't baseline them, then you don't know what you actually need to improve, where you need to improve it. And if something is put in, how much do you actually improve it? And so, again, first principles, what problem I'm trying to solve, that gets into the stakeholders.

And so, I'd mentioned, make sure if you're going to go through a process like this and you have a specific problems, make sure you have the right stakeholders in there, and then start generating those scenarios. You know, in our case here, the sepsis example. If you do those very basic things. And I mean, this is part of healthcare. These are the things that you do. It's not like you have to be a technology whiz. You have to just know, you know, what am I trying to do? You know, what's the baseline? Who needs to be involved? What are the scenarios? And then, you can figure out what you want to start putting in there. There are nuances when you start introducing technology. For example, we haven't touched on this yet. You know, AI, maybe you need a machine learning tool. You know, AI's not exactly ML, and so maybe something that's a little bit more ML. But if you don't do these basic fundamental first principle things, you can't differentiate that. So, that's what you're trying to do, is what problem am I trying to solve?

Because once you start getting in my comment earlier about the culture, when you start potentially to implement these things, those that are actually kind of on the ground floor trying to work these things, they'll say, "Why am I doing this? What problem am I trying to solve?" So if you don't address that upfront first, then it's just going to be friction throughout.

Host: David, along with listening to this podcast, how can healthcare executives learn more about implementing AI?

David P. Rastall, DO, PhD: Yeah, I think this is the moment in history where developing these skills will really help differentiate you and your organization from those that don't take the time. There's a lot of important ways you can learn about this. We have various papers put out by Armstrong Institute and by a lot of other people here at Hopkins, and we're also putting on wonderful programs like the one ACHE is doing, the AI-Enabled Health Systems Executive Playbook, which you can sign up for. We have the next session's coming out October 12th and 13th.

Host: Dr. Matthew Montoya, Dr. David Ratstall, thank you both for joining us and giving some great insight into AI application throughout healthcare and some information, some warning, all the things, going to be very helpful. We appreciate your time today.

Matthew Montoya, DrEng: You're welcome. Appreciate it.

David P. Rastall, DO, PhD: It was a lot of fun and it's an honor to be here.

Host: For more information, go to healthcareexecutive.org. If you enjoyed this podcast, please share it on your social channels and check out the entire Healthcare Executive podcast library for topics of interest to you. I'm your host, Carl Maronich. And this is the Healthcare Executive podcast. Thanks for listening.