John Riggi (Host): Healthcare organizations are rapidly adopting AI-enabled technologies to improve clinical care, operational efficiency, and administrative workflows. However, many AI capabilities are embedded within existing applications or introduced through third-party vendors. That creates risks that traditional security risk assessments and third-party risk management programs were not designed to evaluate.

Welcome to the AHA Bringing Value Series from the American Hospital Association. In this series, we speak with AHA business partners and learn how they support AHA hospital and health system members. I'm John Riggi, your national advisor for cybersecurity and risk at the American Hospital Association. And today, pleased and proud to be here with my good friend and colleague, Carter Groome, CEO at First Health Advisory.

Join us to learn how to expand your existing security risk assessment and third-party risk management processes to account for AI-enabled technologies while continuing to support innovation responsibly. Carter, welcome. It's great to have you here today.

Carter Groome: John, thanks so much for having me. Great to be here.

Host: Carter, let's start with this. Healthcare organizations already conduct security risk assessments and third-party risk management extensively. Why are those approaches no longer enough as AI becomes embedded in healthcare technologies?

Carter Groome: John, the short of it is those approaches have never been enough. AI is another risk domain that has quickly become part of the ecosystem of care. And the fundamentals, the first principles of assessing risk, identification and addressing risk on a continual basis remain the pillars of good hygiene.

And I love that term hygiene because it implies that it's a part of a routine. It's a part of the culture and the ongoing activities to whatever you may want to apply it to, your personal health, the health of your community, or the management of risk in your healthcare enterprise. And I say it's never been enough, John, because as a collective, as a sector, we've never really applied the rigor needed on a continuous basis to truly understand exposure and how best to prioritize or to mitigate risk and cut off the adversarial path that our most critical functions, data, and assets are exposed to.

And I have to add, the tectonic risk, the big dot issues that we face every day in this industry are regulatory movement, economic conditions, and even unforeseen events like pandemics or wars. But my sleep deprivation centers on a relentless pressure to adopt AI, to be bolder, to go faster, while safety and security and privacy are an afterthought. And we're just giving so much control, knowingly or unknowingly, to agents that need access to your data—your data. And all of that's compounded when AI fluency at the board level or executive level is lagging, because they really need to be literate on these risks to apply any effectiveness around governance.

And it's mind-boggling to me how effective the adversary is, aided by AI when they're trying to breach more quickly, more widely. And social engineering is just becoming even that much more sophisticated, where even the most aware human beings are being deceived. And so, I'd say just stick to the fundamentals, least privilege, segmentation, double down on assessing and understanding how to mitigate third-party risk with AI viewed as just another domain. It's still the most effective approach to informing your governance and resilience strategy

Host: Totally agreed with you, Carter. And I just want to add emphasis to a couple of things that you said. Hygiene. We always talk about cyber hygiene. And one of the things that I've often thought about is that cyber hygiene is really as important as medical hygiene to protect the patient. We want to help reduce the spread of infection, one digital type of infection, and one, of course, biological.

And then, of course, the other thing you talked about, we are rushing to use AI within our systems to adopt AI. We see it as that shining light on the hill that will help us overcome workforce shortages, enormous financial pressure, regulatory pressure, which will only get worse after the midterms. So, we really have this environment filled with hope and anxiety and tension. So, so happy to have you here to discuss these. So, let's get back to it here for a bit.

So many organizations don't realize how much AI already exists within their current technology stack. What new risks should healthcare leaders be evaluating? What's your thoughts on that?

Carter Groome: First of all, set aside the black box nature of the frontier model advances that are being deployed more quickly than any technology that we've ever seen in our past. And oh, by the way, it's also becoming a black box to the developers, the frontier labs themselves with recent events that we've seen around Hugging Face.

And so, I'll try to remain a little bit more grounded to say that, yes, organizations must get visibility into where AI and your data is being used in the same way that you would want to get an asset inventory of medical devices or servers or IT assets or even your operational technologies in the environment of care. This is literally front and center in every meaningful risk framework, CIS, NIST, ISO, you name it. Having an understanding of what your inventory is, and that's a really heavy lift when you think of AI. It's a lot of work

And so, what are you going to say to your board, though, when you're reporting out on a major breach or a lockup or an operational discussion and say, "Well, gosh, that was really difficult" or, "That was really hard and that was expensive," right? That's not going to be a great answer when you're creating patient safety risk, there's data exfiltration, there's capital risk. And so, when you have that AI risk in your environment, it's obviously not just with your enterprise, John. It's with your business associates and their enterprise. And you just can't manage third-party or supply chain risk. You can't. That is someone else's risk. But you can mitigate it. You can better understand it, you can prioritize it and have a plan that's pressure tested for that bad day. And again, that's a ton of work, and everything seems to be getting more complex, more systemic, and AI just throws a ton more noise your way just to navigate through.

So, I want to get smart on really what can be automated responsibly and safely in the environment of care. And when you think of yesterday's AI, right, machine learning, they were talkers. They were bots with narrow intelligence and basically had read access. But the AI of today and tomorrow, these are doers. These are agents that have read, write, access, and act for you to become more efficient and effective. And those agents, they need to connect to your data, John. They need your workflows, your ERP system, your EHR system, and other foundational models that they connect to. And if you're unable to control those connections that the agents and the third parties are making, this is what you'll get really concerned about when we say the attack surface is expanding.

And the last thing I'll say, when you're evaluating liability in this new environment and understanding who's accountable if there's a leak, if your PHI is now in the public domain for whatever reason, gosh, your board, they really need to understand how that risk transfer occurs, whether it's to the model or to the provider or to the insurer. And it's so fuzzy right now that you've got to pay attention to this. You've got to lock down your business associate agreements now since we just don't have much precedence on the legal elements of liability here.

Host: Carter, great points. And when you speak about that fundamental practice of inventory, extraordinarily difficult when it comes to AI. So many systems and programs, applications have AI embedded in ways we're not going to have full visibility on, and it's changing constantly. Even if you were able to complete a full and accurate inventory today, it will probably be inaccurate tomorrow as new agents are acquired, others are dropped off.

And the other point you made as well about the need for connectivity for these AI agents, but it is that connectivity which creates risk as well if they are insecure and as we've seen in the headlines of agents acting in ways that were not programmed unexpectedly gaining access to the internet and doing bad things. So, this is really a dilemma for healthcare going forward.

So, how can healthcare organizations balance innovation and AI adoption while protecting patient data, maintaining regulatory compliance, and reducing that operational risk?

Carter Groome: Yeah, John. I mean, that's what we're all talking about out here. And I am all for emerging technologies. Our clients are, the healthcare industry is. And healthcare is arguably the sector that has the most gain from using AI safely. Yet we all know that speed has always been the natural enemy of good security design, good controls, and good governance.

And as security and privacy and risk management consultants, we're naturally allergic to that speed. And AI depends on the systems and the data that it's connected to, and really that tension between security teams who seek to limit those connections and reduce risks, and those promoting AI who want to integrate it through all means possible throughout the organization to gather information and create efficiency and those things you talked about earlier that tension's real. It's real. And I'm not going to judge the revenue and productivity problems that we're trying to solve in financial and operational and clinical areas. I'm just saying, before we get really dragged into the hype here, your organization, no matter what you do, has to ask the tough questions about privacy, about data, about security.

And I just don't see enough of those questions, John. I mean, many of those questions revolve around understanding your most critical business functions and process. And when you understand them, you can then begin to harness what we call the agency to act for these agents or their models. And if healthcare continues to add agentic capability without matching that inventory or visibility, without matching your ability to observe those models and then provide governance and enforce that governance, boy, it can unintentionally make attacks even faster or more broad and even more damaging.

And the last thing, I'll note here is that an effective security governance and compliance model, it needs to see that agent through the entire life cycle of the agent. That governable object is really the whole composition, the model, the agent, the prompting, its memory, the tool access, data sources, the workflow that's involved, and even the authority boundaries that the agent has. These are the things that you need to understand before you're able to even begin reducing risk around AI.

Host: Carter, could not agree more. I like your phrase when you talk about understanding agentic capability, there must be matching visibility. Capability must be followed by visibility to the extent that we can. We see the largest AI companies out there, Anthropic, OpenAI, talking about their agents acting in unanticipated ways. So, we're already at a challenge where even the developers may not be sure. So, we must ensure that our controls are in place.

And the other point you made about speed. Yes, speed to innovate, speed to be bold. Again, for all well-intentioned reasons, we want to improve patient outcomes. We want to save lives. That's job one for hospitals and health systems. Sometimes our technology partners, though, have the opposite, I've heard this in Silicon Valley, "Move fast and break things." We can't do that in healthcare. Those things are people. They're humans. That's why we need to have really extraordinary visibility, controls, and understand the risks that we are, again, the promise of AI, but we must understand the risks.

So with all that, what should executive and boards be asking vendors about AI capabilities before even thinking about it, certainly before purchasing, and then during the implementation phase?

Carter Groome: Yeah. Just double down on we don't want to break things in healthcare that impact patients and lives. in this current environment, John, it's really disorienting for boards and executives. And I like that word, because most of us don't really know how it works. Heck, even the developers that we're starting to learn don't know how it works, right?

And that uncertainty reminds us and boards of how little control we have. And what we do know, though, that's real is the landscape, the threatscape, if you will, is shifting. AI models are upending cyber as we know it. I mean, the last couple months have been like nothing we've ever seen before, and that's not hyperbole. I've been involved in this field long enough, and I can tell you, yeah, it's nothing we've seen before. And we have to navigate all this noise that's just overwhelming to so many of us, and there's a confusion on how best to reduce the blast radius. You always hear this, the blast radius, how we reduce it.

But that confusion itself on how we reduce it becomes a risk in and of itself. And I can say this, that organizations and boards that dismiss this shift as hype are going to miss the critical threat changes that we need to react to. And we just can't afford to wait for disaster to tell us which signals matter, I mean, the cardinal rule of governance is to get disasters right. I think about mayors that have not prepared for snowstorms or governors that have not prepared for hurricanes. Literally, the cardinal rule of governance is to get disasters right.

And so, I think of several assumptions behind current risk metrics that may no longer hold and need to be re-examined. And we've moved into a world of containment and a focus on resilience. And the metrics, John, should now focus on things like speed to recover to normal operations or even mean time to decision. We've got to be faster.

And then, secondarily, the same AI capabilities that this risk creates, it's actually an opportunity for us, right? Organizations can now identify their own weaknesses using AI models before attackers do. You can review vulnerabilities at machine speed and respond to these incidents faster than any human ever could. And the organizations that invest in that type of technology are going to have a really strong defensive posture, and they'll be more resilient if there's an incident.

And those tools that Google, I just saw they're getting into the game too. A lot of them are open source out there. They let us find and fix our own weaknesses before the adversaries do. So the message is, without any attention to buying down risk, meaning investing in defense and resilience. Yeah, you're going to move a lot faster, but the business is going to accumulate risk just as fast.

Host: Thanks, Carter. Again, totally agreed with your points. And also, when we think about AI in the cyber world, so we're trying to anticipate things where we don't have data on, the developers themselves are in some sense struggling to really truly understand the nature of risk with AI.

And then, of course, we're seeing, as you mentioned, the very responsible, proactive use of AI to identify vulnerabilities and help repair them before software is released. So, a lot of kudos to Epic, for example, for announcing, "We're using Mythos." They are using Mythos to identify vulnerabilities and delay release of other models against their own economic interest.

So, that's a very good example of responsible use of AI. AI programs itself may possess vulnerabilities. That's an area I don't think we focused enough on, just like any software program. And then, of course, AI is doing, the agentic AI will take over functions that humans would normally do, and we're going to lose those skills. There'll be a certain de-skilling of the workforce as well. So then, the issue becomes, if we lose those AI capabilities, just like in a ransomware attack, we lose the PACS system, who is available? What is the downtime procedure, in a sense, for that function? Does anybody understand or has the capability to do manually what that AI function is doing in our hospitals?

So as we shift responsibility to AI, to agentic AI to perform certain functions, what is the backup plan if we lose an internet connection or a network connection and that AI agent can't perform that function anymore? Just like downtime procedures for PACS and labs and pharmacy, all of that.

So finally, with all that, which is a lot to really comprehend and digest for all our folks here. Finally, what do you think the practical first steps can our audience take to better understand and manage AI-related risks across their entire environment?

Carter Groome: For me, the fundamentals have not really changed in understanding and managing risk with AI. I mean, certainly there's some things that come to mind like prompts that introduce new methods, tactics and techniques. But first principle cybersecurity still hold true, and proven approaches such as default deny and assume breach are going to become even more accepted.

I am hearing a lot more energy and chatter. I'm having discussions around deception technology, not your sort of old honeypot stuff from 15 years ago, but canaries that are going to alert and throw off autonomous activity within your environment. So, that's something that I'm watching pretty closely.

There's also been a lot of debate on the value of awareness and education, and how social engineering is ever more effective when augmented by AI. And I just don't think you can throw in the towel on cyber awareness when you need to be more creative ultimately in how you're getting the message across, or how people and organizations are getting beat by the adversary.

Like I said, it's a constant debate, but I still believe that education and awareness has its place in this new threat landscape. And in my view, I'd say maybe to wrap up, the currency is judgment in this new landscape, right? Not the outputs, but the person that reads a vendor white paper and says, "Well, this won't work here because of X, Y, and Z," is now more valuable than the person that produces a 40-slide response to it. And the AI landscape shift has really raised the value of judgment, of safety, of verification, and really accountable decision-making. Those are the things I think about today in this landscape.

Host: And those are excellent thoughts, Carter, especially human judgment, human verification, human accountable decision-making, right? We cannot eliminate the human from the loop. To your other point, as we are focused on these ever-accelerating, sophisticated agentic cyber attacks and social engineering, what do we hear? Multinational, multi-billion-dollar corporations, somebody from ShinyHunters calls and deceives, just verbally, an employee to give up their password, username, password, and multi-factor authentication code, and they're in with a simple phone call. So, we cannot lose sight of that. And of course, as you said, if that was accompanied by a expertly and agentic type aspect to it, an email, phone call, it makes it much more effective.

So while we're focused on the very advanced threat, we can't lose sight of that basic calling up an employee and saying, "Hey, this is IT. We need your password." So again, Carter, always great to have a discussion with you. Your insights are extremely valuable. And just again, want to thank you for joining the podcast and being a great partner with the American Hospital Association.

Carter Groome: You got it, John. Thank you for your partnership.

Host: And for our audience, if you'd like to learn more about First Health Advisory, please visit firsthealthadvisory.com. This has been the AHA Bringing Value series podcast, brought to you by the American Hospital Association. And thanks again for joining. Appreciate everything everyone does to defend our networks, care for our patients, and serve our communities. Stay safe, everyone.