I gave a talk recently for Federal agencies, integrators and compliance teams called “Artificial Intelligence (AI) in Regulated Industries.” I told the room upfront it wasn’t going to be a highlight reel, and I meant it. I’ve spent enough of my career sitting at the intersection of product and compliance to know that another vendor promising that everything’s ready isn’t what anyone in that room needed. I wanted to give them an honest map: what’s actually usable today, what’s close and what’s still further out than the demos suggest.
Regulated organizations aren’t short on motivation to adopt AI. Almost everyone I talk to already wants to move. What’s missing is an honest answer about what a specific environment allows today, versus what’s six months or a year away. That gap between appetite and clarity is where most AI initiatives quietly stall.
Deployment Model is a Proxy, Not the Barrier
On-prem versus cloud isn’t the decision that determines AI readiness. It’s a proxy for a deeper set of questions: where data can legally live and move, whether a model’s provenance can be verified, whether a defensible audit trail exists and who’s accountable when an AI-assisted decision turns out wrong.
FedRAMP GovCloud customers are often starting from a real advantage without realizing it. Their platform’s Authority to Operate (ATO) has already made the hardest case there is, the infrastructure risk case. What’s left is a narrower, faster ask, extending that authorization to a specific new capability, not building a case from zero.
On-prem customers get told AI isn’t an option for them, and that’s not true. It is an option, but it has to be genuinely self-contained, a model running entirely inside a boundary, with no calls out to an external Application Programming Interface (API). I keep hearing the phrase “bring your own model,” and that’s exactly what it describes: an open-weight model on your own hardware, connected to your systems through Model Context Protocol (MCP), the open standard that’s become the common way to link a model to real tools and data without leaving your boundary. It’s a real, viable path, and more capable than it used to be. It’s also a serious lift, not a shortcut. A governance gateway that gives you Systems and Organization Controls (SOC) 2 attestation and clean audit logs is genuinely useful. It isn’t authorization. It tells an assessor the gateway vendor runs a sound operation. It doesn’t tell an Authorizing Official (AO) what they need to issue an ATO for a specific use case.
I’ve stopped asking, “Are you on-prem or cloud?” I ask people to walk me through the last capability they tried to add and tell me exactly where it got stuck. Nine times out of ten, the answer isn’t the deployment model. It’s something specific and fixable; either nobody mapped whether a tool’s output was allowed to touch a particular system, or there was no way to log which records a model actually read. That’s a data flow mapping problem, not a deployment problem, and it’s exactly what the authorization process below is built to catch.
Three Honest Tiers, Not a Feature List
A real framework beats a feature list, because a feature list is exactly what lets vendors talk past the people they’re selling to.
Tier one is assistive AI. Classification, anomaly flagging and summarization. Mature, low risk and genuinely deployable today, even in constrained environments. A vendor only offering this isn’t underselling the relationship. It’s often exactly the right starting point.
Tier two is agentic AI with a human in the loop, and it’s where I tell most regulated organizations to aim right now. An agent drafts, recommends or pre-fills something, and a person authorizes the action before it goes anywhere. I like using environment, health, safety and quality (EHSQ) examples here because it’s the world I live in: an agent that engages with someone reporting an incident, fills in the digital report and lets the reporter review before it saves and kicks off an investigation. An agent that clusters similar past incidents and drafts a root-cause hypothesis, validated by a person before it becomes part of the record. An agent that pulls together audit evidence, flags gaps and drafts a corrective action list, reviewed before it’s submitted anywhere. None of that requires exotic infrastructure. It’s the same pattern applied to a workflow that’s probably already running.
Tier three is fully autonomous agentic AI, acting without a human checkpoint, and it’s largely not viable for regulated environments yet. That’s not caution for its own sake. The accountability structures most regulated fields depend on, such as a doctor signing a diagnosis or an auditor signing an opinion, don’t have an equivalent for an autonomous system yet. When a vendor claims otherwise, that’s a signal to ask more questions, not fewer. Where is this actually running in production, not in a demo? Can I see the audit trail from the last time it acted without a human in the loop? No specific answer means the answer is already clear.
These tiers aren’t a maturity ladder to climb as fast as possible. Plenty of organizations do excellent, valuable work permanently at tier one or tier two. That’s not a consolation prize. It’s often the right long-term home for a given use case.
For any capability pitched to me, I keep three questions in mind:
- What data does it touch?
- Where does inference happen?
- Can a human intervene before there’s a consequence?
I keep them written down somewhere I’ll see again. They tell me more about where a capability really sits than any demo does.
What Authorization Actually Requires
Authorization isn’t a single event. It resets every time an agent’s decision surface expands, since an agent that reads and summarizes carries a different risk profile than one that writes back into a system of record (SOR), even on the same underlying model. The real path has four stages:
- Risk categorization
- Data flow mapping
- Human override design
- Logging and review
The last stage is usually the longest and least predictable, because most of that time isn’t spent on any single stage. It’s spent on the back and forth between them, when a question about data flow sends a team back to revisit risk categorization, which changes what the override design needs to look like.
That’s the honest reason authorization timelines run twelve to eighteen months while AI capability cycles move in weeks. For anyone who can’t wait that long, here’s what I actually do: scope the capability narrowly enough to sit inside an authorization boundary that already exists, rather than requiring a new one. Pilot it in a sandboxed, non-production environment to build an evidence base before formally submitting anything. Treat human-in-the-loop as a deliberate design choice, not a stopgap to be embarrassed about.
The Gap is Real, and It’s Closable
AI investment is real and accelerating. Recent research puts AI at roughly 12% of functional budgets, with most leaders planning to increase spend this year. Governance maturity hasn’t kept pace; only about a fifth of organizations report governance mature enough for the scale they’re already operating at. That’s not a reason to slow down. It’s the actual opportunity. The organizations that govern well early are the ones who capture the value everyone else is still chasing.
There are two failure modes I see most often in the leadership conversation: overpromising a timeline that isn’t realistic and reflexively dismissing what’s genuinely available because “we’re too regulated for this.” The fix is three honest buckets:
- What’s possible today without waiting on anything
- What’s possible with workarounds like narrower scope or a sandboxed pilot
- What’s genuinely still pending, a named dependency, not a vague one
Three honest buckets beat one optimistic roadmap slide every time; leadership can’t plan around a number that turns out to be wrong.
Where I’d Start
Pick one thing this week, not all of it at once. I’d ask my next AI vendor the three questions above before getting anywhere near a deployment model debate. I’d check what my ATO actually covers before my next pilot, and I wouldn’t assume platform coverage extends automatically, because it usually doesn’t. I’d bring procurement, compliance and IT security into my next pilot kickoff, not the retrospective. And when picking my next pilot, I’d weigh my team’s readiness to change how it works, not just the projected value. The highest-value use case isn’t the best investment if the team behind it isn’t ready to run with it.
To me, none of this says wait. It says govern as you go, and the return compounds.
Ideagen and the Public Sector
That’s the approach we take at Ideagen, working alongside our Public Sector and critical infrastructure customers to build AI into EHSQ workflows with the right guardrails from day one. If any of this sounds like the conversation your team is having, we’d be glad to continue it with you.
Watch Ideagen’s webinar on “AI in Regulated Industries” to explore the challenges and opportunities awaiting Public Sector organizations looking to incorporate AI into their workflows.
Carahsoft Technology Corp. is The Trusted Government IT Solutions Provider, supporting Public Sector organizations across Federal, State and Local Government agencies and Education and Healthcare markets. As the Master Government Aggregator for our vendor partners, including Ideagen, we deliver solutions for Geospatial, Cybersecurity, MultiCloud, DevSecOps, Artificial Intelligence, Customer Experience and Engagement, Open Source and more. Working with resellers, systems integrators and consultants, our sales and marketing teams provide industry leading IT products, services and training through hundreds of contract vehicles. Explore the Carahsoft Blog to learn more about the latest trends in Government technology markets and solutions, as well as Carahsoft’s ecosystem of partner thought-leaders.
