Insights from Federal data, legal and technology leaders on turning AI potential into mission-driven action
Everyone’s talking about AI. But in Government, where budgets are tight, oversight is strict and the stakes are high, talk isn’t enough. Agencies need AI use cases that solve real problems, not just generate headlines.
At a recent panel discussion in D.C. hosted by ZL Tech and Carahsoft, experts from data, legal and tech roles shared their insights on how Federal agencies can move from experimentation to impact. Their message was clear: success with AI starts with governance, strategy and the right people at the table.
1. Want Real AI? Start at the Top
The biggest challenge agencies face? Starting small and remaining siloed.
“Start at the highest, most strategic level of the organization,” said Matthew Versaggi, a White House Presidential Innovation Fellow for AI. “Don’t begin in your own department, by then it’s too narrow. Instead, ask: what’s the most impactful agency-wide use case we can build toward?”
The panelists emphasized that departmental pain points might improve workflows, but agency-wide pain points tied to the mission are where AI can truly move the needle.
“Without a structured process, you’re just chasing your tail,” added Kon Leong, CEO of ZL Tech. “Start small, but make sure your experiment is scalable and aligned to long-term strategy.”
2. Governance Isn’t a Roadblock. It’s the Roadmap.
AI can’t succeed without trust in the data. And trust depends on governance.
“Governance is accountability,” said Leong. “It’s what separates scalable, sustainable innovation from science experiments.”
Jason Baron, a professor and former senior Government attorney, described governance as a mesh, not a silo: “True governance links your CISO, CIO, records officers, FOIA leads, legal teams—all under shared policy and ownership. We used to work in silos. That has to end.”
And as Matthew pointed out, AI governance isn’t a blocker, it’s an enabler: “AI governance becomes the mechanism for sustaining innovation. If we’re going to compete globally, we have to embrace it.”
3. Talk to Your CDO—Yes, You Have One
One of the most actionable takeaways: if you’re not already talking to your Chief Data Officer, you’re behind.
“Every agency has a CDO,” said Jason. “Go find them. Hopefully you like them. Have a conversation.”
CDOs are uniquely positioned to bridge mission needs with data access and policy. As one attendee noted during the session, “Awareness is the first step. Records and governance leaders are finally getting a seat at the table.”
It’s no longer enough for legal, records and privacy teams to operate in isolation. Building AI responsibly requires alignment—and that starts with the CDO.
4. Unstructured Data Is the Game-Changer
Structured data, like spreadsheets and databases, has been the traditional foundation for reporting and analytics. But that’s not where the majority of Government data lives.
“Unstructured data is radioactive,” said Leong. “That’s where every crisis lives. And now, it’s center stage in AI.”
Unstructured data includes everything from emails and PDFs to file shares, chat logs and documents. It makes up more than 80% of enterprise data, yet many agencies lack visibility or control over it.
Jason gave a real-world Federal perspective: “As a records guy, I’d take out my watch and wait to see how long it took vendors to say ‘FOIA’ or ‘FedRAMP.’ If they don’t understand the challenges around Federal unstructured data, they’re not serious.”
5. Use the Impact vs. Effort Matrix to Prioritize Wisely
With hundreds of possible AI use cases, how can agencies filter out distractions and find the ones worth pursuing?
Panelists recommended the Impact vs. Effort Matrix—a simple yet powerful tool to map use cases by how much effort they require and how much impact they’ll deliver.
What Is the Impact vs. Effort Matrix?
This tool helps agencies focus on what’s worth doing, especially when time, talent and resources are limited. Each AI idea gets placed into one of four categories:
- Quick Wins (High Impact, Low Effort): Prioritize these immediately.
- Major Projects (High Impact, High Effort): Worth the investment—plan carefully.
- Fill-Ins (Low Impact, Low Effort): Do when time permits.
- Thankless Tasks (Low Impact, High Effort): Avoid or minimize these.
“We see hundreds of AI ideas across agencies,” one panelist said. “But when you apply the matrix, only a handful have real traction. The juice has to be worth the squeeze.”
The matrix helps filter noise and ensure teams are spending time on the projects most likely to scale, succeed and support the mission.
6. Build with Scale in Mind, Even If You Start Small
AI is experimental. Not every idea will pan out. But successful projects need a path to grow from day one.
“Do a small test with an enterprise mindset,” said Matthew. “Security, governance and scale should be built in from the start.”
Leong agreed: “Get your data ducks in a row, and everything else will follow. You don’t want to make long-term bets on projects that were never designed to scale.”
7. Custom or Off-the-Shelf? Choose Based on Complexity
Should agencies build custom platforms or adapt off-the-shelf tools? It depends.
“Don’t overpay for generic tools,” said Matthew. “But for deep, high-end capabilities, you may need in-house builds—just know the tradeoffs.”
The more specialized the use case, the more likely a tailored solution is required. But whether buying or building, the panel emphasized the importance of involving records officers, legal teams and SMEs early—not just the CIO chasing the next shiny object.
Final Thought: The Data Is There. The Champions Are Too.
The core message of the session? Agencies already have the data—and they have the people who care about getting it right.
What’s missing is coordination, prioritization and a strong governance foundation.
Start with strategy. Talk to your CDO. Use the matrix. Build with intent.
Carahsoft Technology Corp. is The Trusted Government IT Solutions Provider, supporting Public Sector organizations across Federal, State and Local Government agencies and Education and Healthcare markets. As the Master Government Aggregator for our vendor partners, including ZL Tech, we deliver solutions for Geospatial, Cybersecurity, MultiCloud, DevSecOps, Artificial Intelligence, Customer Experience and Engagement, Open Source and more. Working with resellers, systems integrators and consultants, our sales and marketing teams provide industry leading IT products, services and training through hundreds of contract vehicles. Explore the Carahsoft Blog to learn more about the latest trends in Government technology markets and solutions, as well as Carahsoft’s ecosystem of partner thought-leaders.
