A string of commercial burglaries has run across three towns since August, and the regional task force finally has a lead. The analyst runs the case through the investigative assistant, which pulls plate-reader hits near four of the scenes, cross-references them against prior arrests, and returns a ranked list. The man at the top has a burglary conviction from eleven years ago and a pickup that passed two of the cameras on the right nights. The detective writes the affidavit, the judge signs the warrant, and the man spends nine days in county jail before his lawyer gets a look at the discovery. The lawyer's first question is not about the man at the top of the list. She wants to know who was second.
Nobody can tell her. The tool had ranked a second vehicle, a van that passed three of the cameras, not two, and that van belonged to nobody with a record. The detective never saw it on the screen he used, so it never reached the file and never reached the prosecutor. When it finally came out, the case did not survive the week. A van seen at more scenes than the defendant's truck is exactly the kind of information the government owes the defense. The task force did not hide it. It simply never knew it had it.
I tell that story to public-sector leaders a lot, because it puts the value of AI in government where it belongs. The tool was not wrong, exactly. It surfaced the right van. What failed was everything around it: the record of what it considered, the person who was supposed to check it, and the obligation that follows every piece of evidence the state gathers. In government, that surrounding work is most of the job.
When the output is an exercise of power
Commercial AI programs get measured by throughput and margin. Public agencies inherit those metrics from vendors, so the slides talk about leads generated, backlog cleared, and analyst hours saved. Those numbers are real, but they measure the cheap part. A lead that falls apart at a suppression hearing does not save time. It spends it twice and adds a wrongful detention, a civil claim, and a news cycle on top.
The difference is what the output does. When a bank's model declines a loan, someone loses an option. When a government system flags a person, it can start a chain that ends in a search, an arrest, a denied benefit, an enforcement action, or an entry in a watchlist that follows them for years. The state is the one actor that can compel, detain, and punish, and the Constitution, statute, and decades of case law were written to put conditions on exactly that. An AI system that sits anywhere in that chain inherits all of those conditions.
So the honest return on public-sector AI is not leads produced. It is outcomes that hold: arrests that stand up, assessments that survive a second look, enforcement actions that survive judicial review, and benefit decisions that survive appeal.
A lead is a hypothesis
The most expensive mistake in this space has a well-documented shape. In 2020, Detroit police arrested a man in front of his family on the strength of a face-recognition match to a blurry surveillance image. He was not the person in the picture. The city's 2024 settlement included a commitment that a face-recognition result, on its own, could not be the basis for an arrest. That rule should not have needed a lawsuit. A match is a reason to investigate, not a conclusion, and the same goes for a link chart, a risk score, or a ranked list of vehicles.
The trouble is that people treat a confident machine output as further along than it is. Researchers call it automation bias, and investigators under caseload pressure are as subject to it as anyone. The control is procedural before it is technical: every AI-generated lead has to be corroborated by something the tool did not produce before it can support a warrant, a charge, or an adverse decision. The affidavit should say what that independent corroboration was.
Fairness sits right here, and for government it is a constitutional question as much as an ethical one. NIST's 2019 evaluation of face-recognition algorithms found false-match rates varying widely by demographic group, by factors of ten or more for some algorithms. Systems trained on historical enforcement data have a subtler version of the same problem: they learn where officers looked, which is not the same as where crime happened, and they send officers back to the same places. If an agency cannot show error rates by group for the tools it uses, both before deployment and on live cases, it cannot say whether the burden of its mistakes falls evenly. The people who bear those mistakes will be the ones asking.
What the system set aside belongs in the file
Since Brady v. Maryland in 1963, prosecutors have owed the defense material evidence favorable to the accused, and that duty reaches what the investigating agency knew, not only what the prosecutor personally saw. AI changes the size of that obligation in a way most procurement documents have not caught up with. A tool that ranks twenty candidates and shows the top three has still considered twenty. The confidence scores, the near misses, the alternate matches, and the records it weighed and discounted are all things the government had in its possession.
That means the record of what the system set aside is evidence, and it has to be kept like evidence. Capture the full candidate list, the scores, the sources each one rested on, and the version of the model that produced them, at the moment the lead is generated. We have written about decisions you cannot run twice, and why a log that records what happened is different from a record that lets you re-derive it. In a criminal case that gap has a name. It is a disclosure failure, and courts do not care that the tool's interface hid the second-ranked van.
The affiant swears, not the software
Every consequential act of government has a person whose name goes on it: the detective who swears the affidavit, the analyst who signs the assessment, the examiner who recommends the enforcement action, the caseworker who denies the claim. That person is attesting to what they know. They cannot swear to reasoning they never saw.
The intelligence community has a useful model for this. Its analytic tradecraft standards, set out in Intelligence Community Directive 203, require analysts to describe the quality and reliability of their sources, express uncertainty clearly, and keep underlying intelligence distinct from their own assumptions and judgments. A generated summary that blends a signals report, an open-source article, and the model's inference into one smooth paragraph breaks all three at once. The fix is to make the separation visible. Every claim in the output points to its source, carries a confidence the analyst can see, and is marked as either reported or inferred.
Regulators face the same test in a different forum. An agency that brings an enforcement action or issues a rule must be able to show a rational connection between the facts it found and the choice it made, or a court can set the action aside as arbitrary. "The model flagged them" is not a rational connection. A reviewer who can see which filings, which transactions, and which pattern drove the flag can build one. We argued in a hand on the wheel that oversight means a person who could have decided differently and had what they needed to do it. In government that person is also the one a judge will question.
Collection is the decision made before any model runs
Privacy in public-sector AI is usually discussed as a data-protection problem. It starts earlier than that. The most consequential choice is what the system is allowed to reach in the first place. AI makes it cheap to join plate reads, phone records, social media, commercial data, and agency files into a single picture of a person's life, and the Supreme Court said in Carpenter v. United States in 2018 that a detailed record of someone's movements can carry Fourth Amendment protection even when a third party holds the data.
For an agency, that turns into a few firm rules. Bind each dataset to the purpose and legal authority it was collected under, so a tool built for one case cannot pick it up for another. Scope what the system can see to the case and the person working it, a point we made in the answer that belonged to someone else. Set retention to what the law and the case require, then actually delete. Purpose limits that live only in a policy manual will lose to a search box every time.
Legitimacy is the asset you are spending
A company that loses customer trust loses revenue. A government agency that loses public trust loses cooperation: witnesses who stop calling, jurors who discount testimony, communities that stop reporting crime, and legislatures that ban the tool outright. Several cities have already restricted face recognition for exactly that reason. Legitimacy is the public sector's version of brand equity, and every unexplained AI decision draws it down.
The agencies that keep it tend to do the unglamorous things in public. Federal agencies already publish inventories of how they use AI. State and local bodies are increasingly expected to do the same, along with impact assessments, civilian oversight review, and audit results. An agency that can produce, on request, what a tool does, what it was tested against, how it performs by group, and who reviews its outputs can defend its use in a council meeting. One that cannot will end up defending it in a courtroom instead.
Where Throughline and Neura-Cortex fit
Our work for law enforcement, investigative, regulatory, and intelligence missions starts from the person who will have to answer for the outcome. Throughline, our agentic case management solution, carries each case with its evidence, its sources, and its trail attached, so the investigator can see what supports a lead and what the system weighed and set aside. Lumen keeps the statutes, policies, and guidance an analyst relies on traceable and current. Both run on Neura-Cortex, the shared substrate that holds the guardrails, the identity and access controls that scope what each user and agent can reach, the evaluations, the policy packs for regimes such as ITAR, and the audit log that records each decision when it is made. Because those controls sit in the shared floor under every solution, no unit has to remember to switch them on.
How those systems are engineered stays with us. What we will say plainly is the position behind them: for a public agency, responsible AI and effective AI turn out to be the same program, because the only results worth having are the ones that survive a defense lawyer, a judge, and a reporter.
A private company's AI has to be right. A government's AI has to be right, provably fair, and explainable to the person it was used against, because that person has rights the company's customers never had.
Before the next procurement
Most of the controls above are cheapest to get at the moment an agency buys the tool and hardest to retrofit after. Put them in the contract. Require the vendor to retain the full output of every run, including candidates and scores the interface does not display, in a form your records and discovery staff can produce. Require error rates by demographic group on test data that resembles your population, and the right to rerun those tests on live cases. Require that every output cite its sources and mark what is inferred. Require that your agency owns the audit log, not the vendor. If a vendor cannot agree to those terms, you have learned something important about what you would be defending later.
Run the burglary case again with those terms in place. The analyst sees the ranked list, the van included, with the camera hits behind each entry. The detective corroborates before he writes, and the van turns out to belong to a cleaning contractor's former employee who had keys to two of the buildings. The man with the old conviction never spends a night in jail, the task force closes the series, and the defense, when it gets the file, finds that the government's record of what its tool saw is complete. Nothing about the software changed between the two versions. What changed is that somebody wrote down the van.
MTekLabs designs, deploys, and governs production-grade agentic AI platforms for government and commercial enterprises. Our work for law enforcement, investigative, regulatory, and intelligence missions is built human-in-the-decision-loop, with provenance, disclosure, and access controls designed in from the start. Explore our solutions at mteklabs.com.
Could your agency produce everything its tools considered on last month's cases?
We would be glad to talk through what it takes to put AI into investigative, regulatory, or intelligence work in a way that survives discovery, judicial review, and public scrutiny, from complete decision records to the human review that keeps a sworn statement meaningful.