Getting agency data ready for AI: a practical sequence

Why AI pilots stall

Most agencies that try generative AI hit the same wall within a few weeks. The model works in a demonstration, but the first time it meets real records, the questions begin. Which fields contain personal information? Who is allowed to see the output? Can this data leave our boundary at all? Pilots that cannot answer these questions are paused, and many never restart.

The problem is rarely the model. It is that the data was never prepared for this kind of use. Records were collected for case processing or reporting, governed by rules written for people, not for software that reads everything at once.

Step 1. Inventory and classify

Start with a list of the sources a use case actually needs, not every system the agency owns. For each source, record the owner, the volume, how often it changes and which fields are sensitive. Automated classification tools can scan for common patterns such as Social Security numbers, dates of birth and addresses, but a person who knows the program should confirm the results.

The output of this step is a short table. It is the document the privacy officer and the authorizing official will ask to see first.

Step 2. Tie access to identity you already trust

AI tools should inherit access rules from the identity system the agency already runs. If a caseworker cannot open a record in the case management system, an assistant answering that caseworker should not be able to read it either. Building separate permission lists for AI creates drift and audit findings.

In practice this means passing the user identity through every request and filtering retrieval results against the same groups and roles used elsewhere.

Step 3. Protect sensitive fields before they move

There are four common ways to protect a sensitive field, and the right one depends on the use case:

• Masking replaces part or all of a value so it is unreadable but keeps its shape.
• Redaction removes the value entirely.
• Tokenization swaps the value for a stable token that can be reversed only by an authorized service.
• Synthetic data generates realistic records with no link to real people.

Synthetic data is especially useful for pilots and testing. Teams can build and evaluate an AI application on thousands of realistic records without a data use agreement or live sensitive data in the development environment.

Step 4. Measure quality

AI amplifies data quality problems. Duplicate records, missing values and inconsistent codes produce confident but wrong answers. Score each source on completeness, validity and consistency before it feeds an AI application, and set a threshold below which the source is fixed first.

Step 5. Put a gate in front of the model

The last step is a single, controlled path between the data and any model. Every request passes through one intake point, sensitive fields are filtered according to policy, a reviewer samples the output on a schedule, and only cleansed data reaches the model. This gate is what lets an agency say, with evidence, what data an AI tool has seen.

A 90-day starting plan

1. Weeks 1 to 3: choose one use case and inventory its sources.
2. Weeks 4 to 6: classify fields, agree protection rules with the privacy office and generate synthetic test data.
3. Weeks 7 to 10: build the gate and a working prototype on protected data.
4. Weeks 11 to 13: run an evaluation with program staff and prepare the authorization package for the next phase.

Meritore's Data & AI Engineering practice helps agencies through each of these steps. Learn about the practice.