AI That Cites Its Sources for Louisiana Local Government

Most vendor demos of "AI with citations" show a chat answer with a footnote under it. The footnote proves the system retrieved a document. It doesn't prove the document says what the answer says. When the assistant is answering residents' questions about ordinances, meeting minutes and fees, that gap is the main risk.

I built a citation-first assistant for city government that only answers from cited public records. What follows is the architecture I would hold any vendor to, including myself, and the questions I would ask in a demo.

Why does AI with citations still hallucinate?

Retrieval helps a lot. According to the UNC School of Government, a chatbot built on retrieval augmented generation (RAG) has a far better chance of accurate answers because it limits the universe of information the model draws from, such as a local government's own website. Helping a lot still leaves real errors. A Stanford HAI study summarized by MIT Sloan EdTech found that general-purpose chatbots hallucinated on 58 to 82% of legal research queries on 2023-era models, and specialized legal tools built on RAG still hallucinated more than 17% of the time.

The failures in a municipal or parish system follow predictable patterns:

• Misgrounding. The answer cites a real ordinance section, but the claim isn't in that section.
• Stale grounding. The cited text is real but has been amended or repealed.
• Gap filling. Retrieval returns nothing relevant, so the model answers from its training data, which might be another city's code or general state law.
• Synthesis errors. The answer merges two sections into a rule that neither one states.
• Scope drift. A conversational answer reads like an official response to a records request.

Each one needs its own control. A footnote alone handles none of them.

What architecture does a cited, verifiable assistant need?

The standard RAG flow, as CMARIX lays it out, runs from user query to embedding model, vector database lookup, top-K chunks, the LLM with grounded context, and then a cited response. That flow ends at the model. A public-sector system needs three more stages: provenance at ingestion, a retrieval gate before generation, and a verifier after it.

1. Ingest with provenance

Every chunk has to know exactly what it came from. Store documents as versions rather than files, so the assistant can tell what was in force on a given date.

CREATE TABLE source_document (
  id            bigserial PRIMARY KEY,
  doc_type      text NOT NULL,      -- ordinance, minutes, resolution, fee_schedule
  identifier    text NOT NULL,      -- e.g. 'Sec. 18-42' or meeting date
  version_no    int  NOT NULL,
  adopted_on    date,
  effective_on  date,
  superseded_on date,               -- NULL while in force
  status        text NOT NULL,      -- draft, adopted, repealed
  source_url    text NOT NULL,
  sha256        text NOT NULL,      -- hash of the exact text indexed
  UNIQUE (doc_type, identifier, version_no)
);

CREATE TABLE source_chunk (
  id          bigserial PRIMARY KEY,
  document_id bigint REFERENCES source_document(id),
  locator     text NOT NULL,        -- section, subsection, agenda item, page
  body        text NOT NULL
);

Chunk along legal structure, not by a fixed token count. Ordinances split by section and subsection. Minutes split by matter decided. That maps directly onto Louisiana's Open Meetings Law, which requires minutes to include the date, time and place, the members present or absent, the substance of all matters decided and, on a member's request, individual votes.

Don't build the corpus by crawling the website. The same statute only requires a public body with a website to keep posted minutes there for at least three months, so the website is a window, not an archive. Ingest from the clerk's system of record.

2. Retrieve with hybrid search and a gate

Pure vector search handles exact identifiers like "Ordinance 2026-14" or "Sec. 18-42" poorly. Run lexical search alongside it and merge the results. Filter to in-force versions by default, and include historical versions only when the question asks about a past date.

Then gate the retrieval. If the best results fall below a relevance threshold, the system refuses and routes the resident to a human. Calibrate that threshold on the agency's own test questions. A vendor default isn't good enough.

3. Generate claims, not prose

Have the model return structured output. Each claim carries the chunk ID and the verbatim quote that supports it. Prose gets assembled only after verification.

4. Verify before anything is shown

def verify(claims, retrieved_ids, chunks):
    kept = []
    for c in claims:
        chunk = chunks.get(c["chunk_id"])
        if c["chunk_id"] not in retrieved_ids or chunk is None:
            continue                      # cited something it was not given
        if normalize(c["quote"]) not in normalize(chunk.body):
            continue                      # quote is invented or altered
        if chunk.document.superseded_on is not None:
            continue                      # stale version
        if not entails(chunk.body, c["text"]):
            continue                      # claim goes beyond the quote
        kept.append(c)
    return kept or REFUSE

The substring check is deterministic and cheap, and it catches fabricated quotes outright. The entailment check, done by a second model or by rules for narrow domains like fees, catches claims that stretch a real quote. Unsupported claims get dropped. If nothing survives, the system refuses.

5. Log the whole trace

For each answer, record the query, retrieved chunk IDs and versions, the model and prompt versions, the raw output and the verifier result. Without that log you can't explain a disputed answer. Ask your attorney whether those logs are themselves public records, and plan as if they are.

What do Louisiana law and policy change about the design?

Records requests are a separate channel. Under Section 44:32(D), a custodian must respond within three business days. The Legislature also imposed a $100 per day civil penalty personally on a custodian who arbitrarily refuses to respond in that period. Agencies aren't required to create records or answer general questions instead of producing existing records. So when a question is really a records request, the assistant should hand it to the custodian's intake process right away. It should never produce something that looks like a response.

Records work takes real staff time, and third-party numbers on it need careful reading. MuckRock, an outside site that tracks only the requests its own users file, lists a 48-day average response time for the Ouachita Parish Police Jury. The same page also says the agency completes requests 85% faster than other agencies in its jurisdiction, with a 100% success rate recorded. Those figures come from a small, self-selected set of requests on one platform. They aren't an official measure of any agency's performance. I'm not using them to grade anyone. The design point is narrower: an assistant that answers questions about already-published material can take some load off staff, but only if it never intercepts a request that belongs with the custodian.

State AI policy sets a reasonable bar. Executive Order JML 25-109 barred state agencies from procuring AI before December 15, 2025, and requires CIO or agency-head approval to ensure any AI source is secure and reliable. JML 25-103 required an inventory of AI contracts and use cases. The OTS Acceptable Use Policy, effective September 29, 2025, covers state agencies under OTS authority. WBRZ reported that it requires AI to be reviewed and verified to reduce misinformation risk. These were written for state government, but parish and city IT leads can adopt the same discipline without waiting for a mandate.

Answers may end up in court. The Louisiana Supreme Court Technology Commission issued generative AI guidelines on October 3, 2025, and Act 250 of 2025 requires attorneys to use reasonable diligence to verify the authenticity of evidence, including AI-generated evidence. If a parish attorney ever has to account for what the assistant told a resident, the logged trace is how they do it.

The corpus keeps changing. The Ouachita Parish Police Jury serves 160,000 citizens with more than 500 employees across 12+ departments. It adopted its first comprehensive plan and approved a new planning commission. Those bring new documents and amendments, and zoning questions that turn on which version is in force. Some records also live in other systems. Lincoln Parish land records, for example, are searched through the Clerk Connect portal. The assistant should link out to systems like that rather than copy their contents.

Questions to ask a vendor demoing "AI with citations"

Bring your own questions to the demo. Vendor-scripted demos only show the cases that already work.

For the demo itself, ask about a repealed ordinance. Ask a question whose answer isn't in the corpus. Ask a Monroe-scoped assistant about a West Monroe rule. Try a prompt that tells the model to ignore its instructions. On the isolation row, my post on multi-tenant data isolation with row-level security explains what "enforced at the database layer" should actually mean.

How do you know it keeps working after launch?

Build an evaluation set from the real questions your front desk and clerk's office get. Include questions that should be refused. Track these numbers separately:

• Citation validity: cited chunks exist, were retrieved, and contain the quote.
• Support rate: claims entailed by their quotes.
• Correct refusal rate: unanswerable questions actually get refused.
• Stale citation rate: citations to superseded versions.
• Retrieval recall: the right section shows up in the top results.

Rerun the full set whenever the corpus, model, prompt or threshold changes. Look at failures by category rather than as a single average. One wrong fee amount does more damage than a slightly lower overall score suggests. Costs and timelines for this work depend mostly on how clean and versioned your source records already are, how many systems they live in, and how much of the evaluation set your staff can help write.

Where to start

1. Inventory the sources: the code of ordinances, adopted minutes, resolutions and fee schedules, and the system of record for each.
2. Decide which questions the assistant may answer, and write down the routing rule for anything that looks like a records request.
3. Collect 2 to 3 months of real resident questions to seed an evaluation set.
4. Put the vendor table above into your RFP and score demos against your own questions.
5. If nobody on staff owns this technically, consider fractional CTO help for the evaluation and procurement phase.

If you're scoping a cited assistant for a parish or city, email me at hello@josephleporati.com.