Trust check
How to Verify AI Knowledge Base Citations and Unsupported Claims
Audit a RAG or AI knowledge-base answer claim by claim to find wrong pages, mismatched chunks, stale sources, and unsupported conclusions.
Article packet
Workflows
People debugging AI knowledge-base answers that look grounded but cite the wrong page, chunk, version, or source
5 min read
01
A citation marker proves that a source was attached, not that the source supports the claim.
02
Check each important claim against the exact page, chunk, and source revision.
03
Classify the result as supported, partial, unsupported, or stale before reusing the answer.
01
Quick answer
To verify an AI knowledge-base answer, split it into testable claims, open the exact cited page or chunk for each claim, and record whether the current source fully supports it. Treat missing, broken, wrong-page, mismatched, and stale citations as different failures instead of one generic hallucination score.
Wenlan keeps Sources, atomic knowledge, and maintained Pages separate. Page citations, source IDs, revisions, stale state, lint, and human review make the evidence trail inspectable, but Wenlan does not automatically prove that every source is true or replace review of important claims.
02
When this problem appears
An answer can be factually plausible while its citation points to the first retrieved source, an unrelated chunk, or an older document revision. The failure stays hidden when reviewers check only that citation markers exist instead of comparing each claim with the cited evidence.
03
Run a claim-to-evidence audit
Start with one suspect answer. Keep the audit small enough that another person can reproduce every decision.
- Copy the answer into separate factual claims; ignore transitions and opinions that make no factual assertion.
- For each claim, record the citation marker, source ID, document title, page or section, and source revision when available.
- Open the cited location and check whether it supports the whole claim, only part of it, a different claim, or nothing at all.
- Mark the claim supported, partial, unsupported, or stale; do not convert a missing row or unavailable source into a pass.
- Check numbers, negations, attribution, scope, and dates separately because a nearby passage can still contradict the answer.
- Correct the answer or keep the conclusion unknown before refreshing a maintained page.
- Run lint and human review, then repeat the same question to confirm the repaired answer returns to the intended source.
Inspect and review one Wenlan Page
/pages <topic>
/lint
/curate04
What to check next
This workflow is a diagnostic checklist, not an automatic RAG benchmark. A cited source may itself be wrong, outdated, or non-authoritative, so consequential claims still require the maintained first-party source and appropriate human review.
Verify one answer before expanding the knowledge base
Use Wenlan to inspect the Page, source IDs, stale state, lint findings, and review path, then keep only claims the current evidence supports.
FAQ