AI generates answers; an intelligence system needs evidence
A generative model turns inputs into plausible outputs: syntheses, classification proposals, questions, drafts or explanations. Its usefulness is real, but the fluency of its answer does not show that it has consulted the necessary sources or that the interpretation is correct. An answer can be well written and still rest on outdated facts, misread sources or nonexistent references.
That is why it pays to separate linguistic ability, access to information and reliability of the result. The model can summarise a document supplied by the user; that is not the same as having verified what the document states. It can suggest where to investigate; that does not show it has visited those sources. If it has search tools, it can locate information; even so, you must document what it consulted, when, and what was left out.
A delegation map by risk level
Classify each task with four questions: does it need current data? does it require specialised interpretation? could an omission have consequences? will we be able to verify the result?
| Task | Appropriate role for AI | Recommended human control |
|---|---|---|
| Rephrase an intelligence question | Propose variants and refinements | Validate the link to the decision |
| Summarise supplied material | Extract themes and locatable quotes | Check correspondence with the originals |
| Label records | Apply an explicit scheme | Review a sample, disagreements and borderline cases |
| Compare actors’ moves | Organise differences and raise hypotheses | Check facts, context and omissions |
| Detect a strategic threat | Raise signals and plausible scenarios | Avoid verdicts without evidence and expert assessment |
| Recommend an investment or regulatory compliance | Prepare alternatives and open issues | Human decision and specialised professional review |
The criterion is not “AI yes / AI no”, but which autonomy we authorise under which controls. AI can propose, but it must not present an inference as a direct observation. In relevant decisions, the person in charge must be able to reconstruct which evidence supports each conclusion.
Interactive · Classify
How much autonomy do you give the AI?
Place each task at the right delegation level.
Activity: classify tasks by the level of autonomy you would give the AI.
Company A case · simulation
Fictional caseCompany A case · An overconfident sentence
The assistant receives S01 (corporate research announcement), S03 (patent application) and S04 (press mention of a demonstrator). It returns: “Supplier B already dominates the thermal storage market and Company A must invest immediately”. That answer confuses exploration, industrial property and commercial maturity. None of the signals demonstrates market share, sales or repeatable industrial performance.
A useful alternative is: “There is initial technical activity and at least one demonstration announcement. There are not enough data here to conclude commercial adoption. Independent validations, manufacturing capacity and verified purchases should be cross-checked before changing the strategy”.
Educational simulation: fictional information and organisations.
Interactive · Rewrite
Fix the overconfident sentence
The assistant received S01, S03 and S04. Which version could you sign?
Activity: choose the defensible rewrite of an overconfident AI answer.
“Supplier B already dominates the thermal storage market and Company A must invest immediately.”
Do it yourself
Do it yourself · Delegation agreement (15 min)
Choose three routine tasks. For each one, write: authorised input, the operation the AI will perform, expected output, required check, risk of error and criterion for stopping the automation. Test the agreement with an assistant and check whether it respects the prohibitions.
Deliverable: a “task / autonomy / evidence / validation” sheet that you can add to your monitoring plan.
Classification dictionary
Define unambiguous labels for humans and AI.
See the template →Matrix of hypotheses and alternatives
Assess rival explanations of a signal.
See the template →Check it · self-assessment
Check your delegation agreement
This part needs professional judgement: there is no automatic marking. Complete your work in the practice, rate it with the lesson's rubric and compare it with the indicative solution.
What your answer should include
- Each task defines authorised input, operation, output and check.
- There is an explicit criterion for stopping the automation.
- You prohibit direct publication and the processing of confidential information without permission.
- A human reviewer checks the text and the destination of the data.
Indicative solution
Situation: ask the AI to classify S03 as an event type. Reasoned solution: authorise reading only the record, require `event: patent_application`, keep `evidence_id: S03` and set `grant_date: not_stated`; prohibit direct publication and the processing of confidential information without permission. Stop if it labels "patent granted", if it invents documents or if it does not cite the ID.
Mistake to avoid: Accepting "AI is usually right" as a control.
Illustrative solution for a fictional case: reasoned alternatives are valid.
Common mistakes
Believing that a citation is proof of a complete reading; assuming that an assistant has internet access by default; asking “what is happening?” without delimiting country and dates; entering confidential information without authorisation; confusing elegant recommendations with justified decisions.
Rule to take away: before using an AI answer, ask “what would I have to check to be able to sign off on this conclusion?”. If you have no answer, you do not yet have cross-checked intelligence.
Optional extensionExtension · Before delegating, define the cost of error
Autonomy does not depend on how sophisticated a task seems, but on what happens if the output is wrong and nobody detects it. An exploratory classification can tolerate uncertainty if it ends up in a review tray. A message addressed to management that attributes an acquisition to a competitor can alter decisions and call for stronger checks. To design the delegation, document five dimensions: exposure of information, ability to check, reversibility, recipient and deadline for detecting the failure.
Three operational levels: (A) draft assistance: the AI proposes, the human selects; (B) supervised execution: it operates on authorised sources and structure, but no conclusion is disseminated without control; (C) bounded automation: repetitive low-risk actions, monitored and reversible, with stop criteria. These levels are a pedagogical proposal of the Academy, not an external standard or an automatic property of a product.
Company A case · One task, three different agreements
| Agreement | Permitted input and output | Human test | Immediate stop |
|---|---|---|---|
| Headline draft | S01–S04; list of possible changes | Go back to the cited record | A source is invented |
| Event classification | IDs, R09 taxonomy; typed table | Review ambiguity and a sample | Confuses a patent application with a granted patent |
| Recommendation to management | R10 matrix; options and premises | Expert validation and sign-off | Presents intent or a pilot as a sale |
An instruction such as “act as an expert” controls none of these risks. By contrast, “do not issue a commercial conclusion if contracts, clients or corroboration are missing” is an observable test. The agreement must specify who approves dissemination and who keeps the evidence outside the chat.
Acceptance practice · An agreement that another person can run
In R08, write an agreement for classifying Company A’s technical signals. Include five permitted inputs, three prohibitions, the output structure and three adversarial tests: a duplicated source; a title that suggests a sale but describes a pilot; and a record with no date for the event. Ask another person to run it with authorised data and note exactly where uncertainty appeared. It passes if the system declares “not stated” when data are absent, distinguishes interpretation from fact and leaves a traceable record of the reviewer.
Reusable output: R08 AI delegation and use agreement; R07 as a complement for auditing any claim;
References and limits
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (2023). DocumentAI risk management as a voluntary framework. The Academy's audit of claims is its own work, not a NIST procedure.Reference verified on 2026-10-18.
- NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (2024). DocumentRisks specific to generative AI, including confabulation.Reference verified on 2026-10-18.
- Company A is a fictional case: its data, companies and figures do not describe real markets or organisations.
- The AI guidelines are the Academy's own work. They are not a NIST procedure and do not replace a risk assessment by your organisation.
Intelligence Lab · L05
Evidence auditor
Ready to apply it? Open the Evidence auditor in the Lab and review each claim: fact, inference, error or lack of evidence. Output: claim audit. Works in the browser, no server, no account and no connected AI. The result may need reviewing and adapting to your case.