From reading texts to designing a taxonomy
Automatic classification can help organise large volumes of news, patents, calls for proposals and corporate communications. But first we must decide what each label is for. A useful taxonomy is not a huge list of categories; it is a consistent set that makes it possible to answer questions and trigger actions.
For competitor monitoring we might need: actor, event date, geography, type of event, technology involved, maturity stage, relationship to the CWF, source and verification status. Not all fields can always be extracted: the value not_stated is usually better than an inference disguised as data.
Minimum schema design
| Field | Rule | Teaching example |
|---|---|---|
event |
One of: research, patent_filed, pilot, agreement, sale, regulation, other | patent_filed |
actor |
Name present in the source, not invented | Supplier B |
event_date |
Date of the event or not_stated; keep separate from publication |
not_stated |
phase |
Exploration, pilot, commercial, undetermined | exploration |
claim |
Neutral summary of what the source states | «An application was published» |
evidence_id |
ID that lets you go back to the material | S03 |
review |
correct, doubtful, incorrect | doubtful |
A schema must document definitions, positive and negative examples and how to treat ambiguous records. If we call both a prototype and a sale a «launch», signals will be hard to compare.
Interactive · Cards
The schema fields, one by one
Flip each field to see its rule and the error it prevents.
Fields of the minimum classification schema, with their rule.
A process that learns from its errors
Start by classifying a small sample manually. Then ask the AI to label it with the same schema and to explain which excerpt supports each non-obvious field. Compare disagreements and correct instructions or categories. A second sample, different from the first, serves to check whether the adjustment generalises.
It is not enough to say that «accuracy is high»: any quality indicator requires a reference set and definitions of what counts as correct. For this training exercise, the priority is to learn to detect false positives, false negatives and fabricated fields.
Company A case · simulation
Fictional caseCompany A case · The difference between pilot and commercialisation
If S04 mentions a laboratory demonstrator, the category sale is wrong; if S03 describes a published application, patent_granted would be wrong too. Asking the AI to choose among explicit categories and justify with an excerpt removes part of the arbitrariness, but edge cases still need review.
The resulting structured data is useful for a time matrix: when does an application appear? when a test? is there verifiable adoption? Classification does not by itself settle the decision: it prepares the ground for the analysis that follows.
Educational simulation: fictional information and organisations.
Do it yourself
Do it yourself · Design and test five labels (20 min)
Create a taxonomy of five event types. Define the rules, label S01–S05 manually and ask an AI for a second classification with tabular output. Compare row by row. Document at least one change to a definition and one example of not_stated.
Deliverable: data dictionary and disagreement log.
Classification dictionary
Define unambiguous labels for humans and AI.
See the template →Log of signals and evidence
Distinguish observation, source and interpretation.
See the template →Check it · closed answer
Label each record
Use the label that the excerpt allows, without inferring more than it says.
S01 · Supplier B announces research on industrial storage.
See the answer
Answer: research_activity. Do not infer sales.
S02 · Blog: «a new technology will replace all batteries in 2027».
See the answer
Answer: needs_review. It is an opinion: do not label a promise as a contract.
S03 · Publication of an application on a heat exchanger.
See the answer
Answer: patent_application. The grant is not on record (`concesion: not_stated`).
S04 · Supplier B presents a laboratory demonstrator.
See the answer
Answer: pilot_demonstrator. Do not infer profitable exploitation.
And now, your own answer
This part needs professional judgement: there is no automatic marking. Complete your work in the practice, rate it with the lesson's rubric and compare it with the indicative solution.
What your answer should include
- Five event types with inclusion and exclusion rules.
- An example of «not_stated».
- A row-by-row disagreement log against the AI's classification.
- At least one documented change to a definition.
Indicative solution
What is assessed is the justification by ID and the disagreement log; the aim is not to measure the real performance of the classifier with four examples.
Illustrative solution for a fictional case: reasoned alternatives are valid.
Common mistakes
Creating overlapping categories, not distinguishing publication date from the date of the event, asking for arbitrary probabilities, inferring absent entities, measuring the volume of labels instead of their usefulness for decisions, or adjusting the schema without versioning it.
Rule to take away: a label is not true because an AI wrote it; it is operational when another analyst understands its definition, can reproduce it and knows when to review it.
Optional extensionDeep dive · Classifying is not filling in the gaps
An empty field is useful information if it means «the source does not provide this data». The error appears when a classifier fills gaps with plausible conventions. It helps to separate three values: not_stated (data absent from the material), not_applicable (the field does not apply to the event) and needs_review (the source allows rival interpretations). This distinction makes records comparable and avoids conclusions based on apparent precision.
Set up an R09 dictionary with five initial classes: research_activity, patent_application, pilot_demonstrator, alliance_agreement, commercial_adoption. The labels are designed for the exercise; they do not describe any provider’s ontology. For each category, define an inclusion criterion, a counterexample and the minimum textual evidence. If a news item mixes an application and an alliance, allow multiple labels when the schema provides for them, or separate the events: never force a single label just for the convenience of the report.
Company A · Classification test and discrepancies
| Fictional record | Justifiable label | Error to prevent |
|---|---|---|
| S01 · Research announced | research_activity | Inferring commercial adoption |
| S03 · Application published | patent_application | Turning it into a grant |
| S04 · Demonstrator | pilot_demonstrator, according to the exact text | Turning it into a sale |
| S02 · Speculative opinion | needs_review or another document type | Labelling a promise as a result |
Validation protocol: prepare eight fictional records with a human label before testing the prompt, use four to adjust definitions and four to check that the instructions do not merely memorise examples. Keep the discordant answers, the evidence each reviewer used and the version of the dictionary. Do not publish an accuracy figure without stating the size and composition of the sample; eight examples teach a method, they do not demonstrate production performance.
Practice · Review of two errors
Ask the assistant to return the fields event, actor, event_date, publication_date, evidence_id, passage, review. Deliberately introduce a date that is not present and a claim that mixes an announcement with execution. The exercise is passed if the fields without evidence remain not_stated, the system identifies at least one ambiguity and can explain each classification using the text supplied.
Output: R09 Classification dictionary, R06 Log of signals and evidence;
Interactive · Decide
not_stated, not_applicable or needs_review
An empty field is useful information. Choose the right value in each case.
Activity: choose between not_stated, not_applicable and needs_review.
S03 is a patent application. Field: grant_date.
S06 is a regulatory consultation. Field: “patent applicant”.
A news item says Supplier B “closed a deal”, without clarifying whether it is exploratory or a supply deal. Field: event.
S04 mentions a demonstrator without saying when the test was done. Field: event_date.
S02 claims that the technology “will replace all batteries in 2027”. Field: event.
References and limits
This lesson cites no external sources. It develops the Academy method from Innguma's training material.
- Company A is a fictional case: its data, companies and figures do not describe real markets or organisations.
- The guidelines on AI are the Academy's own work. They are not a NIST procedure and do not replace a risk assessment of your organisation.
Intelligence Lab · L04
Signal log
Ready to apply it? Open the Signal log in the Lab and keep observation, source and interpretation separate in your log. Output: signal log. Works in the browser, no server, no account and no connected AI. The result may need reviewing and adapting to your case.