Filtering means deciding what we might stop seeing
A source can be good and still produce an unmanageable amount of information. Filters help select records, but they also introduce false negatives. A filter that is too narrow can remove a critical signal; one that is too broad forces you to review hundreds of trivial results. That is why we do not design filters as a list of “right” words: we design them as testable hypotheses.
Innguma’s materials taught how to automate capture and activate rules to reduce noise in RSS channels. The principle still holds, but the specific tools and their interfaces change. Today we can use Boolean rules, field-based filters, automatic classification or AI. No approach removes the need for testing, precision tracking and coverage checks.
Designing the work chain
Capture → normalisation → duplicate identification → selection → classification → checking → dissemination. Each step asks a different question: have we received the publication? Have we identified its date and origin? Is it a repeat? Is it relevant to our CWF? Which category does it fit? Can it be verified? Who needs to know about it?
Before activating a filter, gather a small sample labelled by hand: relevant, irrelevant and doubtful records. Run the filter over that sample. Examine the relevant records it discards and the irrelevant ones it keeps. Reducing the total number of alerts is not enough. If there are few critical signals, it is more important to keep them visible than to boast about a large noise reduction.
When AI is used to classify, require defined categories, examples, the possibility of review, traceability of origin and a “needs review” label. A relevance prediction is not evidence of the fact described. A generated summary does not replace the source either.
Interactive · Put in order
Put the workflow in order
Each step answers a different question. Put them in order.
Activity: put the seven steps of the workflow in order.
- Capture · have we received the publication?
- Normalisation · are the date and origin identified?
- Duplicates · does it repeat?
- Selection · is it relevant to the CWF?
- Classification · which category does it fit?
- Verification · can it be verified?
- Dissemination · who needs to know about it?
Company A case · simulation
Fictional caseCompany A case · One word, different consequences
Suppose Company A monitors “thermal storage”. A news item about a trade fair may mention it without adding anything relevant. A public consultation document may be important even though it uses different vocabulary, such as “industrial thermal flexibility”. A strictly literal filter could keep the first and discard the second. To test it, define positive and negative examples on Company A’s corpus, review synonyms, entities and event type, and document which cases fall in the grey zone.
A notice becomes an assessable signal when you record the title, the date of the event (if known), the publication date, the source, the link to a CWF, an explanation of relevance and the validation status. An alert that arrives late can still be important; a new publication about an old event is not necessarily a strategic novelty.
Educational simulation: fictional information and organisations.
Interactive · Filter lab
Lab: test your filter with Company A's mini test set
Turn terms and exclusions on or off. The table instantly compares what your rule does with human judgement.
Filter lab with publications F01–F08 from the Company A case.
Inclusion terms
Exclusions
| ID | Publication | Human judgement | Rule | Result |
|---|---|---|---|---|
| F01 | “Industrial pilot for thermal storage confirmed” | relevant | ||
| F02 | “Trade fair looks back on ten years of thermal storage” | irrelevant | ||
| F03 | “Consultation on thermal flexibility in industrial processes” | relevant | ||
| F04 | “Operational contract for a heat accumulation solution” | relevant | ||
| F05 | “Scientific article with preliminary result and no industrial test” | doubtful | ||
| F06 | “Advertisement with no verifiable information” | irrelevant | ||
| F07 | “Date of an already announced contract is corrected” | relevant | ||
| F08 | “Incidental mention in an old thermal storage catalogue” | irrelevant |
Do it yourself
Do it yourself · Filter test (20 minutes)
Write an inclusion rule in plain language: “Select publications that announce verifiable industrial pilots, development alliances or regulatory milestones relating to thermal storage in the defined markets”. Add three possible exclusions, but never “exclude all content without the exact word”. Build a sample of 10 fictional or authorised records and label them by hand. Compare the filter’s decision with your judgement and explain the disagreements. Adjust the rule and save the version, date and owner.
Quality test. You must be able to explain why a result gets in or out, and have a way to correct it. A smaller number of alerts on its own does not prove better intelligence.
Query and filter test
Check for omissions and false positives before activating rules.
See the template →Log of signals and evidence
Distinguish observation, source and interpretation.
See the template →Check it · closed answer
What does the literal rule do with each publication?
Compare human judgement with the literal rule “thermal storage” from the table in the lesson.
F01 · “Industrial pilot of thermal storage confirmed” (relevant; the rule keeps it).
See the answer
Answer: Correct. Relevant and kept.
F02 · “Trade fair looks back over ten years of thermal storage” (irrelevant; the rule keeps it).
See the answer
Answer: False positive. The rule keeps an irrelevant publication because it contains the literal word.
F03 · “Consultation on thermal flexibility in industrial processes” (relevant; the rule discards it).
See the answer
Answer: Omission. Relevant but discarded: a semantic variant is missing.
F04 · “Operating contract for a heat accumulation solution” (relevant; the rule discards it).
See the answer
Answer: Omission. Relevant but discarded: “heat accumulation” does not match the literal word.
F05 · “Scientific article with a preliminary result and no industrial test” (doubtful; the rule discards it).
See the answer
Answer: Needs review. Treat it as a review case, neither as a confirmed fact nor as a safe discard.
F06 · “Advertisement with no verifiable information” (irrelevant; the rule discards it).
See the answer
Answer: Correct. Irrelevant and discarded: the rule gets it right.
And now, your own answer
This part needs professional judgement: there is no automatic marking. Complete your work in the practice, rate it with the lesson's rubric and compare it with the indicative solution.
What your answer should include
- Your sample has at least ten manually labelled records.
- There is at least one example of an omission and one of a false positive.
- You save a new version of the rule with a date and an owner.
- You do not invent performance figures.
Indicative solution
The literal-word filter keeps F01 and also F02, which is irrelevant, but it may omit F03 and F04, which are relevant. Fixing it requires including semantic variants of the event and reviewing the discards. You can try your versions in Lab L03 · Filter test.
Illustrative solution for a fictional case: reasoned alternatives are valid.
Common mistakes
Optimising only for volume; changing filters without versioning; mixing up the publication date with the date of the event; treating automatic labels as truth; hiding which rules caused exclusions. Key takeaway: a capture policy and a filter test sheet.
Optional extensionExtension · Designing a test with observable errors
Every filter sacrifices something. Precision expresses what proportion of what it selects is pertinent; recall, what proportion of the pertinent records in a known sample it manages to keep. They are not guarantees about the world: they depend on the sample, its human labels and whether it contains hard cases. If a team analyses only the alerts that were selected, it cannot estimate how many relevant ones were left out; it must also review a sample of the discards.
Before changing the rule, declare which failure costs more: missing a regulatory milestone or flooding management with trivial mentions. When the cost of omission is high, a human review queue for doubtful cases is advisable. AI can help classify but can also produce unjustified confidence; keep examples, versions and human decisions, not just percentages.
Company A case · Mini bank of eight fictional publications
| ID | Simulated publication | Human judgement | Literal rule “thermal storage” |
|---|---|---|---|
| F01 | “Industrial pilot of thermal storage confirmed” | Relevant | Keeps |
| F02 | “Trade fair looks back over ten years of thermal storage” | Irrelevant | Keeps |
| F03 | “Consultation on thermal flexibility in industrial processes” | Relevant: review | Discards |
| F04 | “Operating contract for a heat accumulation solution” | Relevant | Discards |
| F05 | “Scientific article with a preliminary result and no industrial test” | Doubtful | Discards |
| F06 | “Advertisement with no verifiable information” | Irrelevant | Discards |
| F07 | “Date of an already announced contract is corrected” | Relevant: correction | Depends on text |
| F08 | “Incidental mention in an old catalogue” | Irrelevant | Depends on text |
With that data you cannot calculate a real performance figure for the filter, since we have no executed queries and no representative sample. The risk can be observed, though: the literal rule keeps F02 and loses F03 and F04. Redesign it to include entities, synonyms, event stages and a separate rule for corrections. Treat F05 as a review case; do not invent it as a confirmed fact.
Acceptance exercise · Two versions, one error report
Build a set of at least ten records labelled relevant, irrelevant, needs review. Apply rule V1, identify two false positives and two possible omissions, formulate V2 and keep both versions. Write a report of at most 120 words for the decision-maker: what improved, what might be lost and when you will review the rule. Do not simply celebrate that fewer alerts arrive.
Verifiable deliverable: R04 with V1/V2 rules, sample, labels, errors, owner and re-evaluation date. Connection: R04 and R06; Lab L03 “Filter test”. The review of AI-generated claims is expanded in 3.2.
References and limits
- Innguma · internal training material. Definición de un sistema de vigilancia (presentación, 2021-12-01) (2021).Methodological basis of the watch cycle and of critical watch factors.Innguma training material, not public.
- Innguma · internal training material. Construyendo un sistema de vigilancia (presentación, 2022-04-28) (2022).Sources, filtering, management and dissemination.Innguma training material, not public.
- Company A is a fictional case: its data, companies and figures do not describe real markets or organisations.
- Timeframes, frequencies and lists in the plan are adjustable teaching proposals, not mandatory procedures of a standard.
Intelligence Lab · L03
Filter test
Ready to apply it? Open the Filter test in the Lab and find out which signals your rule misses and which ones it picks up unnecessarily. Output: comparison of filtering rules. Works in the browser, no server, no account and no connected AI. The result may need reviewing and adapting to your case.