From activity to usefulness
Counting alerts, newsletters or visits can help manage workload, but it does not demonstrate strategic impact. The value of intelligence depends on a chain: need → product → recipient → use → decision → review. Each step requires observation: a briefing that has been sent has not necessarily been read; one that has been read does not imply that a decision changed; and a modified decision does not by itself prove that the product caused the result.
It is worth combining three kinds of indicators:
- Operational health: active and failed sources, coverage of priority topics, delivery times, review delays.
- Quality and usage: traceability of products, substantive corrections, recipients who consult/rate, questions resolved or pending.
- Influence on decisions: documented decisions where the product was considered, alternatives discussed, actions triggered or dropped, subsequent learning.
Metrics must be defined with a denominator, periodicity and owner; otherwise they can be used to tell incompatible stories.
Three measurement traps
Trap 1: volume = value. A month with more alerts may indicate wider coverage or more noise. Evaluating it requires looking at relevance and effort.
Trap 2: excessive attribution. That management approves a project after reading a report does not mean the report is its only cause. Recording a documented contribution is more defensible than announcing «X euros generated by monitoring» with no methodology.
Trap 3: optimising for the indicator. If we reward the number of newsletters, the team may publish more instead of synthesising better. A useful measure must come with a question: «which operational decision will we take when this indicator changes?».
Interactive · Decide
Which trap does each sentence fall into?
Identify the measurement trap.
Activity: identify the measurement trap in each statement.
“This month we sent 40% more alerts: the system works better.”
“Monitoring generated 2 million euros: management approved the project after reading the report.”
“Since we started rewarding the number of newsletters, we publish twice as many.”
“The briefing caused the decision to invest.”
Control question for every metric: what operational decision will we take when this indicator changes?
Minimum measurement table
| Dimension | Illustrative indicator | How it is obtained | Which decision it enables |
|---|---|---|---|
| Coverage | Priorities with reviewed sources / active priorities | Coverage audit | Review sources or reduce scope |
| Quality | Products with verifiable evidence / reviewed | Editorial sampling | Strengthen validation |
| Timeliness | Median days between signal and dissemination | Workflow dates | Redesign escalation |
| Usage | Products with useful feedback / sent | Confirmations and interviews | Change format or audience |
| Learning | Reviews that lead to a documented improvement | Change log | Update the process |
We do not set universal targets. An improvement in speed that degrades quality is not necessarily progress; the indicators must be read together.
Company A case · simulation
Fictional caseCompany A case · Did the briefing help?
Company A’s fictional report did not trigger an immediate investment. However, it helped identify a small technical test, avoid an exaggerated claim and set criteria for moving forward. A responsible evaluation records that the briefing was discussed, which alternatives it changed and what will need to be checked after the test. It does not invent economic savings.
Educational simulation: fictional information and organisations.
Do it yourself
Do it yourself · Design three actionable indicators
Choose a monitoring priority. Create one operational indicator, one quality indicator and one usage/decision indicator. For each, add a definition, source, frequency, owner, risk of misinterpretation and the associated action if it worsens. Write one sentence on what that indicator does not allow you to conclude.
Deliverable: measurement sheet linked to improvement decisions.
Maturity assessment and 90-day plan
Identify verifiable improvements and assign them a short cycle.
See the template →Intelligence product specification and briefing
Define the need and deliver actionable conclusions.
See the template →Check it · closed answer
Calculate and read the indicators
State the denominator and avoid causality.
8 out of 12.
See the answer
Answer: 66.7%. 8/12 = 66.7%. State the denominator when you communicate it.
5 out of 12.
See the answer
Answer: 41.7%. 5/12 = 41.7%.
3 out of 12.
See the answer
Answer: 25%. 3/12 = 25%. None of these figures is ROI or causal success.
And now, your own answer
This part needs professional judgement: there is no automatic marking. Complete your work in the practice, rate it with the lesson's rubric and compare it with the indicative solution.
What your answer should include
- One operational indicator, one quality indicator and one usage or decision indicator.
- Each with a definition, source, frequency, owner and action if it worsens.
- One sentence on what it does not allow you to conclude.
Indicative solution
Coverage of priorities with audited sources (operational), products with verifiable evidence over a reviewed sample (quality) and briefings discussed with documented feedback over those delivered (usage).
Illustrative solution for a fictional case: reasoned alternatives are valid.
Mini reproducible table · The denominator matters
Entirely fictional data for one month of Company A: 12 products delivered, 8 with documented review, 5 with feedback received from their recipients and 3 cited in a discussion of alternatives. These are practice counts, not client results.
| Indicator | Formula | Example result | What it does not prove |
|---|---|---|---|
| Documented review | 8 / 12 deliveries | 66.7% | That all findings are true |
| Feedback received | 5 / 12 deliveries | 41.7% | That recipients changed decisions |
| Explicit use in discussion | 3 / 12 deliveries | 25% | That the product caused the decision |
Before repeating these figures, define «delivered», «reviewed» and «used» and check that the counts correspond to the same period. If the team delivers fewer products but reviews them better, a drop in volume is not necessarily a deterioration. If false positives are reduced, the number of alerts may fall while quality rises.
Usefulness test: interview two recipients and ask which decision they faced, which part of the product they used, what was missing and what they would do differently. Record negative answers too. Do not attribute causality or financial return without an additional evidence design.
Interactive · Calculator
The denominator matters
Change the fictional Company A counts. Each percentage comes with what it does NOT prove.
Indicator calculator with numerator and denominator.
References and limits
This lesson cites no external sources. It develops the Academy method from Innguma's training material.
- Company A is a fictional case: its data, companies and figures do not describe real markets or organisations.
- The indicators and rubrics are didactic instruments, not measured results of Innguma or its clients.
Library · R16
Maturity assessment and 90-day plan
Putting it into practice? Download template R16 · Maturity assessment and 90-day plan and fill it in with your case.