ViLiQ

Probability and base rates

Statistics · lesson 1

26 minute read60 VILIQ Points

By the end
Apply base rates correctly and recognise when a compelling story is overriding the underlying frequency.

  • Quant +16
  • Risk +5

A base rate is how often something happens across all comparable cases. It is the single most useful and most neglected number in forecasting, because a specific compelling story feels more informative than a general frequency and almost never is.

Example — The classic failure

A test for a condition affecting 1 in 1,000 people is 99% accurate. Someone tests positive. Most people estimate the probability they have the condition at around 99%. The correct answer is about 9%. In 10,000 people there are 10 true cases and roughly 100 false positives, so a positive result is a true case about one time in eleven.

Applied to markets: a pattern that "predicted the last three crashes" needs the denominator. How many times did the pattern occur without a crash? A signal firing 40 times to catch 3 events is not a crash predictor, however impressive the three hits look when displayed alone.

Common belief

"This indicator called the last four major turns."

What is actually true

Presented without the false positive count, this is unfalsifiable. The relevant figures are how often it fired, how often it was right, and how it compares to the base rate of the event. Almost no popular indicator is presented with those three numbers, and the omission is the finding.

This is why VILIQ stores every prediction with its outcome definition fixed in advance, and reports calibration with the sample size attached. A hit rate without a denominator is not a measurement, and a model that only shows its hits cannot be evaluated at all.

Glossary

Base rate
How often something occurs across all comparable cases.
False positive
A signal firing when the event does not occur.
Denominator
The total number of cases. Without it a count of successes means nothing.

Check your understanding

0 of 3 answered

Pass mark 70%: at least 3 of 3 correct.

  1. 1.A condition affects 1 in 1,000. A 99% accurate test returns positive. Roughly what is the probability the person has it?
  2. 2.An indicator "called the last three crashes". What is the missing information?
  3. 3.Why is a highly accurate signal for a rare event still mostly false positives?

Challenge — Evaluate a signal properly

A signal fires 30 times over ten years. Recessions began 4 times. The signal preceded 3 of them and fired 27 times without one. Compute its precision and recall, compare with the base rate, and state whether it is useful and for what.

What a good answer contains

  • Computes precision (3/30) and recall (3/4) correctly
  • Compares against the base rate of the event
  • Reaches a defensible conclusion about usefulness that accounts for both figures

Sign in to submit a challenge. Your answer is reviewed and counts towards your skill scores.

Sign in

Put it to work

Read a market with what you just learned, then practise with simulated money. No real order is ever placed.

VILIQ provides market intelligence, research and educational information. It is not financial product advice and does not take your personal circumstances into account. Consider your own situation and seek licensed advice before making financial decisions.

VILIQ provides market intelligence, research and educational information. It is not financial product advice and does not take your personal circumstances into account. Consider your own situation and seek licensed advice before making financial decisions.

VILIQ is operated by LTM Trading Pty Limited (ACN 659 211 426), Australia.