How do precision and recall apply to alarm thresholds?

Four counts: TP, FP, FN, TN

Each alarm, or each time window, is classed by comparing it with what really happened. A true positive (TP) is an alarm that matches a real event. A false positive (FP) is an alarm with no event behind it. A false negative (FN) is a real event that produced no alarm. A true negative (TN) is a quiet period with no event.

Precision = TP / (TP + FP) looks at the alarms: how far can they be trusted? Recall = TP / (TP + FN) looks at the events: how many were caught? Moving the threshold shifts the balance between them. The F1 score, F1 = 2PR / (P + R), combines the two into one number, but it does not say which error is more expensive.

Hypothetical comparison of two thresholds

Suppose a line has 10 real events in one month. At threshold A the model raises 12 alarms, 8 of them real: TP = 8, FP = 4, FN = 2.

• Precision = 8 / 12 = 0.667
• Recall = 8 / 10 = 0.80
• F1 = 2 × 0.667 × 0.80 / (0.667 + 0.80) ≈ 0.727

At threshold B, which is higher, the model raises 7 alarms, 6 of them real: TP = 6, FP = 1, FN = 4.

• Precision = 6 / 7 ≈ 0.857
• Recall = 6 / 10 = 0.60
• F1 = 2 × 0.857 × 0.60 / (0.857 + 0.60) ≈ 0.706

Precision rises from 0.667 to 0.857, but recall falls from 0.80 to 0.60, and F1 slips from 0.727 to 0.706. Whether B is better depends on what a missed event costs.

Common mistakes

The first mistake is the unit of counting. A single event that keeps a signal above threshold for 30 minutes should count as one event, not as 30 alarms. Sample-level and event-level precision can differ a lot, so state which one you report.

The second mistake is relying on accuracy. When events are rare, a model that never alarms can still score high accuracy. The third is choosing the threshold on the test period. Tune it on a separate validation period and use the test period once, at the end. A matching rule is also needed: how many minutes before or after an event still count as a hit?

Follow-up questions