Metrics show how well a model makes predictions. Without them you can’t tell whether the model is working.
Confusion Matrix
The foundation of all metrics. For binary classification:
Predicted
Positive Negative
Actual
Positive TP FN
Negative FP TN
- TP โ correctly identified positives
- TN โ correctly identified negatives
- FP โ false alarm (negative predicted as positive)
- FN โ miss (positive not found)
def confusion_matrix(predictions, actuals):
"""Compute TP/TN/FP/FN."""
tp = sum(1 for p, a in zip(predictions, actuals) if p == 1 and a == 1)
tn = sum(1 for p, a in zip(predictions, actuals) if p == 0 and a == 0)
fp = sum(1 for p, a in zip(predictions, actuals) if p == 1 and a == 0)
fn = sum(1 for p, a in zip(predictions, actuals) if p == 0 and a == 1)
return {"TP": tp, "TN": tn, "FP": fp, "FN": fn}
preds = [1, 0, 1, 1, 0, 1, 0, 0, 1, 1]
actuals = [1, 0, 0, 1, 0, 1, 1, 0, 1, 0]
print(confusion_matrix(preds, actuals))
# {'TP': 4, 'TN': 3, 'FP': 2, 'FN': 1}
Precision, Recall, F1
def precision(tp, fp):
"""Of all "positive" predictions, how many are correct?"""
return tp / (tp + fp) if tp + fp else 0
def recall(tp, fn):
"""Of all real positives, how many did we find?"""
return tp / (tp + fn) if tp + fn else 0
def f1_score(prec, rec):
"""Harmonic mean of Precision and Recall."""
return 2 * prec * rec / (prec + rec) if prec + rec else 0
# Example
tp, fp, fn = 80, 20, 20
p = precision(tp, fp) # 0.80
r = recall(tp, fn) # 0.80
f = f1_score(p, r) # 0.80
print(f"Precision: {p:.2%}, Recall: {r:.2%}, F1: {f:.2%}")
Precision vs Recall โ trade-off: a strict model gives high Precision (few FP) but low Recall (many FN). A lenient model is the opposite. F1 balances both.
Choosing a Metric
| Task | Metric | Reason |
|---|---|---|
| Spam filter | Precision | Can’t delete legitimate emails |
| Disease detection | Recall | Can’t miss a diagnosis |
| Fraud detection | Recall | Catch every case |
| Recommendations | F1-Score | Balance precision and coverage |
| Balanced classes | Accuracy | Classes are even |
Practical Example
m = confusion_matrix(preds, actuals)
p = precision(m["TP"], m["FP"])
r = recall(m["TP"], m["FN"])
f = f1_score(p, r)
total = sum(m.values())
acc = (m["TP"] + m["TN"]) / total
print(f"Accuracy: {acc:.2%}")
print(f"Precision: {p:.2%}")
print(f"Recall: {r:.2%}")
print(f"F1-Score: {f:.2%}")
Common Mistakes
Accuracy on imbalanced classes โ if 95% of examples are negative, a model that always says “no” gets 95% Accuracy but 0% Recall. Use F1.
One metric for all tasks โ choose the metric that fits the context: medicine โ Recall, spam โ Precision.
๐ฌ Comments (0)
No comments yet
Be the first to share your opinion about this article!