Evaluation Metrics

Accuracy alone is misleading. On a 99% negative dataset, a model that always predicts negative gets 99% accuracy but detects nothing. Choose your metric based on the cost of each error type.

Confusion Matrix: TP (true positive), TN, FP, FN Precision = TP / (TP + FP) — Of all predicted positives, how many are actually positive? Recall (Sensitivity) = TP / (TP + FN) — Of all actual positives, how many did we catch? F1 Score = 2 · (Precision · Recall) / (Precision + Recall) — Harmonic mean, balances both. AUC-ROC — Area under the ROC curve. 0.5 = random, 1.0 = perfect. Threshold-independent.

Spam detection → Prioritize Precision (don't block legitimate emails) Cancer screening → Prioritize Recall (don't miss any cancer cases) Fraud detection → Balance both with F1