## Definition The **Receiver Operating Characteristic (ROC) curve** plots the **True Positive Rate** (recall) against the **False Positive Rate** ($\text{FPR} = FP / (FP + TN)$) across all possible decision thresholds. **AUC (Area Under the Curve)** summarises the entire curve in a single number. ## Construction A probabilistic classifier outputs scores. Sweep the threshold from very high to very low: - At high thresholds: few positive predictions → low TPR, low FPR. - At low thresholds: many positive predictions → high TPR, high FPR. Each threshold gives one point on the curve. The curve traces from $(0,0)$ to $(1,1)$. ## What AUC Means $ \text{AUC} = P(\hat y(\text{positive}) > \hat y(\text{negative})) $ The probability that a randomly chosen positive is ranked higher than a randomly chosen negative. - **AUC = 0.5** → no discrimination (random guessing). - **AUC = 1.0** → perfect ranking. - **AUC < 0.5** → worse than random (often a sign of an inverted prediction). ## When ROC-AUC Shines - **Threshold-independent comparison.** Captures the ranking quality without committing to an operating point. - **Balanced classes.** Works well when the positive and negative classes are similar in size. - **Calibration-agnostic.** Only ranking matters; scale of probabilities is irrelevant. ## When ROC-AUC Misleads - **Highly imbalanced classes.** ROC plots TPR vs FPR; FPR can stay small even when many positives are missed if negatives are very numerous. **AUC-PR (Precision-Recall AUC)** is usually more informative here. - **Different decision regimes.** ROC-AUC averages over all thresholds; in practice you operate at one. Care about that region. - **Comparing on different test sets.** AUC depends on label distribution; not always comparable across datasets. ## Multi-class Extension - **One-vs-rest AUC** — compute binary AUC per class against all others; average. - **One-vs-one AUC** — average AUC over all class pairs. - **Hand-Till** — generalisation to multi-class. ## Related - [[Confusion Matrix]] - [[Precision and Recall]] - [[F1 Score]]