- True-positive rate (TPR)
- False-positive rate (FPR)
- The area under the ROC curve (AUC)
- Gini coefficient
An ROC curve shows how much a model is capable of distinguishing between classes. It is a graph showing the performance of a classification model at various classification thresholds, ranging from 0 to 1.
- True Positive (TP, positive prediction that was actually positive)
- True Negative (TN, negative prediction that was actually negative)
- False Positive (FP, positive prediction that was actually negative)
- False Negative (FN, negative prediction that was actually positive)
- TPR – The true positive rate also known as sensitivity, is calculated as TP/TP+FN. TPR is the probability that an actual positive is predicted as positive by the model.
- FPR – The false positive rate is calculated as FP/FP+TN. FPR is the probability that an actual negative is predicted as positive by the model.
The ROC curve shows the tradeoff between sensitivity (or TPR) and specificity (1 – FPR). Typically, a lower decision threshold identifies more positive cases, because you set a lower bar to classify an observation as positive. However, as you classify more observations as positive due to lenient threshold, you might misclassify more negative cases as positive as well. A better classifier makes fewer tradeoffs to catch more of both classes correctly. A ROC plot illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied.
AUC stands for "Area under the ROC Curve." That is, AUC measures the entire two-dimensional area underneath the entire ROC curve from (0,0) to (1,1).
AUC provides an aggregate measure of performance across classification thresholds.
An AUC of 1 indicates a perfect classifier, an AUC of 0 indicates a classifier that always predicts the opposite of the actual class, and an AUC of 0.5 indicates a classifier that performs as good as random guessing.