A receiver operating characteristic (ROC) curve shows the performance of a binary classification model as its discrimination threshold varies. For a range of thresholds, the curve plots the true positive rate against the false positive rate.
The Receiver Operating Characteristic (ROC) function takes a set of prediction-actual pairs for a binary classification model and calculates the following values for a range of discrimination thresholds:
- True positive rate (TPR)
- False positive rate (FPR)
- Area under the ROC curve (AUC)
- Gini coefficient
A prediction-actual pair for a binary classifier consists of:
- Predicted probability that an observation is in the positive class
- Actual class of the observation
A discrimination threshold determines whether an observation is classified as positive (1) or negative (0). For example, suppose that a model predicts that an observation will be classified as positive with 0.55 probability. If the threshold above which an observation is classified as positive is 0.5, then the observation is classified as positive. If the threshold is 0.6, the observation is classified as negative.
You can create prediction-actual pairs for ROC with these functions:
- AdaBoostPredict (ML Engine)
- DecisionForestPredict_MLE (ML Engine)
- DecisionForestPredictPerSegment (ML Engine)
- DecisionTreePredict_MLE (ML Engine)
- GLMPredict_MLE (ML Engine)
- GLML1L2Predict (ML Engine)
- GLMPredictPerSegment (ML Engine)
- KNN (ML Engine)
- NaiveBayesPredict_MLE (ML Engine)
- NaiveBayesTextClassifierPredict_MLE (ML Engine)
- SVMDensePredict (ML Engine)
- SVMSparsePredict_MLE (ML Engine)
- XGBoostPredict (ML Engine)