A binary classifier has a ROC-AUC of 0.5. What does that mean?
Options
- Its accuracy is always exactly 50%
- Half of its predictions are wrong
- Its probabilities are perfectly calibrated
- It ranks examples no better than chance: a random positive scores above a random negative half of the time (Correct option)
Answer
It ranks examples no better than chance: a random positive scores above a random negative half of the time
Why
ROC-AUC is the probability that the model gives a higher score to a random positive than to a random negative. 0.5 is a coin flip, 1.0 is a perfect ranking, and below 0.5 the ranking is inverted.
Accuracy depends on the threshold and the class balance, so an AUC of 0.5 does not fix it at 50%.
AUC says nothing about calibration.
Remember
AUC 0.5 = random ranking.