Can a ROC curve be partly above and partly below the diagonal?

Question

A ROC curve has particular meaning about how the sensitivity and specificity change as the classification threshold changes for two groups of data. The x- and y-axes both range from zero to one, and the curve is weakly increasing.

We want our ROC curves to be above the $45$-degree line, but terrible performance can result in a curve below this diagonal (though some calibration might fix such an issue).

However, can a ROC curve exist that is partially above the diagonal and partly below? What conditions would the data have to satisfy for this to happen? I imagine something like the shape below.

library(ggplot2)
x <- seq(0, 1, 0.01)
y <- pbeta(x, 9, 9)
d <- data.frame(
  FPR = x,
  TPR = y,
  Curve = "ROC"
)
ggplot(d, aes(x = FPR, y = TPR, col = Curve)) +
  geom_line(linewidth = 2) +
  geom_abline(slope = 1, intercept = 0) +
  theme(legend.position = "none")

Answer 1

Multi-modality does the trick!

Consider model predictions that are, for whatever reason, something like this.

In such a situation, the ROC curve has the sigmoid-style shape given in the question, with half of the curve below the diagonal and half above.

Consequently, yes, there are model predictions that can yield such a curve.

library(pROC)
library(ggplot2)
N <- 10000
p0 <- rbeta(N, 1/2, 1/2)
p1 <- rbeta(N, 2, 2)
y0 <- rep(0, N)
y1 <- rep(1, N)
y <- c(y0, y1)
p <- c(p0, p1)
r <- pROC::roc(y, p)
d1 <- data.frame(
  p = p,
  Group = as.factor(y)
)
d2 <- data.frame(
  FPR = 1 - r$specificities,
  TPR = r$sensitivities,
  curve = "ROC"
)
Plot the KDE

ggplot(d1, aes(x = p, fill = Group)) +
  geom_density(alpha = 0.2) +
  xlab("Model Prediction") +
  ylab("")
Plot the ROC curve

ggplot(d2, aes(x = FPR, y = TPR, col = curve)) +
  geom_line(linewidth = 2) +
  geom_abline(slope = 1, intercept = 0) +
  theme(legend.position = "none")