How to handle overlapping classes

Question

I am working on the classification of a dataset which contains ambiguous and noisy data - the result of which means I have class overlap in the feature space.

There seems to be a few papers on this topic:

• A Classification Scheme for Applications with Ambiguous Data.

• Handling Class Overlap and Imbalance to Detect Prompt Situations in Smart Homes

I am still unable to find a more popular approach with hopefully a solution in python. My question is therefore how do people suggest handling overlapping classes for classification problems?

Answer 1

If overlapping classes means that a single data instances are assigned multiple classes, you basically two options:

• Make the problem a single-class classification by having a separate class for all class combinations in the training data (there might be too many of them, some of them might not make sense because you said the data is noisy)

• Have an independent predictor for each of the classes and treat the problem more as assigning independent tags to each data instance.

If you want to use neural nets, in the latter case, it makes sense to have a shared representation and do the same architecture as if you did standard classification. However, instead of the softmax, you would use just a sigmoid non-linearity (for prediction between 0 and 1) and binary cross-entropy loss function. The target vector is in this case indication vector with ones for the active classes and zeroes elsewhere.

Answer 2

The second paper you referred, describes the methods to handle overlapped instances.

Handling samples of overlapping regions is as important as identifying such regions. Xiong et al. [16] proposed that the overlapping regions can be handled with three different schemes: discarding, merging and separating.

Discarding: Ignores the overlapping region and learns on rest of the data that belongs to the non-overlapping region.

Merging: Considers the overlapping region as a new class and uses a 2-tier classification model.

Separating: The data from overlapping and non-overlapping regions are treated separately to build the learning models.

So as the author in the paper describes, you may ignore the overlapping region, or consider the overlapping region as a new class, or treat overlapped and non overlapped regions separately.

Reference : Handling Class Overlap and Imbalance to Detect Prompt Situations in Smart Homes by Barnan Das, Narayanan C. Krishnan, Diane J. Cook

Answer 3

Let's look at an example where there are regions where one classification is obvious but also a region where there is overlap and ambiguity.

In this picture, the upper right and lower left are clearly dominated by red, while the upper left and lower right are clearly dominated by blue. Thus, if you have a point like $(2, 2)$, the prediction should be a high probability of the red category, while $(2, -2)$ should lead to a high probability of the blue category.

At a point like $(0, 0)$, the category to which the point belongs is not clear, and I would want my model to reflect this. Sure, it is desirable to get confident predictions, but the data in this case do not allow for such confidence. It really is the case that there is ambiguity, and to force a model to predict with confidence is to dismiss reality. Given how I generated this plot (code below), the probability that $(0,0)$ belongs to red is $1/2$, same as the probability that $(0,0)$ belongs to blue. If you force some other probability, you will be in a position to make mistakes.

I would say that, if you have overlapping classes like we do here, the way to proceed is to embrace the fact that there can be ambiguity. For instance, in this example, you really cannot accurately predict the category to which $(0,0)$ belongs.

library(MASS)
library(ggplot2)
set.seed(2023)
N <- 250
X0 <- MASS::mvrnorm(N, c(0, 0), matrix(c(
  1, 0.9, 
  0.9, 1
), 2, 2))
X1 <- MASS::mvrnorm(N, c(0, 0), matrix(c(
  1, -0.9, 
  -0.9, 1
), 2, 2))
d0 <- data.frame(
  x1 = X0[, 1],
  x2 = X0[, 2],
  y = "Category 0"
)
d1 <- data.frame(
  x1 = X1[, 1],
  x2 = X1[, 2],
  y = "Category 1"
)
d <- rbind(d0, d1)
ggplot(d, aes(x = x1, y = x2, col = y)) +
  geom_point()