2

Given a set of 128x128 images from three classes, I obtained an accuracy of 50% with a SVM on the flattened images (16384 'features').

Is this an upper bound on the performance of a SVM using any features extracted from the images?

Christian
  • 193
  • 1
    Accuracy is a bounded metric and the upper bound is 100%. But it does not seem to be what you mean, so what is it? – Tim May 23 '23 at 13:43
  • Related: How to know that your machine learning problem is hopeless? – Stephan Kolassa May 23 '23 at 13:48
  • Whether it's possible to accurately predict the target from the input features is impossible to say from this level of detail - the target may be defined by a single input feature, or it may be completely unrelated to all of them. If the target value is a coin flip that has no relationship to the images, you of course can't do any better than 50% accuracy. If you're trying to tell apart pictures of cats and dogs, on the other hand, there's no reason accuracy should be limited to 50%. – Nuclear Hoagie May 23 '23 at 13:48
  • ... especially if the dataset is 90% cats and 10% dogs. – Stephan Kolassa May 23 '23 at 13:49
  • @Tim Let me rephrase: If the classifier reaches a certain accuracy by using all the available information (the raw pixel data), is it even possible to improve this by using any feature extraction methods on the data? – Christian May 24 '23 at 09:42
  • @NuclearHoagie 50% is already better than chance for three classes. My question is whether this value can be improved although the classifier is already using all available information. – Christian May 24 '23 at 09:45
  • @Christian your follow-up question can be re-phrased to "if one algorithm gave accuracy X% does this mean that any algorithm will give a performance not higher than this" and the answer is obviously no, different algorithms (including different hyperparameters, ways of preprocessing the data, etc) can perform differently. – Tim May 24 '23 at 09:50
  • @Tim I want to use the same classification algorithm, my question is only concerned with feature extraction. For example, can the SVM in principle perform better on features extracted by a CNN etc.? – Christian May 24 '23 at 10:09
  • @Christian different algorithms would perform differently for different datasets, see https://en.wikipedia.org/wiki/No_free_lunch_theorem "Algorithm" is here everything together: how the features are created, the hyperparameters, the ML algorithm used, etc. – Tim May 24 '23 at 10:11

1 Answers1

0

Is this an upper bound on the performance of a SVM using any features extracted from the images?

I SAY NO, at least not necessarily.

From the comments:

If the classifier reaches a certain accuracy by using all the available information (the raw pixel data), is it even possible to improve this by using any feature extraction methods on the data?

I SAY YES

I can imagine two scenarios.

  1. You overfit to all of the pixels. Reducing the feature space dimension through some feature extraction technique leads to a simpler model with less opportunity to overfit, possibly leading to improved out-of-sample performance.

  2. You use your domain knowledge to extract useful features that an SVM might struggle to figure out on its own, and these features provide improved ability to distinguish between the categories.

Dave
  • 62,186