10

Can a linear SVM support more than 2 classes for classification?

john
  • 223
  • 1
    An addition to the good answers of user20160 & MightyCurious: there is also the approach of Crammer-Singer which casts the multi-class SVM into one single optimization problem. Also: most well-written libraries for SVM (e.g. Liblinear, LibSVM) already support some of these methods out-of-the box (OvR and OvO should be supported everywhere, Crammer-Singer is sometimes supported and often not as practical as the former approaches). – sascha May 30 '16 at 10:18

2 Answers2

10

SVMs (linear or otherwise) inherently do binary classification. However, there are various procedures for extending them to multiclass problems. The most common methods involve transforming the problem into a set of binary classification problems, by one of two strategies:

  1. One vs. the rest. For $k$ classes, $k$ binary classifiers are trained. Each determines whether an example belongs to its 'own' class versus any other class. The classifier with the largest output is taken to be the class of the example.
  2. One vs. one. A binary classifier is trained for each pair of classes. A voting procedure is used to combine the outputs.

More sophisticated methods exist. Search for "multiclass SVM".

user20160
  • 32,439
  • 3
  • 76
  • 112
3

Yes, support vector machines were originally designed to only support two-class-problems. That is not only true for linear SVMs, but for support vector machines in general. There are ways to work around this, but they usually come as a kind of afterthought. There is an ongoing discussion about the merit of these approaches. If you would like to delve into this discussion, this paper may serve as a starting point.

MightyCurious
  • 506