25

What are the hallmarks or properties that indicate that a certain learning problem can be tackled using support vector machines?

In other words, what is it that, when you see a learning problem, makes you go "oh I should definitely use SVMs for this" rather than neural networks or decision trees or anything else?

Ethan
  • 1,633
  • 9
  • 24
  • 39
Ragnar
  • 511
  • 1
  • 5
  • 16
  • Removed the supervised learning tag, as SVM's can also be used in unsupervised learning problems. – Dawny33 Jan 11 '16 at 07:48
  • Can explain how svm can be used for unsupervised problem and which package implements it? – GeorgeOfTheRF Jan 12 '16 at 05:37
  • @ML_Pro Please go through the link I have included in my comment. – Dawny33 Jan 12 '16 at 05:59
  • @ML_Pro He is referring to Support Vector Clustering – Ragnar Jan 12 '16 at 06:01
  • 4
    @Dawny33, the application of SVMs to unsupervised learning is the exception and not the rule. SVMs are a supervised learning method. – AN6U5 Jan 12 '16 at 06:54
  • 1
    @AN6U5 Thanks for informing :) By exception, you mean that it is just a tweak and not the convention, right? – Dawny33 Jan 12 '16 at 08:40
  • Yes, that's what I mean. The supervised learning label is very much appropriate for a post on SVM and the unsupervised learning label is an outlier (I guess we could data mine 1000 publications and test this rigorously :-). Thanks for adding it back and thanks for your contributions to the site! – AN6U5 Jan 12 '16 at 14:49

2 Answers2

12

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non linear problems.

Until 2006 they were the best general purpose algorithm for machine learning. I was trying to find a paper that compared many implementations of the most known algorithms: svm, neural nets, trees, etc. I couldn't find it sorry (you will have to believe me, bad thing). In the paper the algorithm that got the best performance was svm, with the library libsvm.

In 2006 Hinton came up with deep learning and neural nets. He improved the current state of the art by at least 30%, which is a huge advancement. However deep learning only get good performance for huge training sets. If you have a small training set I would suggest to use svm.

Furthermore you can find here a useful infographic about when to use different machine learning algorithms by scikit-learn. However, to the best of my knowledge there is no agreement among the scientific community about if a problem has X,Y and Z features then it's better to use svm. I would suggest to try different methods. Also, please don't forget that svm or neural nets is just a method to compute a model. It is very important as well the features you use.

hoaphumanoid
  • 901
  • 8
  • 19
  • 1
    @HoapHumaboid I know about SVC and SVR and that we can use Kernels to apply SVMs to NLPs and I'm aware of papers that compare SVMs with other learning algorithms...I was wondering about the characteristics of a prospective learning problem that would make it evident that it should be tackled with an SVM. In other words, what is it that when you see a learning problem makes you go "oh I should definitely use SVMs for this'' rather than NNs or Decision trees or anything else – Ragnar Jan 12 '16 at 02:59
  • @Ragnar please see my edited response – hoaphumanoid Jan 12 '16 at 09:26
5

Let's assume that we are in a classification setting.

For svm feature engineering is cornerstone:

  • the sets have to be linearly separable. Otherwise the data needs to be transformed (eg using Kernels). This is not done by the algo itself and might blow out the number of features.
  • I would say that svm performance suffers as we increase the number of dimensions faster than other methodologies (tree ensemble). This is due to the constrained optimization problem that backs svms. Sometimes feature reduction is feasible, sometimes not and this is when we can't really pave the way for an effective use of svm
  • svm will likely struggle with a dataset where the number of features is much larger than the number of observations. This, again, can be understood by looking at the constrained optimizatiom problem.
  • categorical variables are not handled out of the box by the svm algorithm.
pincopallino
  • 73
  • 5