0

I understood that KNN (K-Nearest-Neighbors) was non-parametric, after reading the beginning of the wikipedia article here:

In pattern recognition, the $k$-Nearest Neighbors algorithm (or $k$-NN for short) is a non-parametric method used for...

But, then later in the article it talks of estimating the "parameters"??

The best choice of $k$ depends upon the data; generally, larger values of $k$ reduce the effect of noise on the classification,[5] but make boundaries between classes less distinct.

Am I missing the difference between parameters and hyperparameters? Thanks.

Gilles
  • 1,032
makansij
  • 2,279
  • 9
  • 31
  • 42
  • 2
    There are a number of similar questions on the distinction between parametric and non-parametric. (1) http://stats.stackexchange.com/questions/46588/why-are-gaussian-process-models-called-non-parametric/46621#46621 (2) http://stats.stackexchange.com/questions/103833/are-non-parametric-methods-well-defined (3) http://stats.stackexchange.com/questions/185909/why-is-kernel-density-estimation-still-nonparametric-with-parametrized-kernel/185941#185941 – Sycorax Dec 30 '15 at 17:35
  • 2
    short answer: non-parametric can been interpreted as unfixed number of parameters (or all possible solutions cannot be defined by a finite number of parameters) . So because the final number of parameters is determined by the data, it can be considered non-parametric. – Cliff AB Dec 30 '15 at 17:55
  • Thanks. I'm interested in this for the context of machine learning, (random forest, kNN, SVM, etc.) So, two clarifications: fixed for what - the function that your ML algorithm produces for predicting new labels/target variables? – makansij Dec 30 '15 at 22:36
  • 2
    @Candic3: the function that is produced is a function of various parameters. The question is: before seeing your data, do you know how many parameters are necessary to describe this function (or moreover, the model that the function uses to produce new predictions)? – Cliff AB Dec 31 '15 at 00:28

0 Answers0