3

I have trouble understanding the delineation of "non-parametric" methods. It seems to me most non-parametric methods are parametric, just on a different space than the "standard ones".

Graph-based inferences are often described as non-parametric, but graphs are fully described by the set of adjacency matrix parameter values. Likewise rankings can be considered a transformation of values onto a different parameter space. Non-parametric seems to get tossed around whenever something is parametric in a high dimensional way on a space we don't typically use.

If the definition is a question of the use of a distribution, coming from a Bayesian background, I guess my intuition is often that usually when we don't put a distribution on something we're often neglecting to capture its uncertainty, rather than something intrinsic to the method.

Hence there seem to be Bayesian formulations of "non-parametric" methods which involve parameter distributions, suggesting that a lack of a distribution isn't inherent to the representation or method, but more of a modeling decision.

kjetil b halvorsen
  • 77,844
user4733
  • 2,714
  • 3
    Non-parametric $\neq$ no parameters. – Bitwise Jun 18 '14 at 18:02
  • 1
    I wouldn't say the problem with the term "nonparametric" isn't so much that it's ill-defined as it is variously defined. Some people use it to mean infinite-parametric, others distribution-free (though in many contexts the two are related), some in other ways. It's even possible to encounter more than one sense of the word in the same context (as when applied to both the assumptions about the relationship between y and x and to the assumptions about the distribution of y. – Glen_b Jun 19 '14 at 00:12

2 Answers2

1

Frequentists understand that non-parametric methods entail non-parametric distributions, and use these (or approximations to them) in inference.

I think "I guess my intuition is often that usually when we don't put a distribution on something we're often neglecting to capture its uncertainty" is interesting. My take on the use of non-parametric methods is that they are generalizing from more strict (More certain? What do you mean by "uncertain"?) inferences about the processes driving sample distributions. For example, in inferring differences in paired data, we might like to compare mean differences (paired t test), but if we cannot ascribe normality to the sample distribution of mean differences, we might instead settle for a more general statement than mean difference, and make inference about the stochastic dominance of one group versus another instead (sign test or signed-rank test).

So we haven't describe the continuous distribution driving our continuous data, and instead sacrificed greater discrimination for more generally applicable inference based on distributions of ranks.

Alexis
  • 29,850
1

The short answer is yes: nonparametric statistical methods allow the data to determine the complexity of the model.

Frequentists (usually) do this by not completely specifying the likelihood function, and Bayesians do this through expanding the prior with infinitely many parameters. Here is a not so gentle comparison of the two from a Bayesian perspective while trying to answer basically your question. I highly recommend reading it to get a better theoretical notion. The whole class series of presentations might interest you.

  • A Bayesian can also avoid specifying the likelihood using a "limited-information" approach. Placing a prior on infinite-dimensional spaces (such as spaces of real functions or spaces of distributions) is a quite delicate issue. – guy Jun 19 '14 at 01:12
  • @guy Those methods are not fully Bayes but yes they do exist: "quasi-Bayes". –  Jun 19 '14 at 01:19
  • It seems to me that they often approximate fully-Bayes inference from the perspective of someone who is given only a particular insufficient statistic (e.g. the fully Bayes inference I would do if my secretary deleted my data except for some summaries). From this standpoint, the Bayesian justification of my analysis may be stronger here than if I used a full model and (inevitably) wrote down the wrong likelihood. – guy Jun 19 '14 at 01:51
  • @guy that's a valid point and I 100% agree with you.. in fact you could make the argument (and Dunson does so in the lecture I linked) that parametric Bayesian methods may not even be fully Bayesian in the sense that they are too inflexible, hence a justification for nonparametric Bayesian tools (even quasi-Bayes if one is so inclined) –  Jun 19 '14 at 02:00