8

Introduction

For some time now I have been struggling to understand how theoretical results can be applied in practice. Fortunately in most cases the link between theory and practice is not hard to find, for instance:

  • You can directly use a theoretical result for your calculations.
  • You cannot actually find a solution, but you at least have an upper or lower bound that can indicate how good your real solution is.
  • By observing the theoretical formulas you can make something 'similar' and hopefully with similar performance
  • The general case is only theoretical, but in special cases it can be applied directly.

Well, this is not everything, but I won't try to make a complete list of how statistical theory can be useful. Until now I have always succeeded in at least grasping the idea of how theory can be applied in practice.

However, it now appears that I have stumbled opon something of which I cannot see how it could possibly be applied.

I was studying some smoothing methods, amongst which (nonparametric) kernel smoothing, as described on Wikipedia , and found that there is a theoretical solution for the AMISE, as well as the bandwidth,hAMISE that optimizes it. However, both in the notes that I was reading, as well as the Wikipedia page I have been unable to find any practical application of this.

The Question

Is there any practical application for the AMISE and its optimal bandwidth?

Discovering that there actually is an application for would be very motivating so I hope it can be achieved!

Danica
  • 24,685
Dennis Jaheruddin
  • 456
  • 1
    Regarding the interpretation of AMISE, you have to understand first the "expected $L_2$ risk function". There are many distances that you could use instead of this, but MISE is the most common in the context of KDE. This is a helpful criterion given that it leads to an interpretable and formal choice of the bandwidth. There are several applications of this, for instance see this answer in the context of reliability. –  Nov 26 '12 at 16:30
  • @Procrastinator Thank you for your thoughts, I understand that MISE is a measure to define which estimation is "better". However, this does not yet make it clear to me what the implications are of a theoretically optimal, yet always unknown AMISE. – Dennis Jaheruddin Nov 27 '12 at 09:20
  • 1
    My last two cents: I think one of the uses of the MISE and AMISE is for comparing the performance of different kernels in terms of efficiency. See Comparison of Data-Driven Bandwidth Selectors. –  Nov 27 '12 at 09:34

1 Answers1

11

The AMISE allows one to obtain an expression for the optimal bandwidth for the unknown density $f$. Unfortunately, the expression is in terms of derivatives of $f$. However, it is possible to derive a similar expression giving the optimal bandwidth for a kernel estimate of those derivatives. This is expressed in terms of even higher derivatives of $f$. And so on.

This seems like it might be an unending sequence of pointless theory. But the neat thing is, that for some sufficiently high order derivatives you can just assume that $f$ is normal. Then you can work your way back through the levels to find the bandwidth for $f$. It turns out that this works really well and almost nothing is lost provided $f$ is sufficiently smooth and enough levels of iteration are used (usually only 2 or 3 levels are needed).

The practical result is a bandwidth selection method which is general and quite robust. The most popular version of this is the Sheather-Jones plug-in method which is implemented in several software packages. In R, you can get a density estimate using the Sheather-Jones method:

density(x, bw="SJ")

That usually gives better results than the default bandwidth.

Rob Hyndman
  • 56,782
  • (+1) Nice explanation, particularly the conditions for obtaining a good performance of the Sheather-Jones method. –  Nov 30 '12 at 13:13