The Matérn covariance function converges to the squared exponential covariance function.
Many sources, amongst them the GPML book and Wikipedia, state this result. None of them provide details.
I am looking for references that provide details and a proof. In particular, I wonder about the kind of convergence.
The Matérn function can be written in terms of
$$f_{\nu}(x) = C_\nu |x|^{\nu} K_{\nu}\left(|x|\right)\tag{*}$$
where $C_\nu$ is a normalizing constant (to make the value of $f_\nu(0)$ equal to $1$) and $x = \sqrt{2\nu}\, d/\rho.$
(This agrees with the Wikipedia notation where $x$ represents $\sqrt{2\nu} d/\rho.$)
As shown at Moment generating function of the inner product of two gaussian random vectors (using elementary techniques),
the Matern function is proportional to the density function for the distribution of the dot product of two random vectors where each has $2\nu+1$ components and all components are independently distributed as standard Normal variables.
Such an inner product is the sum of the $2\nu+1$ independent and identically distributed products of corresponding components of the vectors. Each of those is the product of two independent standard Normal variables $X$ and $Y$ and therefore has mean $0$ and variance
$$\operatorname{Var}(XY) = E[(XY)^2] = E[X^2]E[Y^2] = (1)(1) = 1.$$
Consequently the inner product has mean $(2\nu+1)(0) = 0$ and variance $(2\nu+1)(1)=2\nu+1.$
The Central Limit Theorem asserts that the normalized versions of these inner products therefore approach a standard Normal distribution almost surely. The effect of normalization is to replace $x$ by the square root of its variance, $x\sqrt{2\nu+1},$ which changes the probability element $f_{\nu}(x)\mathrm{d}x$ by
$$f_{\nu}(x\sqrt{2\nu+1})\mathrm{d}(x\sqrt{2\nu+1}) = \sqrt{2\nu+1} f_{\nu}(x\sqrt{2\nu+1})\mathrm{d}x.$$
This differs from $(*)$ (where we may take $\rho=1$ without any loss of generality, because it merely establishes the distance unit of measurement) only insofar as $x$ is multiplied by $\sqrt{2\nu+1}$ instead of $\sqrt{2\nu}.$ Since the ratio of these terms approaches unity, in the limit it makes no difference which one is used. Consequently, the convergence is almost sure.
One tiny nicety is that because $f_\nu$ is normalized to have a peak height of $1,$ which is $\sqrt{2\pi}$ times the peak height of the standard Normal density, the convergence is to $\sqrt{2\pi}$ times the standard Normal density rather than the density itself. Re-introducing the scale factor $\rho$, we have deduced--using purely statistical thinking!--that
$$\lim_{\nu\to\infty} f_\nu(d) = \exp\left(-\frac{d^2}{2\rho^2}\right)$$ almost surely.
This agrees with what Wikipedia asserts.
This plot shows graphs of $f_2$ (blue), $f_5$ (red), and the limiting Gaussian (gold). The convergence occurs by pulling the tail in to fill out the peak.
