4

How can we prove that the normalized kernel is a kernel?

That is how can we show

$\frac{K(x,y)}{ \sqrt{( K(x,x) K(y,y) )}}$ is a valid kernel.

Question: Also in real world, why do we normalize the kernels?

Do we also normalize the Gaussian Kernel ?

We can show that multiplication of two kernel is a kernel, but how about a normalized one.

http://ttic.uchicago.edu/~dmcallester/ttic101-07/lectures/kernels/kernels.pdf enter image description here

Ferdi
  • 5,179
BhishanPoudel
  • 215
  • Note that normalising the Gaussian kernel changes nothing as $K(x, x) = 1 \quad \forall x$ – Epimetheus Aug 29 '18 at 12:49
  • Also note that alex-r's answer shows that normalising the kernel is equivalent to replacing the feature map with a normalised version: $\Phi'(x) = \frac{\Phi(x)}{\lVert \Phi(x) \rVert}$ – Epimetheus Aug 29 '18 at 12:51

1 Answers1

9

\begin{align*} K'(x,y)&:=\frac{K(x,y)}{\sqrt{K(x,x)K(y,y)}}\\ &=\frac{\Phi(x)\cdot\Phi(y)}{\|\Phi(x)\|\|\Phi(y)\|}. \end{align*}

The denominator is always non-negative, hence this is a kernel since the numerator can be shown to be positive definite. Also $K'(x,y)=K'(y,x)$ so you're done.

In general, if $K(x,y)$ is a kernel then so is $g(x,y)K(x,y)$ where $g(x,y)$ is non-negative and symmetric.

Alex R.
  • 13,897
  • What if $K(x,x)=0$ or $K(y,y)=0$. It is clear that either implies $K(x,y)=0$ but still, why is this well defined? Do you have a reference for the general remark for $g(x,y)$ non-negative? – math May 15 '22 at 15:53