Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen?

Question

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen? If so, to what dimension does it transform the data?

Answer 1

If you are referring to the kernel as a kernel in machine-learning literature, then yes, it is a kernel.

More generally, we can consider the family of Gaussian kernels, parametrized by $\sigma$:

$$K(x,x') = e^{x^Tx'/\sigma^2 }$$

Using the power series expansion of the function exponential, we can rewrite the expression of $K$ as:

$$ K(x, x') = \sum_{n=0}^{\infty} \frac{(x^T \cdot x' )^n}{\sigma^{2n}n!} $$

Recall kernels are closed under summation, even infinite sums. $K$ then is sum of other (polynomial) kernels , thus is still kernel. The polynomial kernel, $(x\cdot x')^d$ can be shown to map $x$ to monomials of degree $d$. Thus the Gaussian kernel maps $x$ to all the polynomial kernels.

Example: In two dimensions, $x = (x_1,x_2), \; x' = (x_1', x_2')$, the secord order polynomial kernel $(x \cdot x')^2$ maps the data to look like a new inner product:

$$( x_1^2, x_2^2, x_1 x_2 ) \cdot (x_1'^2, x_2'^2, x_1' x_2')$$ The polynomial kernel is actually computing the above, which looks like it mapped $(x_1, x_2)$ to all monomials of degree 2. The Gaussian kernel does the same thing but for degree 1, degree 2, degree 3...

Side Note

Often in ML literature, the Gaussian kernel is defined as

$$K'(x,x') = \exp{\Big( \frac{||x - x'||^2}{2\sigma^2} \Big) }$$

But this is actually the normalized Gaussian kernel. A normalized kernel, $K'$, is defined:

$$ K'(x,x') = \frac{K(x,x')}{\sqrt{ K(x,x) K(x',x') } } $$

If we use $K(x,x') = \exp{\Big( \frac{x^Tx'}{\sigma^2 } \Big) }$, we get:

$$ K'(x,x') = \frac{e^{x^Tx'/\sigma^2 }}{ \exp{ \Big(\frac{||x||^2}{2\sigma^2} \Big)} \exp{\Big(\frac{||x'||^2}{2\sigma^2}\Big) } } $$

$$ = \exp{ \Big( -\frac{||x' - x||^2}{2\sigma^2} \Big) }$$