Consider this minimal example:
\documentclass{beamer}
\usefonttheme[onlymath]{serif}
\begin{document}
$fft$
\end{document}
With PDFLaTeX, it turns out as expected:
If I compile with XeLaTeX, however, the kerning is very different:
If I use the article class, it is always the first variant. If I use beamer with the lmodern package in XeLaTeX (which I am not supposed to do, as I understand), it also turns out right again.
How can this different behavior be explained, and what is the correct way to make sure that I always get the first variant? I am using Debian stretch with TeX Live package version 2016.20170123-5.
(Note: I know that I should use \mathit to name multi-letter variables.)
Weirdly, on an older installation with Debian jessie and TeX Live 2014.20141024-2+deb8u1, also get the second variant with PDFLaTeX and lmodern, but I do not get the first variant with XeLaTeX and lmodern.

