0

Let $X$ be an independent variable and $Y$ the dependent variable. Suppose we have the relationship $Y = f(X) + \epsilon$ for some unknown function $f(x)$ and some noise $\epsilon \sim N(0,1)$. If $f(x)$ is highly nonlinear, then we know that the performance of linear regression is going to be poor.

However it might be possible that we can perform a nonlinear transformation $X\mapsto g(X)$ on $X$ such that the independent variable becomes linear. For example, if $f(x)=2^x$ then we can let $g(x)=\log_2 x$.

The question I have is, for any given function $f(x)$, does there exist a function $g(x)$ such that $f(g(x)) = kx$ for some constant $k$? (i.e. $g(x)$ is “almost a right inverse of $f$”) Is there any criteria for such function to exist? (for example $f$ being bijective is a sufficient condition) I think this is the same as $f$ having a right inverse because if we have $f(g(x)) = kx$, then we can replace $x$ with $x/k$ and we have $f(g(x/k))=x$ and hence $f^{-1}(x)=g(x/k)$.

More generally, given a set samples $(X_1,Y_1),\cdots, (X_n,Y_n)$, how do we know if we should do a nonlinear transformation (and if so what kind of nonlinear transformation to do)?

I tried to google about it but couldn’t find much useful information. Any help/ book recommendations/ reference is appreciated, thanks in advance.

Borcherdsbb
  • 1
  • Welcome to CV. The preliminary mathematical questions are straightforward and you're doing a good job addressing them yourself. Concerning the statistical issue of transforming samples, I have answered that at https://stats.stackexchange.com/questions/35711. Certainly there are more ways to do it and many more considerations generally, but why don't you start there and use that information to edit your post and focus it on some aspect of your own problem that might not yet be addressed here. – whuber May 16 '23 at 15:56
  • I'm a little confused by your notation. If $y=f(x)+\epsilon$ and $f(x) = 2^x$, the transformation we want to make to x is $g(x)=2^x$ (whence $k=1$), not $g(x)=\log_2 (x)$. – John Madden May 16 '23 at 16:52
  • @whuber thank you so much for your help! – Borcherdsbb May 16 '23 at 18:50
  • The basic criteria for existence of an inverse function still apply (suitably modified where necessary). If the function is not monotonic (either increasing or decreasing), you'll generally have issues where the inverse could be any of several values (inverting a many-to-one function leaves you with a one-to-many) -- how do you choose? ... there can sometimes be other issues, but I expect you'll want to assume those away. – Glen_b May 17 '23 at 00:06

0 Answers0