1

[update] It seems I had my question's original title misfocused a bit; concerning the use of partial derivatives I found some explanation/confirmation in wikipedia's partial derivative There it is stated -and explained with a 3-d-picture of a surface-, that indeed the three partial derivatives suffice to find the/a common extremum. In the occurrence of my problem I seem to have had a misleading intuition, based on the observation of non-independence of the x and x-square-data and felt thus cautious (too much, apparently).
Full exposition of my proceeding/reasoning and source of question is here

I adapted the subject of my question; if this seems inappropriate please feel free to roll it back. The original question follws below [end update]

I'm just trying to reinvent the wheel - I want to understand (and implement for an example) the computation of the polynomial regression. For the simplest nonlinear approach let's use the estimated model

$$\hat y_k=a+b x_k + c x_k^2 \qquad \text{(for k=1 to n)}$$

with the minimizing criterion

$$\sum (y_k - \hat y_k)^2 =\sum (y_k - (a+bx_k+cx_k^2))^2= \text{Min}$$

I have expressed all the occurrng and needed variances and covariances in a matrix, cofactored with $a,a^2,b,b^2,c \text{ and }c^2$ . To find the minimum I must use the derivatives; however I'm not much familiar with partial derivatives.

So my question is: do I get the global minimum if I solve my matrix-expression setting the partial derivatives resp to $a,b,c$ separately (and find the then possible common solution)? Or do I need to involve some "joint derivative" or "common derivative" where, for instance something like $\partial ab$, thus a and b, are simultanously used in one derivative?

whuber
  • 322,774
Gottfried Helms
  • 1,594
  • 2
    The answers to this question should shed some light. Note that polynomial regression is only a special case of multiple linear regression. – mpiktas Aug 04 '11 at 19:00
  • @mpiktas: thanks, I took a look at it. Well, I know that the interaction terms and terms of higher exponents are often simply used as additional terms in a multiple linear regression, but I've not yet understood the legitimation for this other than that it's practical. I've just put my thoughts together in that sketch: http://go.helms-net.de/stat/div/nonlinregr1.htm The result with some experimental data was not correct - maybe, the whole ansatz is useless... but then I'd like to understand (and possibly improve) that problem – Gottfried Helms Aug 04 '11 at 20:28
  • 3
    This is a three variable optimization problem; the variables are $a$, $b$, and $c$. The objective function is quadratic in those variables, so you don't even need calculus: you can use elementary methods to find the minimum if you like. But it's easiest to characterize the unique global minimum as the point where the gradient $(\partial/\partial a, \partial/\partial b, \partial/\partial c)$ vanishes. (Consider the one-dimensional analog: if you want to minimize $\alpha a^2 + \beta a + \gamma$ with respect to $a$, which derivatives would you take?) – whuber Aug 04 '11 at 20:42
  • @whuber: yes, that's the matter with the partial derivatives. I was/am unsure, whether it is ok to just use them separately. With my experimental data I had a simple typo and after correction the method gave now a very well solution - modulo that it is finally correct at all (didn't check concurring methods, for instance Excel/Spss, so far). – Gottfried Helms Aug 04 '11 at 20:52

0 Answers0