1

I'm starting to learn a bit about gradient descent, and that method attempts to minimize the cost function. I've seen examples using linear regression, and the corresponding cost function, but now I'm wondering, given an arbitrary function $g(x)$ which computes an output $y$, will the cost function to use for gradient decent always be:

$$y_{k} - g(x_{k})$$

resulting in the gradient descent of

$$y_{k+1} = y_{k} - {d \over dx} g(x_{k})$$

atomsmasher
  • 330
  • 2
  • 14

1 Answers1

0

Commonly used cost functions are:

  • Squared error cost function
  • Negative log likelihood

Regardless of what cost function you use the gradient descent rule is always applicable. The gradient is normally calculated analytically, which depends on the cost function used.

Newstein
  • 418