Questions tagged [bias-variance-tradeoff]

In predictive modeling, unbiased models can have higher variance, & thus be less accurate. Modelers may prefer some bias to maximize accuracy. Use this tag also for questions about the bias-variance decomposition.

The bias-variance tradeoff is a fundamental issue in predictive modeling. Estimators / fitting algorithms for models that are unbiased (i.e., that have sampling distributions that are asymptotically centered on the true values) can have higher variance (i.e., be further from the true value in any given instance), and thus be less accurate. Modelers often prefer models that are somewhat biased so as to maximize accuracy.

257 questions
2
votes
1 answer

Bias-variance decomposition of squared error

In the equation for expected test error we are summing the function's bias, variance and noise. I am not quite sure why we are also summing the variance of the function. My intuition says that more flexible functions, which have higher variance, can…
Druudik
  • 143
1
vote
0 answers

Bias-Variance Tradeoff, computing bias theoretically

Bias, in machine learning, is mathematically defined as $f-E(\hat{f})$, where $f$ is the true model and $\hat{f}$ is the estimate. I was wondering how we can compute theoretically $E(\hat{f})$, given some data points $\{ (x_i,y_i)\}$. For a simple…
cgo
  • 9,107
1
vote
0 answers

How do you detect variance and bias problems?

In practice, given a dataset how do I assess that I have variance or bias problem such that I can move my efforts towards reducing them by using proper models for each case (for example: RandomForests for decreasing variance and GradientBoosting for…
Krushe
  • 51
1
vote
0 answers

what's in a name: Bias?

I'm studying machine learning and one of the key points I make sure is to understand the origin and the reasoning why such a name or term is used. It could be a rich and insightful dive into the concept it is trying to convey. So, about…
cgo
  • 9,107
1
vote
1 answer

Is variance the only cause of overfitting in any Machine Learning Algorithm?

I am used to associating high variance with overfitting - I don't know how else to think of the physical manifestation of variance wrt an algorithm. Also, the only cause of overfitting seems to be high variance. Is there a causal relation between…
rahs
  • 633
0
votes
0 answers

How do bias-effect and variance-effect reduce to bias squared and variance?

I'm currently reading James1997 - Generalizations of the Bias/Variance Decomposition for Prediction error. My ultimate goal is to see how, in the special case of squared loss, bias-effect and variance-effect reduce to the commonly known bias…
ngmir
  • 341