Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

"Regression" is a general term for a wide variety of techniques to analyze the relationship between one (or more) dependent variables and independent variables. Typically the dependent variables are modeled with probability distributions whose parameters are assumed to vary (deterministically) with the independent variables.

Ordinary least squares (OLS) regression affords a simple example in which the expectation of one dependent variable is assumed to depend linearly on the independent variables. The unknown coefficients in the assumed linear function are estimated by choosing values for them that minimize the sum of squared differences between the values of the dependent variable and the corresponding fitted values.

29789 questions
71
votes
8 answers

Regression with multiple dependent variables?

Is it possible to have a (multiple) regression equation with two or more dependent variables? Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it would capture any relationship between the two DVs?
Jeff
  • 3,927
28
votes
4 answers

Difference between regression analysis and analysis of variance?

I am learning right now about regression analysis and the analysis of variance. In regression analysis you have one variable fixed and you want to know how the variable goes with the other variable. In analysis of variance you want to know for…
Le Max
  • 3,729
24
votes
2 answers

Minimal number of points for a linear regression

What would be a "reasonable" minimal number of observations to look for a trend over time with a linear regression? what about fitting a quadratic model? I work with composite indices of inequality in health (SII, RII), and have only 4 waves of the…
Francoise
  • 241
23
votes
2 answers

Linear regression, conditional expectations and expected values

Okay so just a bit hazy on a few things, any help would be much appreciated. It is my understanding that the linear regression model is predicted via a conditional expectation $$E(Y|X)=b+Xb+e$$ Do we assume that both $X$ and $Y$ are random…
William Carulli
  • 333
  • 1
  • 3
  • 6
21
votes
2 answers

Extreme learning machine: what's it all about?

I've been thinking about, implementing and using the Extreme Learning Machine (ELM) paradigm for more than a year now, and the longer I do, the more I doubt that it is really a good thing. My opinion, however, seems to be in contrast with scientific…
davidhigh
  • 1,410
  • 9
  • 20
17
votes
2 answers

Formula for weighted simple linear regression

This wiki page Simple linear regression has formulas to calculate $\alpha$ and $\beta$. Could anyone tell me how to derive the formulas in weighted case?
Wei Shi
  • 315
15
votes
3 answers

What is the difference between $\beta_1$ and $\hat{\beta}_1$?

Suppose I have a random sample $\lbrace x_n ,y_n \rbrace_{n=1}^N$. Suppose $$y_n = \beta_0 + \beta_1 x_n + \varepsilon_n$$ and $$\hat{y}_n = \hat{\beta}_0 +\hat{\beta}_1 x_n$$ What is the difference between $\beta_1$ and $\hat{\beta}_1$?
Stan Shunpike
  • 4,333
14
votes
2 answers

More predictors than observations?

What does it mean when statisticians talk about having more predictors than observations in a regression model? How could that even be possible? Why is it a problem in regression? Apologies, I am new to quant analysis and stats so not quite sure why…
user3424836
  • 177
14
votes
5 answers

Why divide by $n-2$ for residual standard errors

I was just watching a lecture on statistics and someone was calculating something called the residual standard error. It looked a lot like finding the average of the square of the residuals, the residuals being the difference between the prediction…
sebastianspiegel
  • 259
13
votes
1 answer

Linear Regression with a Dependent Variable that is a Ratio

I'm doing linear regressions where the dependent variable is a ratio that can range from 0.01 to 100. Is it ok to take the log of the dependent variable and the regression on that? I'm matching the results of a study and that is what they did. What…
Aaron Kreider
  • 231
  • 1
  • 2
  • 4
13
votes
2 answers

Why use linear regression instead of average y per x

Concretely, if we're interested in predicting house price (dollars) from house size (square meters), we can calculate the best fitting line and use that for predicting new values. But why not simply calculate the average price per square meter and…
user153009
13
votes
5 answers

Why study Linear Regression?

Given two random variables $\xi$ and $\eta$ we can compute their "correlation coefficient" $c$, and form the line of best fit between these two random variables. My question is why? 1) There are random variables, $\xi$ and $\eta$ which are…
Nicolas Bourbaki
  • 2,859
12
votes
1 answer

Linear Regression + confounder

Suppose I'd love to access the effect size and significance between outcome Y and variable X adjusted by confounder Z. My question is that if there is any difference to determine the effect size and significance of X between following scenario. put…
WCMC
  • 1,058
12
votes
1 answer

Regressing out a variable?

What exactly does it mean to "regress out" a variable or quantity? This is something I hear about all the time, but it's strangely difficult to find a precise description online. Any pointers - especially to algorithmic descriptions - would be much…
HansSchwabing
  • 123
11
votes
1 answer

Is a Longitudinal Regression a type of Markov Chain?

I have always wondered about this point : Longitudinal Regression models for repeated measures essentially allow for previous response measurements for the same patient to influence future response measurements for the same patient. In a very naive…
stats_noob
  • 1
  • 3
  • 32
  • 105
1
2 3
…
73 74