0

I am researching the effect of parameters on energy consumption. To determine the effect of parameters, I want to use the R Studio's F-test. In this way I want to investigate if the model with the parameter (dummy categories) is significantly better than a model without the parameter. Here I run into a problem. My dataset has a very high skewness and kurtosis, resulting in fact that the linear regression assumptions are violated. I tried to solve this with a transformation (10log) of the dependent variable. The skewness is now lower (less than 1), but the kurtosis is still high (around 4, which was 30 without transformation). The assumptions now seem to be a bit better, but there are still fat tails in the qqplot. I was wondering if the test is reliable after the transformation? Or is the F test not affected by violated assumptions and is it possible to make the F-test reliable without transformation? And if the F-test isn't relaible with and without transformations, which other way can I use to test the significance of a parameter.

Thanks in advance:)

These are the regression outputs of the regression on the dependent variable (energy consumption) without transformations

enter image description here enter image description here enter image description here

After transformation (The F-statistic has become significant (at 0.05 and 0.01 now) enter image description here enter image description here enter image description here

Boxplot enter image description here

Siebe
  • 11
  • Welcome to Cross Validated! There is no assumption about normality of $y$. Further, if you transform $y$, you now have a different model, and inferences about the original data need not be legitimate. // This seems like an XY problem where you have problem X (effect of parameters on energy consumption) that you think you can solve using method Y (regression F-test). Then, when you have trouble implementing Y, you ask about Y instead of X. What exactly do you want to do with X? – Dave Jun 09 '22 at 14:03
  • Dear Dave, thank you for your reaction. My goal is to prove that my variables (for example the function type) have a significant effect on the energyconsumption (the dependent variable). So I just want to show that including dummies for the function types (office, sport etc.) will give a better model than a model without these variable (and I tought this could be done by looking at the F-statistic) – Siebe Jun 09 '22 at 20:19
  • And the problem is that I learned to check the assumptions before interpretating the results of the regression. However the normal residuals assumption is violated (I think) because the qqplot is not in line with the normal distribution. And also the residuals vs the fitted values plot is showing some sort of trend. Maybe I can send you pictures about the resisdual plots, but I can not find out how to do that. – Siebe Jun 09 '22 at 20:27
  • Please feel free to edit your original post to include additional information that would be valuable. – Dave Jun 09 '22 at 21:23
  • I added pictures about the residuals plots and the regression output. I thought I wasn't allowed to intepret the regression output when the assumptions were violated. Is that true or can I still say something abou the F-test? Or is there another way to show that including dummies for the function types (office, sport etc.) will give a better model than a model without this variable? – Siebe Jun 10 '22 at 06:57
  • Comment 1: Regarding the comment from @Dave about the transformation. With your updated information we can see that there is no problem here since you are using a model with only dummy variables. There is no modeled curve that becomes disturbed due to the transformation. – Sextus Empiricus Jun 10 '22 at 08:28
  • Comment 2: You have very different variances in the different categories. Or, at least, it appears like this but it might not be because of the different number of data points per category (you could easily verify it by computing the deviation for each class) but it would not be weird if the error in the estimate of the energy consumption increases when the energy consumption is larger. This heterogeneity might be more problematic than whether or not the data is conditional normal distributed. (Look for ANOVA with unequal variances) – Sextus Empiricus Jun 10 '22 at 08:33
  • Comment 3: Your quantile plot is a lot skewed because of the effect from the previous comment. You should show a plot of residuals, for a single category and after transformation. – Sextus Empiricus Jun 10 '22 at 08:43
  • Okay thank you! I have indeed very different amounts of datapoints for each category. Do you think it is justified now to say that the H0 (the model with type of building dummies is not a better model than a model without these parameters) can not be rejected at the 0.01 and 0.05 significance level (because the F-statistic is lower than the critical value) based on this dataset. Or do I need to fransform the dependent variable first? (I've added the outputs and residuals plots after the transformation too) – Siebe Jun 10 '22 at 08:47
  • Sorry, but I don't know how to show a plot of the residuals for a single category? – Siebe Jun 10 '22 at 08:51
  • Can you also show boxplots of the various object types vs. the transformed y? – Christian Hennig Jun 10 '22 at 08:59
  • Yes, I've added the boxplot – Siebe Jun 10 '22 at 09:07

1 Answers1

1

Model assumptions are mathematical constructs and are never precisely fulfilled in reality. The relevant question is not whether assumptions are fulfilled or not (as they never are), but rather whether assumptions are violated in critical ways, i.e., ways that lead to misleading conclusions. Unfortunately assessing this is rather subtle. Roughly said, in your situation, extreme outliers and strong skewness are very problematic, as would be dependence between observations. Heterogeneity of variances can be a problem but has often rather mild influence. Data with untransformed $Y$ seem very problematic in this respect, so you were right to worry. Data with transformed $Y$ don't look perfect but much better.

I'd personally think that the model with transformed $Y$ is good enough to interpret the F-test despite model violations still being visible (but as I said before, the model is never true anyway). As I said, these things are subtle, so here are some considerations that back up this opinion (that is admittedly to some extent subjective, so if you publish it, reviewers may disagree):

  1. The p-value of the F-test is not borderline but small enough that it cannot be explained by rather mild assumption violations only.

  2. The effect of too large kurtosis is usually that there is loss of power, meaning that significances are still meaningful (but harder to obtain).

  3. There seems to be a mild heteroscedasticity issue, but I don't think it has much of an impact.

  4. Boxplots show fairly convincingly (at reasonably large sample size) that Kantoor values tend to be higher by and large.

But keep in mind that $R^2$ is very low at 7%, so your model doesn't explain much of the variation of the transformed $Y$.

I've written more generally about model assumptions here: Relevance of assumption of normality, ways to check and reading recommendations for non-statisticians

Christian Hennig
  • 23,655
  • Thank you very much for this explanation. When I include extra variables, the r squared goed up to about 20% so I know this isn't much and I will point this out in my report. The thing I wasn't sure about is wheter I can interpret the F-statistic and the R squared, when the assumptions are violated. Now I know it's better to use the tranformed dependent variables, because the assumptions are less violated than in the model without transformation. Last thing I am wondering now, is wheter I can intepret the F-statistic and the R squared when I transform my dependent variable? – Siebe Jun 10 '22 at 10:11
  • Because on internet and in these comments, I see different answers regarding this question. My dependent variable has totally different values now and I am not sure if I can still point out the significance (or unsignificance) of explanatory variables when the dependent variable is transformed to a totally different scale – Siebe Jun 10 '22 at 10:13
  • @Siebe You need to interpret results with transformed variables making reference to the transformation. So you can say for example (see my elaboration in the answer) that there's evidence that the Kantoor object type tends to produce, on average, larger values of log(y) if log was the transformation you have used. – Christian Hennig Jun 10 '22 at 10:16
  • Okay clear, but does that also say something about the Kantoor object type tending to produce on average larger values of y? Because otherwise this isn't very usefull if I want to argue wheter to include object type in a model to predict the actual (not transformed) energyconsumption, or is it? – Siebe Jun 10 '22 at 10:20
  • @Siebe This is another subtle issue. log is a monotonic transformation, so if values of log(y) tend to be larger, roughly said, also values of y tend to be larger. However, speaking of "on average", this does not necessarily hold, as outliers, kurtosis and skewness mean that the act of averaging in itself is problematic. Averages are critically affected by outliers. So I'd say if we're talking about what is going on on average (as such models do), the transformed scale is the more appropriate one to comment on. – Christian Hennig Jun 10 '22 at 10:23
  • Great, it's clear to me now. Thank you (and everyone else) very much for your help! – Siebe Jun 10 '22 at 11:00
  • One last thing. If I only want to comment on the significance (and not on the fact that log(y) becomes higher or lower), is it than still necessary to say: the type of building is significant for the log of the energy consumption or is it justified to say that the type of building is then also significant for the energy consumption? – Siebe Jun 10 '22 at 11:44
  • @Siebe Understanding what goes on and communication of it are two different matters, and the latter depends on who you are communicating with, in what circumstances etc. Talking with non-statisticians I wouldn't use the word "significant" in this way in the first place, and I'd certainly talk of logs where I use logs for analysis, however it is not up to me to license your way of communicating with your colleagues (or whoever). – Christian Hennig Jun 10 '22 at 15:59
  • Logically, if the building type makes a difference for log(y), it also makes a difference for y. The thing is that the kind of difference may not be the same. – Christian Hennig Jun 10 '22 at 16:07
  • Dear Christian, I understand and I am sorry. I just discovered this platform when I was running into questions with my regression. I am a student and just starting in this field so I wasn't aware of any inappropriate communication, but I understand your point. I am sorry and I will rethink my sentences in the future. Thank you for helping me despite my communication:) – Siebe Jun 10 '22 at 16:32
  • @Siebe No need to worry, your questions are generally sensible. – Christian Hennig Jun 10 '22 at 16:58