0

I am doing a personal project to see how well does FIFA potential player stats predict the actual overall stat after 3 years.

  1. Meaning, if a player has a potential of 85 in 2015, how accurate should I expect it to be to predict the overall player score in 2018. Should I use R2 for this? Maybe MAPE?
  2. I also want to check if the histogram of errors (potential_2015 - overall_2018) has a normal distribution. Do I need to use Chi-squared for this?
  3. If the prediction is overestimating the player, I would like to know by how much. Should I use Standard Deviation on the errors?

If you have a list of key statistical tests, I would appreciate if you could list them so I can research and learn.

Thank you,

Xavier
  • 3

1 Answers1

1
  1. Here is a list of point forecast accuracy measures. The entire textbook is very much recommended. Also relevant:
  2. There is a number of normality tests, the most common is the Shapiro-Wilks test. In 15 years of forecasting, I have never seen anyone test normality of forecasting errors, but you may have reasons for doing so. Note that normality tests address a question we already know the answer to: since your stats are (presumably) nonnegative, your errors are bounded, but the normal distribution is unbounded, so the errors can't be normally distributed. A test may still be useful in assessing whether they are "too" non-normal. As above, this is assuming you have a reason to be interested in this.
  3. You can simply calculate the error per player, or if you have multiple forecasts and actuals for a player, take the mean error. The standard deviation (of what?) will not be very useful.
Stephan Kolassa
  • 123,354
  • The book at this link goes heavily into the theory of forecasting and the different methods. I haven't read the whole thing but I remember the parts of it I did read being pretty good. Even though it's econometrically focused, a lot of the material is general. Of course, take a look before you purchase because there's a lot material on the net and Stephen's answer is useful also. https://www.amazon.com/Economic-Forecasting-Graham-Elliott/dp/0691140138/ref=sr_1_5?keywords=econometric+forecasting&qid=1637309187&sr=8-5 – mlofton Nov 19 '21 at 08:10
  • Thank you @Stephan,

    1. After checking your resources and going into the rabbit hole I decided to use MAE or RMSE instead of MAPE, as I don't want to put emphasis on negative errors.

    2. My goal to test the normality of errors is because it should not follow a bell curve, as my assumption is that the forecast has a bias to overestimate players potentials. Graphically, the histogram should be skew right (error: actual - potential).

    I'll use Shapiro-Wilks, thank you.

    1. I found I can use Mean Bias Error.
     – Xavier Nov 22 '21 at 05:12
  • Thanks for the feedback, just one comment: whether the errors are normally distributed or have a bias is not a contradiction, they could be both, e.g., normally distributed with a nonzero mean. It looks like you are mainly interested in whether the forecasts have a nonzero bias. A t-test would be appropriate, without worrying about the normality. – Stephan Kolassa Nov 22 '21 at 06:19