For an experiment where I simulated n=10000 games of chess with random moves I obtained the following results:
White won = 1674 Black won = 1721 Draw = 6605
Now I am interested in testing whether white has an unfair first-move advantage or not. Ofcourse in classic chess the answer is yes, but for random moves it seems not so simple.The first thing I wonder is, can I simply ignore the draws? It reduces my number of samples quite a bit but it's not relevant for this particular problem I think.
If I ignore the draws, then $n=1674+1721=3395$ and $\hat{p}=1674/3395=0.4931$. Then we have Bernoulli experiments where $x$ is the number of games won by white, and $\hat{p}=x/n$. Furthermore I define $\alpha=0.05$ and my hypothesis test:
$H_0: p=0.5$ and $H_0: p \neq 0.5$
I take a two sided test because I would also like to know if Black has an advantage even though that is not the problem statement. Since $n$ is large and we are far from $p=0$ or $p=1$ we can use the normal approximation to the binomial distribution. We then have:
$\sigma_x = \sqrt{np_0(1-p_0)} = \sqrt{3395\cdot 0.5 \cdot 0.5} = 29.13$
The z-score of our statistic then becomes:
$z_0 = \frac{x - np_0}{\sqrt{np_0(1-p_0)}} = \frac{1674 - 3395\cdot 0.5}{\sqrt{3395\cdot 0.5 \cdot 0.5}} = -0.8066 $
Which has a p-value of about 0.42 which is higher than our significance $\alpha=0.05$ and therefore we accept the null hypothesis. However, this number seems very high. I think intuitively this is because the sample probability lies so close to $p_0$. However I have repeated the simulation quite a few times and I consistently get that black has a higher percentage of winning than white. So did I make a mistake somewhere?
I input the same data and I again get a p-value of 0.21. So I guess it's correct.
– heyjude123 Feb 22 '23 at 17:22