I was wondering if there is any particular reason why Cohen's Kappa is defined as this particular ration $\frac{p - r}{1 - r}$, where $p$ is the agreement rate between two, say, classifiers, and $r$ is the rate of agreeing at random.
Why is Kappa not defined, for instance, simply as $\frac{p}{r}$? Is the present form only for the upper limit for Kappa to be $1$?
$$ \kappa = \frac{p_o - p_c}{1 - p_c} $$
This formula does have a deeper meaning and also restricts the range between -1 and 1 (not 0 and 1). Kappa is meant to be the ratio of observed non-chance agreement to possible non-chance agreement.
The numerator ($p_o - p_c$) is the observed non-chance agreement because $p_o$ is the total observed agreement and we are removing the estimated chance agreement $p_c$.
The denominator ($1 - p_c$) is the possible non-chance agreement because $1$ is the maximum value that $p_o$ could take on and we are removing the estimated chance agreement $p_c$.
I think that the ratio of observed agreement to chance agreement ($p_o/p_c$) would be reasonable; it just wouldn't be bounded.
Cohen Kappa is defined so $\kappa$=0 means a random agreement, and $\kappa$=1 is the maximum possible Kappa, that is 100% accuracy.
As far as I know there is no deeper principle behind the formula - just the requirement described (0 -> random, 1 -> maximum). The wikipedia page onb Cohen's kappa list a few other inter-rater agreement measures and links to some papers that describe problems with the measure.
