3

Assume there are a total of 1000000 contracts. Assume there are 120 bad contracts in total.

Assume that company 1 issues 500000/1000000 contracts and 70 are deemed bad. Assume that company 2 issues 100000/1000000 contracts and 50 are deemed bad

Assume there are lots of companies.

Clearly from the example above, company 1 is better than company 2, it issued 5 times as many contracts but only 20 more contracts are bad.

How do I go about ranking the companies based on who's most reliable at issuing contracts?

Greg
  • 31
  • 1
    You can rank on number of bad contracts or on proportion of bad contracts. Statistical advice won't contradict what will be evident: you get to decide what is interesting or useful, but there is a warning that the interpretation of analyses will be different. – Nick Cox Nov 24 '15 at 12:37

1 Answers1

2

You can simply calculate proportion of bad contracts,

$$ P = {n_b \over n}, $$

so that $P_1 = 1.4 \cdot 10^{-5}$ and $P_2 = 5 \cdot 10^{-4}$, but it's better to add a correction for total number of contracts issued. One way to correct is to add "dummy" bad contracts,

$$ P' = {n_b +10 \over n}. $$

Companies with a large number of contracts see their modified percentage alter very little from their real percentage ($P_1' = 1.6 \cdot 10^{-5}$ and $P_2' = 6 \cdot 10^{-4}$), but companies with relatively few contracts will see their modified percentage move considerably toward high values ($n_b =3$,$n= 30000$, so $P_3=1 \cdot 10^{-4}$, and $P_3'= 4.3 \cdot 10^{-4}$).

This is known as "Bayesian averaging". In effect, the companies with many contracts will rank higher than those companies with the same percentage but fewer contracts.

Tomaso Neri
  • 776
  • 5
  • 18