2

So say I have 30 teams and I want to schedule their games with the objective getting the most information concerning who's better than who as quick as I can. How do I do this. Let's say we have a ranking system like elo. After every game we pick the next pair. How do we quantify the uncertainty in our system and how do we then pick the pairing that will most reduce this uncertainty.

I'm totally out of my depth here and can't even figure out what terms to search for.

colwem
  • 31

1 Answers1

2

If you want to most reduce uncertainty: play a round robin. Then play another one. Then another one. And so on.

If you want to rank as quick as you can in a tournament, use swiss-system tournement.

If you're interested in quantifying the uncertainty, check ratings reliability in Glicko system, or in its nephew TrueSkill.

If you're interested in fullfilling the objective getting the most information concerning who's better than who, choose a ranking system, like most known Elo, or aforementioned Glicko or TrueSkill, or rankade, our free to use ranking system (here's a comparison), use it, and play a lot.

Edit after comments:

  • Opposite to Elo and Glicko, rankade can manage multiplayer matches, so if you want just giving them each two items asking them which is higher and which is lower, you can ask users to compare 3+ items (it's difficult to compare 10+ things, but doing this for something like 3-7 items should be easy and useful), getting the most useful information before the user gets bored and leaves.

  • You can build something easy for creating next matchup. If all items have to be checked same number of times, just minimize delta for matchups (see aformentioned swiss system). If number of matches should be different for items, try something like minimizing

$$ m = (n_p)^a \cdot { \left [ n_1 + n_2 \over 2 \right ]^b} \cdot \Delta_p $$

where $n_p$ is the number of matches played for chosen pair, $n_1$ and $n_2$ are the numbers of matches played for two chosen item, $a$ and $b$ are custom exponents, and $\Delta_p$ is difference in their ranking score.

Tomaso Neri
  • 776
  • 5
  • 18
  • Thanks, that was a lot of good information. Though I'm still not sure how to solve my problem. Let me explain in more detail exactly what I want to do. In general when trying to rank a list of something it's much easier to rank the top few and the bottom few than to rank the middle bunch. This isn't so surprising since that's what you would expect if the criteria you're ranking is normally distributed. But what I would like to do is make a program that multiple users can use to jointly create a ranking of a list. – colwem May 02 '16 at 00:06
  • It will just give them each two items ask them which is higher and which is lower. The idea though is that no one is actually going to sit through all n/2 * (n-1) "games". So I need to chose which matches to ask that will improve the accuracy of the list the most so that we get the most useful information before the user gets bored and leaves. – colwem May 02 '16 at 00:06
  • Intuitively this mean that we'll quickly figure out that we don't need to ask about the 1st vs the 10th or even the 1st vs the 2nd anymore because that's pretty settled. But we're going to need to ask about the 10th vs the 11th many many times because the answers will vary greatly and there will be a lot of uncertainty about which is better. But as we get more responses it will start to become clearer. – colwem May 02 '16 at 00:06
  • Another complicating issue is that there may legitimately be cycles like A is better than B is better than C is better than A. This happens in sports all the time due to matchup effects. So we don't just want to ask 10 vs 11 a bunch of times but also 10 vs 12 and 9 and 8 and 7 etc. So that we can capture that "graph". So the question is how do I quantify which matchup to present next in order to best decrease this uncertainty? – colwem May 02 '16 at 00:06
  • I edited my answer after your comments. – Tomaso Neri May 10 '16 at 08:42