1

I have one numeric score S1 to (ascending) sort a dataset. Now I have developed another method that returns a numeric score S2 that should sort the dataset in similar way. My question: with what methods can I evaluate the sorted results to compare performance of S2 to the "ground truth" S1?

If S1 returns ABCDEFG and S2 returns ADEFGBC , how can I compare the result of this permutation to the ground truth?

mdewey
  • 17,806
El Dude
  • 211

1 Answers1

0

I would suggest metrics from information retrieval which are comparing ordered list of search results to assess the quality of a search engine. Try looking at MRR (Mean reciprocal rank) or Spearman rank correlation (but if not mistaken, the later needs a score associated to each ordered item).

gdupont
  • 223
  • 1
    Thanks, I'll into it. I also had the idea to simply count the number of required swaps to get the original list back. but Spearman rank makes sense, too. I would use the original sorted score vector and correlate it to the new score vector. – El Dude Feb 05 '15 at 18:12