Imagine a fairly standard Thumbs Up/Down feedback system.
- For each piece of content they consume, every user has the ability to give a binary positive/negative feedback that piece of content.
- They're not required to give feedback, and they can't give any more detail than pos/neg.
- The content creators want to see the overall opinion of each piece of content, but that feedback must be anonymised, and condensed to a very short numerical summary. 1 figure, maybe 2.
So far, so standard.
Obviously if every user had to give feedback on every piece of content, then just the taking the ratio of positive to negative votes would be a reasonably accurate representation of the feedback.
However, the fact that users can choose whether or not to give feedback at all adds new information into the system, that isn't reflected in the ratio.
It would be reasonable to assert that a +1 vote from someone who only votes on 1% of content is more significant that the same +1 from someone who votes on 95% of content.
Similarly, the bias in a user's vote is another source of information: a -1 vote from someone who gives upvotes on 90% of content they vote on, seems like a much bigger deal than a -1 vote who gives about 50:50 up and down votes.
It seems like it ought to be possible for the system to apply a numerical weighting to a user's vote, given their previous voting history, so that "unusual" votes have more impact on the final normalised score.
Is this a known problem, with an established solution?
If so, what are the search keywords I should be looking for?
If not, does anyone want to make concrete suggestions?
