1

I have a a set of films, and for each films, a set of reviews - varying between 1 review and several hundred reviews for each film. Each review has a star rating from 1 to 5.

I am using Wilson's confidence interval for a Bernoulli parameter to estimate whether the film is likely to be good or not, taking into account the number of ratings (I just count any 3+-star reviews as positive, anything else as negative).

However, I'd also like to figure out how likely to be divisive a film is, given the number of ratings.

So a film with 200 reviews - 100 1-star reviews and 100 5-star reviews - is more likely to be divisive than a film with 2 reviews = 1 1-star review and 1 5-star review. However, both films clearly have the same standard deviation of ratings.

I don't think I can use the same Wilson's confidence interval calculation that I'm using for 'goodness', since the ratings aren't Bernoulli in nature (EDIT: I'm assuming they are normal).

Does anyone have any ideas on how to measure 'divisiveness' in this way?

kjetil b halvorsen
  • 77,844
Richard
  • 31
  • Could you please tell us what you mean by "divisive"? Re the edit: because on the face of it no distribution of just five values can reasonably be termed "normal," what exactly are you assuming? Perhaps that they are binned values of some unobserved underlying normal distribution? – whuber Nov 30 '12 at 17:33
  • 1
    Divisive: a film that everyone rates at three stars is not divisive, but a film that half of people rate as 1 star and half of people rate as 5 star is divisive. (Basically, the larger the standard deviation, the more divisive the film.) – Richard Nov 30 '12 at 18:27
  • 1
    Re the edit: okay, sorry if I am wrong. Basically, they are not binomial, so I need to find a different way to look at the problem. – Richard Nov 30 '12 at 18:28
  • Would this be a fair interpretation? "Given a set of reviews (assumed to be a random sample of all film viewers), what is the probability that the population standard deviation exceeds some particular fixed threshold chosen as a marker of 'divisiveness'?" – whuber Nov 30 '12 at 20:09
  • I'm looking for the particular individual films that are most clearly divisive. So... I'm looking for a way to rank them. If that interpretation can help me do that, then yes. – Richard Nov 30 '12 at 21:05
  • Now that you put it that way, Richard, it sounds like your question has been asked quite a few times before. Why don't you review some of the previous threads to see whether a good answer appears there? If not, come back and edit your question to reflect the way(s) in which it asks something new. Thanks. – whuber Nov 30 '12 at 22:18
  • 2
    People have asked questions about how to find the best or worst items based on star rating. Unless I'm missing something though, I don't think they have asked about how to find the most divisive items, though, which is why I started a new question. Perhaps I'm wrong, in which case please do point me to the relevant question. – Richard Dec 01 '12 at 17:17

1 Answers1

1

What you calls divisiveness seems analogous to what in ecology (and other fields) is called diversity, as in biodiversity. There are many possible measures, much used ones is entropy and the Simpson index. There are many posts on this site, some is

Now, the usual diversity indices takes the categories (in ecology, species) as categorical and treats them all the same. Applied to your application of "diviseveness" or "disagreement" it would treat probability vectors $(1/2,1/2,0,0,0)$ the same way as $(1/2,0,0,0,1/2)$, which is not appropriate. So we need to take into account distances among the rating scores, 4 and 5 being more similar that 1 and 5! That means to treat the rating scores as a metric space, and we can use the metric $$ d(i,j)= \vert i-j \vert $$
and use the ideas outlined at How to include the observed values, not just their probabilities, in information entropy?

That can lead to (a version of) the Rao's quadratic entropy, with $$ d(i,j)= 1- e^{\vert i-j \vert} $$ as a measure of disagreement. A paper with theoretical studies of Rao's quadratic entropy. Some numerical examples, in R:

 quadent( rep(0.2, 5))
          [,1]
[1,] 0.6403669
> quadent(c(1, 0, 0, 0, 0))
     [,1]
[1,]    0
> quadent( c(0.5, 0.5, 0, 0, 0))
          [,1]
[1,] 0.3160603
> quadent( c(0.5, 0, 0, 0, 0.5))
          [,1]
[1,] 0.4908422
kjetil b halvorsen
  • 77,844