Rotten Tomatoes, IMDB and the Wisdom of Crowds

In the Introduction of James Surowiecki’s The Wisdom of Crowds, the author writes that “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them”. This prescient book, written in 2004, was describing the crowd-sourcing, data driven world that we live in today. If you want information, you type a couple of words into Google and you find exactly what you were looking for on the first page of links. If you are visiting a new city and you’re looking for a good restaurant, you check Yelp to identify the highest rated restaurants. And, if you want to go to the movies, you check Rotten Tomatoes and IMDB to see which of the movies you are considering is the highest rated.

The “right circumstances” for groups to be intelligent, according to Surowiecki, is that the group has to be big enough, diverse, and individual decisions within the group need to be made independently. Rotten Tomatoes is independent enough, most of the critic reviews are made prior to the release of the movie without knowledge of how other critics are rating the movie. Diversity is an interesting question. They are all movie critics after all and most of them are men. Still, they certainly bring a diverse set of life experiences. So, diversity isn’t optimal but still exists. The biggest question mark is whether the group is big enough. Star Wars: The Force Awakens is the most reviewed movie I’ve come across on Rotten Tomatoes with a little more than 335 critics reviews counted in the rating. My database average is 104 reviews. That is not a big sample size for statistical analysis. While, logically, movies rated Certified Fresh 95% should be better than Certified Fresh 75% movies, my data doesn’t support that.

There is virtually no difference between movies rated higher than Certified Fresh 88% and those less than Certified Fresh 88%. On the other hand, when you just look at Certified Fresh vs. Fresh vs. Rotten movies, the group allocates the movies intelligently.

It turns out that crowds of critics are pretty smart.

IMDB certainly meets the criteria for an intelligent group. It is big enough, Star Wars: The Force Awakens has over 450,000 votes, for example. While not as diverse demographically as one might like, it is much more diverse than a crowd of critics. And, moviegoers who vote on IMDB cast their vote independently (how influenced they are by other ratings is a subject for another day). When I rank the movies in my database by Avg. IMDB Rating and allocate them in groups identical to the Rotten Tomatoes table, you get the following results:

Crowds of moviegoers are pretty smart as well.

Let’s go one step further. What would these results look like for movies that Rotten Tomatoes rated Certified Fresh and IMDB rated 7.4 or higher:

How about if Rotten Tomatoes rated the movie Rotten and IMDB had an average rating of 6.7 or less:

This is the basis for my rating system. When you combine movie recommender systems together, you improve your chances of selecting movies that you will “really like” and avoiding movies you won’t “really like”. It turns out that crowds of critics and moviegoers are the wisest crowds of all.

Author: Mad Movie Man

I love good movies. In my prior life I worked with predictive models. I've combined my love of movies with my prior experience to create a simple Bayesian probability model to help select movies that you will probably "really like". View all posts by Mad Movie Man