For “Really Like” Movies, Trust Your Memory More Than IMDB and Rotten Tomatoes

I find that, while I forget the specifics of the movie, I maintain sense of whether I liked it or not and that often times drives whether I want to see it a second time. What is a more reliable indicator of whether I will “really like” a movie the second time around. Is it my general memory of whether I liked it or not or the recommendations of IMDB and Rotten Tomatoes?

As of today, there are 1,984 movies in my database. Of those 1,984 movies, I’ve watched 446 movies more than once. The general rule I’ve set for myself is to wait at least 15 years before seeing a movie an additional time. I’ve found that over that time span I retain a general idea of what happens in the movie but forget the specific details of what happens. When I watch the movie the additional time it feels like a new movie to me.

I find that, while I forget the specifics of the movie, I maintain sense of whether I liked it or not and that often times drives whether I want to see it a second time. The question I’m exploring today is; When deciding to watch a movie a second time, should I trust the impression of the movie I remember or the recommendations of IMDB or Rotten Tomatoes?

The data suggests that the answer to this question is pretty clear-cut. The movies that I’ve seen before have a much greater probability that I will “really like” them whether or not they are recommended by IMDB or Rotten Tomatoes.


When I watched a Movie, I had: And it was IMDB: Probability I Will “Really Like”
Seen Before Recommended 80.1%
Seen Before Not Recommended 69.2%
Never Seen Before Recommended 50.6%
Never Seen Before Not Recommended 33.6%
When I watched a Movie, I had: And it was Rotten Tomatoes: Probability I Will “Really Like”
Seen Before Recommended 80.5%
Seen Before Not Recommended 65.1%
Never Seen Before Recommended 49.8%
Never Seen Before Not Recommended 31.8%

While there are qualitative differences for IMDB and Rotten Tomatoes recommendations, the movies that your memory wants you to see a second time are clearly superior indicators of whether it will be a “really like” movie or not. And, the probabilities are higher for a non-recommended movie you’ve seen before than a recommended movie you haven’t seen before.

The other aspect of this data is that it reveals the weaknesses of IMDB and Rotten Tomatoes as recommenders of movies you haven’t seen. Strip the data of movies that you have seen before and a recommended movie has only a slightly better chance that you will “really like” it than selecting a movie randomly from the database. It reinforces the notion that these sites are better at weeding out movies that you won’t like rather than identifying movies that you will “really like”. Later this week, in Thursday’s post, I’ll share how this cut of the data looks for the websites that are sensitive to my individual taste in movies. Theoretically, it should better replicate the wisdom of movie memories.

For now, though, the wisdom of crowds of moviegoers and critics can’t compete with those golden memories you have of great movies seen in the past when it comes to “really like” movies.


Will Family Movie Night Be Cringeworthy?

A fairly new site has been created, cringeMDb, which allows you to input a movie released between 1995 and 2015 to determine whether it is safe to watch with your parents.

One of the activities we enjoy doing as a family is a trip to the movie theater to watch a movie we all want to see. In the summer of 2005, that movie was The 40 year-old Virgin. The R rating didn’t scare us away from this well-reviewed movie, since the kids were aged 18 to 23. As we walked out of this funny, but raunchy, film, my wife turned to me and said, “That was uncomfortable”. Sitting with our children through explicit sexual references, one after the other, turned family “fun day” into family “awkward” day. While we had no way of knowing, since we didn’t want to discuss it with them, it had to be uncomfortable for my daughter and two sons as well.

Fortunately, there is a website that can help avoid those embarrassing movie viewing situations. A fairly new site has been created, cringeMDb, which allows you to input a movie released between 1995 and 2015 to determine whether it is safe to watch with your parents. It’s not going to help you decide if a recent release is cringeworthy, and I don’t know if the site has plans to periodically add newer movies to the site but it will include many of the movies you might watch as a family in front of your own TV.

To provide some perspective on what movies might qualify as cringeworthy, I tested the tool against the eight 2016 Best Picture nominees:

2016 Best Picture Nominee cringeMDb Rating
Big Short, The Certified Cringeworthy
Bridge of Spies Certified Parent-Safe
Brooklyn Certified Cringeworthy
Mad Max: Fury Road Certified Parent-Safe
Martian, The Certified Cringeworthy
Revenant, The Certified Cringeworthy
Room Certified Parent-Safe
Spotlight Certified Cringeworthy

The site goes beyond the traditional rating system. Note that the R rated Mad Max: Fury Road is Certified Parent-Safe while the PG-13 Brooklyn is Certified Cringeworthy. In my opinion, the tool needs some more work. CringeMDb has almost no tolerance for any nudity or sexuality. I mean really, The Martian is cringeworthy? Not all sexual situations are cringeworthy. Spotlight, which my wife and I watched with my son, generated discussion of the topic, not embarrassment. You can register whether you agree or disagree with the rating the site generates and ideally your input would feed their algorithm. Whether it does or not I can’t say. Hopefully, it does. I’m confident the “wisdom of crowds” would do a fairly good job of defining the line where a movie crosses over into the cringeworthy zone.  It is an interesting and useful idea for a site and it is well worth watching to see if the developers do more with it.

Done right, cringeMDb, my children will thank you.





Rotten Tomatoes, IMDB and the Wisdom of Crowds

In the Introduction of James Surowiecki’s The Wisdom of Crowds, the author writes that “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them”. This prescient book, written in 2004, was describing the crowd-sourcing, data driven world that we live in today. If you want information, you type a couple of words into Google and you find exactly what you were looking for on the first page of links. If you are visiting a new city and you’re looking for a good restaurant, you check Yelp to identify the highest rated restaurants. And, if you want to go to the movies, you check Rotten Tomatoes and IMDB to see which of the movies you are considering is the highest rated.

The “right circumstances” for groups to be intelligent, according to Surowiecki, is that the group has to be big enough, diverse, and individual decisions within the group need to be made independently. Rotten Tomatoes is independent enough, most of the critic reviews are made prior to the release of the movie without knowledge of how other critics are rating the movie. Diversity is an interesting question. They are all movie critics after all and most of them are men. Still, they certainly bring a diverse set of life experiences. So, diversity isn’t optimal but still exists. The biggest question mark is whether the group is big enough. Star Wars: The Force Awakens is the most reviewed movie I’ve come across on Rotten Tomatoes with a little more than 335 critics reviews counted in the rating. My database average is 104 reviews. That is not a big sample size for statistical analysis. While, logically, movies rated Certified Fresh 95% should be better than Certified Fresh 75% movies, my data doesn’t support that.

“Really Like” Don’t “Really Like” Total % “Really Like”
CF > 88% 284 155 439 64.7%
CF < 88% 283 154 437 64.8%

There is virtually no difference between movies rated higher than Certified Fresh 88% and those less than Certified Fresh 88%. On the other hand, when you just look at Certified Fresh vs. Fresh vs. Rotten movies, the group allocates the movies intelligently.

“Really Like” Don’t “Really Like” Total % of Total Database % “Really Like”
 CF 567 309 876 44.6% 64.7%
F 324 399 723 36.9% 44.8%
R 91 272 363 18.5% 25.1%

It turns out that crowds of critics are pretty smart.

IMDB certainly meets the criteria for an intelligent group. It is big enough, Star Wars: The Force Awakens has over 450,000 votes, for example. While not as diverse demographically as one might like, it is much more diverse than a crowd of critics. And, moviegoers who vote on IMDB cast their vote independently (how influenced they are by other ratings is a subject for another day). When I rank the movies in my database by Avg. IMDB Rating and allocate them in groups identical to the Rotten Tomatoes table, you get the following results:

Avg. IMDB Rating “Really Like” Don’t “Really Like” Total % of Total Database % “Really Like”
> 7.4 552 324 876 44.6% 63.0%
6.7 to 7.4 361 362 723 36.9% 49.9%
< 6.7 69 294 363 18.5% 19.0%

Crowds of moviegoers are pretty smart as well.

Let’s go one step further. What would these results look like for movies that Rotten Tomatoes rated Certified Fresh and IMDB rated 7.4 or higher:

“Really Like” Don’t “Really Like” Total % of Total Database % “Really Like”
370 156 526 26.8% 70.3%

How about if Rotten Tomatoes rated the movie Rotten and IMDB had an average rating of 6.7 or less:

“Really Like” Don’t “Really Like” Total % of Total Database % “Really Like”
24 193 217 11.1% 11.1%

This is the basis for my rating system. When you combine movie recommender systems together, you improve your chances of selecting movies that you will “really like” and avoiding movies you won’t “really like”. It turns out that crowds of critics and moviegoers are the wisest crowds of all.