On Monday I posed the question, is your memory of a movie that you’ve already seen the best predictor of “really like” movies. Based on Monday’s analysis memory certainly comes out on top against IMDB and Rotten Tomatoes. Today, I’m extending the analysis to Criticker, Movielens, and Netflix. By reconfiguring the data used in Monday’s post, you also can measure the relative effectiveness of each site. For example, let’s look again at IMDB.
Probability I Will “Really Like” Based on IMDB Recommendation | ||||
Recommended | Not Recommended | Percentage Point Spread | ||
Seen Before | 80.1% | 69.2% | 0.11 | |
Never Seen Before | 50.6% | 33.6% | 0.17 |
It’s not surprising that the probabilities are higher for the movies that were seen before. After all it wouldn’t make sense to watch again the movies you wished you hadn’t seen the first time. But by looking at the gap between the probability of a recommended movie and a non-recommended movie, you begin to see how effectively the movie recommender is at sorting high probability movies from low probability movies. In this instance, the small 11 point spread for Seen Before movies suggests that IMDB is only sorting these movies into small departures from average. The low probabilities for the Never Seen Before movies suggest that, without the benefit of the memory of a movie seen before, IMDB doesn’t do a very good job of identifying “really like” movies.
Rotten Tomatoes follows a similar pattern.
Probability I Will “Really Like” Based on Rotten Tomatoes Recommendation | ||||
Recommended | Not Recommended | Percentage Point Spread | ||
Seen Before | 80.5% | 65.1% | 0.15 | |
Never Seen Before | 49.8% | 31.8% | 0.18 |
Rotten Tomatoes is a little better than IMDB at sorting movies. The point spreads are a little broader. But, like IMDB, Rotten Tomatoes doesn’t effectively identify “really like” movies for the Never Seen Before group.
Theoretically, when we look at the same data for the remaining three sites, the Percentage Point Spread should be broader to reflect the more personalized nature of the ratings. Certainly, that is the case with Criticker.
Probability I Will “Really Like” Based on Criticker Recommendation | ||||
Recommended | Not Recommended | Percentage Point Spread | ||
Seen Before | 79.3% | 56.4% | 0.23 | |
Never Seen Before | 45.3% | 18.9% | 0.26 |
Like IMDB and Rotten Tomatoes, though, Criticker isn’t very effective at identifying “really like” movies for those movies in the Never Seen Before group.
When you review the results for Movielens, you can begin to see why I’m so high on it as a movie recommender.
Probability I Will “Really Like” Based on Movielens Recommendation | ||||
Recommended | Not Recommended | Percentage Point Spread | ||
Seen Before | 86.6% | 59.6% | 0.27 | |
Never Seen Before | 65.1% | 22.3% | 0.43 |
Unlike the three sites we’ve looked at so far, Movielens is a good predictor of “really like” movies for Never Seen Before movies. And, the spread of 43 points for the Never Seen Before movies is dramatically better than the three previous sites. It is a very effective sorter of movies.
Last, but certainly not least, here are the results for Netflix.
Probability I Will “Really Like” Based on Netflix Recommendation | ||||
Recommended | Not Recommended | Percentage Point Spread | ||
Seen Before | 89.8% | 45.7% | 0.44 | |
Never Seen Before | 65.7% | 21.4% | 0.44 |
What jumps off the page is that there is no memory advantage in the allocation of movies for Netflix. As expected, the Seen Before probabilities are higher. But, there is an identical 44 point gap for Seen Before movies and movies Never Seen Before. It is the only site where you have a less than 50% chance that you will “really like” a movie you’ve already seen if Netflix doesn’t recommend it.
“If memory serves me correctly, I “really liked” this movie the first time I saw it.” That is an instinct worth following even if the movie websites suggest otherwise. But, if Netflix doesn’t recommend it, you might think twice.
***
6/24/2016 Addendum
I’ve finalized my forecast for the last three movies on my June Prospect list. My optimism is turning to pessimism regarding my hopes that Independence Day: Resurgence and Free State of Jones would be “really like movies”. Unfavorable reviews from the critics and less than enthusiastic response from audiences suggest that they could be disappointments. Of my five June prospects, Finding Dory seems to be the only safe bet for theater viewing, with Me Before You a possibility for female moviegoers. The IMDB gender split is pronounced for Me Before You with female voters giving it an 8.1 rating and males a 7.3 rating. It is also one of those rare movies with more female IMDB voters than males.