In one of my early articles, I wrote about the benefits of including older movies in your catalogue of movies to watch. I used the metaphor of our preference for holding onto shiny new pennies rather than tarnished old quarters. One of the things that has been bothering me is that my movie selection system hasn’t been surfacing older movie gems that I haven’t seen. Take a look at the table below based on the movie I’ve watched over the last 15 years:
|Movie Release Time Frame||# of Movies Seen||% of Total|
|2007 to 2016||573||29%|
|1997 to 2006||606||31%|
|1987 to 1996||226||11%|
|1977 to 1986||128||6%|
|1967 to 1976||101||5%|
|1957 to 1966||122||6%|
|1947 to 1956||109||6%|
|1937 to 1946||87||4%|
|1920 to 1936||25||1%|
60% of the movies I’ve watched in the last 15 years were released in the last 20 years. That’s probably typical. In fact, watching movies more than 20 years old 40% of the time is probably unusual. Still, there are probably quality older movies out there that I’m not seeing.
My hypothesis has been that the databases for the movie websites that produce my recommendations are smaller for older movies. This results in recommendations that are based on less credible data. In the world of probabilities, if your data isn’t credible, your probability stays closer to the average probability for randomly selected movies.
I set out to test this hypothesis against the movies I’ve watched since I began to diligently screens my movies through my movie selection system. It was around 2010 that I began putting together my database and using it to select movies. Here is a profile of those movies.
|Seen after 2010|
|Time Frame||Average Rating||# of Movies Seen||% of Total Seen|
|2007 to 2016||7.2||382||55%|
|1997 to 2006||7.9||60||9%|
|1987 to 1996||7.9||101||15%|
|1977 to 1986||7.8||57||8%|
|1967 to 1976||7.9||23||3%|
|1957 to 1966||8.2||26||4%|
|1947 to 1956||8.2||20||3%|
|1937 to 1946||8.4||17||2%|
|1920 to 1936||6.9||4||1%|
It seems that it’s the shiniest pennies, that I watch most often, that I’m least satisfied with. So again I have to ask, why aren’t my recommendations producing more older movies to watch?
It comes back to my original hypothesis. Netflix has the greatest influence on the movies that are recommended for me. So, I compared my ratings to Netflix’ Best Guess ratings for me and added the average number of ratings those “best guesses” were based on.
|Movie Release Time Frame||My Average Rating||Netflix Average Best Guess||Avg. # of Ratings per Movie||My Rating Difference from Netflix|
|2007 to 2016||7.2||7.7||1,018,163||-0.5|
|1997 to 2006||7.9||8.0||4,067,544||-0.1|
|1987 to 1996||7.9||8.1||3,219,037||-0.2|
|1977 to 1986||7.8||7.8||2,168,369||0|
|1967 to 1976||7.9||7.6||1,277,919||0.3|
|1957 to 1966||8.2||7.9||991,961||0.3|
|1947 to 1956||8.2||7.8||547,577||0.4|
|1937 to 1946||8.4||7.8||541,873||0.6|
|1920 to 1936||6.9||6.1||214,569||0.8|
A couple of observations on this table;
- Netflix pretty effectively predicts my rating for movies released between 1977 to 2006. The movies from this thirty year time frame base their Netflix best guesses on more than 2,000,000 ratings per movie.
- Netflix overestimates my ratings for movies released from 2007 to today by a half point. It may be that the people who see newer movies first are those who are most likely to rate them higher. It might take twice as many ratings before the best guess finds its equilibrium, like the best guesses for the 1987 to 2006 releases.
- Netflix consistently underestimates my ratings for movies released prior to 1977. And, the fewer ratings the Netflix best guess is based on, the greater Netflix underestimates my rating of the movies.
What have I learned? First, to improve the quality of new movies I watch, I should wait until the number of ratings the recommendations are based on is greater. What is the right number of ratings is something I have to explore further.
The second thing I’ve learned is that my original hypothesis is probably correct. The number of ratings Netflix has available to base its recommendations on for older movies is probably too small for their recommendations to be adequately responsive to my taste for older movies. The problem is, “Oh, what to do about those tarnished old quarters” isn’t readily apparent.