Recommending for new users is surprisingly difficult
By Daniel Kluver on
For years MovieLens has required users to enter 15 ratings before they are allowed to get personalized recommendations. This design makes sense: how can we make recommendations for a user we know nothing about? That said, we don’t know if this provides users with the best experience. Why should users have to enter fifteen ratings, why not ten or five? What would happen if we let users into the system without any ratings? To answer these questions we need to understand how our algorithms behave for users with very few ratings.
To understand how algorithms behave for users just joining the system, we looked at historic MovieLens ratings. We trained three popular recommender algorithms: ItemItem, UserUser, and SVD on this rating data. While training, we limited some users to have only a small number of ratings. We used the ratings that were not given to the algorithm to measure several things:
- How accurate are the predictions? Can the algorithm accurately predict the user’s future ratings?
- How good are the recommendations? Does the algorithm suggest movies for the user that the user would like?
- What type of recommendations does the algorithm generate? Is there a good diversity of movies? Are the movies popular, or more obscure?