Recommender Lab

About

Every streaming service, shop and social feed faces the same question: out of thousands or millions of items, which ten should this person see right now? This lab builds the three classic answers side by side on a small film catalogue and lets you watch them disagree.

What you are looking at

• Latent space. Matrix factorization gives every film and every viewer a short vector of numbers. The map shows each film’s vector projected onto its two most important directions. Genres were never given to the model, yet films of the same colour drift together as training runs.
• Your ratings. You are viewer number 101. Your stars are added to the training data, and the model retrains from scratch.
• Top 10. The films you have not rated, ranked by the chosen method, with the evidence behind each pick.
• Error and recall. How well each method predicts ratings and recovers hidden favourites that were held out of training.

To learn the ideas in depth, read the Recommender Systems lesson.

How it works

Popularity

Count how many viewers gave each film 4 or 5 stars and recommend the top of that list to everyone, minus what they have already rated. It knows nothing about you, yet it is a stubbornly strong baseline because people really do watch popular films more.

Item-item collaborative filtering

For each pair of films, look at the viewers who rated both, subtract each viewer’s average (so a harsh rater’s 3 counts as praise) and compute the cosine between the two columns of centred ratings. Pairs with few shared viewers are shrunk towards zero: sim · n / (n + 5). Your predicted rating for a film is your average plus a similarity-weighted average of how far above or below your average you rated its neighbours. This is the approach Amazon described in 2003.

Matrix factorization

Model every rating as r̂ = μ + b_u + b_i + p_u · q_i: the global average, a personal bias, a film bias, and the dot product of a k-dimensional viewer vector with a k-dimensional film vector. Training visits every known rating in random order and nudges the parameters to shrink the error:

That is stochastic gradient descent on squared error plus an L2 penalty λ(|p|² + |q|² + b²), the recipe made famous by the Netflix Prize and set out by Koren, Bell and Volinsky in 2009. One epoch is one pass over the roughly 1,250 training ratings.

The map

With k above 2 the vectors cannot be drawn directly, so the map shows the two directions along which the film vectors vary most (the top two singular vectors of the film matrix, without centring, so the origin keeps its meaning). A film’s predicted score for you depends on the dot product with your vector, so films lying further out in the direction of your dashed arrow tend to score higher. Viewers are drawn on a shared scale adjusted to fit the plot.

Evaluating

Before training, 20% of each synthetic viewer’s ratings are set aside. Nothing in training ever sees them. Two numbers are computed on that hidden set:

Reading the curves

Training error always falls as you train longer or raise k: the model gets more freedom to fit the ratings it sees. Held-out error falls at first and then, with weak regularization, turns back up. That upturn is overfitting. Set λ to 0 and k to 8 and train for 300 epochs to see it clearly, then raise λ and watch the gap close.

Offline is not online

These are offline metrics on logged data. Real services also run online A/B tests, because a model that recovers past ratings well may still be worse at what matters: whether people find something they enjoy, and come back. See the evaluation section of the lesson.

Cold start

Collaborative filtering learns only from interactions. A film nobody has rated has no interactions, so there is nothing to learn from. Press “Add a new film” and look at each method:

• Matrix factorization never updates the new film’s vector, because SGD only touches films that appear in a rating. It stays at its tiny random starting value near the origin, so its prediction for everyone is roughly the global average plus their personal bias.
• Item-item has no viewer who rated both the new film and anything else, so every similarity is zero.
• Popularity ranks it dead last, which guarantees nobody sees it, which guarantees it stays unrated. This loop is why cold start is a product problem as well as a modelling one.

Now press “Give it 5 ratings”. Five simulated viewers whose hidden taste suits it rate it, the model retrains, and the film moves towards the films those viewers liked. The same logic applies to new users: rate a single film and your dot on the map is placed with very little evidence; rate six and it settles.

Try this

1. 01Watch genres emergePress Reset, then Epoch a few times. At first the dots are a random cloud. Keep stepping: colours start to gather, although the model was never told any genres.
2. 02Change your mindLoad the "Sci-fi and action" preset, note your top 10, then rate Titanic 5 stars. Which recommendations change, and what reasons do they give?
3. 03Overfit on purposeSet λ to 0, k to 8 and epochs to 300. Training RMSE drops towards zero while held-out RMSE rises. Then raise λ to 0.2 and compare.
4. 04Too few factorsSet k to 1. Every film now lies on a single line, roughly "how much people like it". Compare recall@10 with k = 4.
5. 05Beat the baselineFind settings where matrix factorization beats item-item on recall@10. Is the best setting for RMSE also the best for recall?
6. 06Rescue a new filmAdd the new film. Rate some sci-fi highly and check where it ranks for you. Give it 5 ratings and check again.