Recommender systems
Two-Stage Recommender: Retrieval + LambdaRank
+46%
NDCG@10 vs MF (0.161 → 0.235); Hit Rate@10 0.53 → 0.63
I started by comparing matrix factorisation baselines and ended up rebuilding the pipeline as retrieval followed by ranking. My first two-tower model scored worse than plain MF, and finding out why is where most of the value came from.
All movies
1,682
Two-Tower retrieval
InfoNCE + hard negatives
200 candidates / user
MF ranker
LambdaRank fine-tune
Top-10
Approach
- Temporal split (train to Feb 1998, validate March, test April) because a random split leaks future interactions.
- Retrieval: user and movie towers, each an embedding plus an MLP with a residual connection and LayerNorm, L2-normalised for cosine similarity. Trained with InfoNCE using in-batch negatives plus one hard negative per pair sampled from the top 30 items a pretrained MF model likes but the user did not pick, and a learnable temperature clamped to [0.01, 1].
- Ranking: retrieval returns 200 candidates per user, and an MF model is fine-tuned with LambdaRank on exactly those candidates at a lower learning rate (5e-5 vs 1e-3), early-stopping on validation NDCG@10.
What moved the numbers
- Training the ranker on retrieved candidates instead of on all user-item pairs was the biggest single jump: it has to learn the distribution it will actually see.
- Hard negatives and the temperature parameter fixed a retrieval model whose validation Recall@100 stopped improving after one epoch.
- A subtle bug: excluding already-seen items when generating training candidates removed every positive and produced an empty dataset. Seen-item exclusion belongs only at test time.
- The original two-tower reached NDCG 0.102, below both MF baselines. Content features also lowered NDCG for MF, even as recall improved.
- Remaining bottleneck: validation Recall@100 plateaus near 0.22, so about 78% of relevant items never reach the ranker.
Stack
PyTorchInfoNCELambdaRankMovieLens 100KPandas