NLP
IMDB Sentiment: LSTM vs BERT
0.919
validation F1 for BERT vs 0.895 for the LSTM
A controlled look at what transfer learning buys on a classic benchmark, with tuning done per model instead of reusing one setup.
What I did
- Modular training code (data, models, training, utils) with checkpointing and seeding; tuning results for each model saved as CSV.
- The LSTM uses embeddings, a bidirectional recurrent layer (256 hidden units) and a small MLP head; BERT adds a linear classifier on the pooled output.
- A smaller batch size (8) generalised better for BERT, and weight decay 0.01 was used.
Stack
PyTorchHugging Face TransformersStreamlit