Probabilistic modelling
Fraud Detection with Gaussian Mixtures
0.697
PR-AUC (primary) · ROC-AUC 0.969 · F1 0.727 on held-out test
With fraud at roughly 0.17% of transactions, accuracy is meaningless. I wanted to see how far a small, explainable probabilistic model could go before reaching for gradient boosting, and to build the evaluation properly.
Build-up
- Single-feature Gaussians first: the best feature (V12) reaches F1 0.62, so no single signal is enough.
- A joint GMM on V14 and V17 reaches PR-AUC 0.59, and the final model uses five features (V11, V12, V14, V16, V17).
- Final scorer: fit a 2-component GMM on normal transactions and a 3-component GMM on fraud, then score each transaction by fraud log-likelihood minus normal log-likelihood.
- The threshold is chosen by an O(n log n) sweep over sorted scores that maximises F1 on the validation set, then applied unchanged to the test set (precision 0.754, recall 0.703).
Honest notes
- The fraud-side mixture is fitted on labelled fraud, so this is a class-conditional generative classifier rather than pure unsupervised anomaly detection. The README also notes that supervised methods may beat it when labels are plentiful.
- PR-AUC is the headline metric because ROC-AUC is inflated by the huge number of true negatives.
Stack
scikit-learnNumPyPandasMatplotlib