ASL Fingerspelling Recognition

The model

• MediaPipe finds 21 landmarks per hand in each frame, giving 84 numbers per frame (x and y for two hands, zeros when a hand is missing). Each sequence is centred on the wrist and scaled so the model sees hand shape rather than where the hand sits in the frame, then resampled or padded to 64 frames.
• A Conformer encoder (6 layers, 384 dimensions, 6 heads, macaron feed-forward blocks and a depthwise convolution module) feeds a 6-layer Transformer decoder over a character vocabulary. Beam search turns it into text. 27.8M parameters.
• Trained on more than 115,000 sequences from the Kaggle Google ASL Fingerspelling data (the main and supplemental sets).

Evaluation I can defend

The number in my original notebook was computed with teacher forcing and on a different evaluation, so I did not trust it. The repo now scores every model the same way.

• The split is 70/15/15 by signer, so validation and test signers never appear in training. Test is scored once, at the end, on a seeded random sample of 3,000 clips from the 14 held-out signers.
• Every comparison reports per-signer CER and a paired interval across signers instead of one number, because only 14 signers means a single average can mislead.
• Choices (checkpoint, decoding, thresholds) were made on validation only. The 0.43 to 0.44 figures from the original notebooks came from a different evaluation and are not comparable with the table below.
• For scale: the winning Kaggle solutions scored about 0.18 to 0.19 in these units. This model does not reach that, and the write-up says so.

Results on held-out signers

• Both improvements were found by looking harder at the model, not by training more. The last checkpoint beats the one with the best validation loss because that loss is teacher-forced: it rose slightly from epoch 13 while the decoded text kept improving. It is better for 12 of 14 signers (0.013 lower on test).
• Beam search with a length penalty of 0.0 (no pressure toward longer output) is another 0.015 better (12 of 14 signers, paired interval -0.023 to -0.007), and 0.027 better than the first deployed model (13 of 14 signers). It fits the failure mode below: the invented text is longer than what was signed.

Where it fails

• The errors are lumpy, not spread out. 36% of clips are decoded perfectly, while the 12% of clips with CER of 0.9 or more account for 72% of all the error.
• Most of those failures are not near misses. In two thirds of them the model outputs a different kind of phrase than the one signed (a phone number comes out as an address) and the output is longer than the target, as if the decoder made up something fluent when it could not see the hands.
• Which signer it is matters more than anything else. Per-signer CER runs from 0.03 to 0.78 and correlates at -0.96 with how often MediaPipe found a hand in that signer's frames. Signers with a hand detected in under a third of frames sit at 0.6 to 0.8 CER.
• So the remaining error is mostly an input problem, not a modelling one: when the hands are not detected, there is nothing to read.

Knowing when it is wrong

Since the model fails by making text up, I made the system say so. A rule was fitted on validation only and applied once to test.

• The API returns a flagged field with a reason (low hand visibility or low confidence) and Prometheus counters for each, and the demo shows a warning. The thresholds are environment variables.
• A caveat I state in the repo: in the app, hand rate comes from MediaPipe on the visitor's own video, while the threshold was fitted on the competition's landmarks, so it may need re-tuning.

Six ideas that did not help

I implemented five ideas plus a character language model, each switchable in config and unit tested, then ran an ablation on a Kaggle T4 (about 5 hours). Every variant warm-started from the deployed weights and was scored with the same protocol.

• Nothing beat the deployed weights, and fine-tuning made them worse (0.065 higher on average, worse for all 14 signers, paired interval 0.047 to 0.089). I first read a0 as a fair baseline and only caught this once I scored the untouched weights with the same protocol.
• I also noted why this is not a fair test of the ideas: each variant started from weights trained for something else and trained for a few epochs, so anything that changes the input or adds a head had to be adapted onto them. That is the question the next section answers.
• More training on the same signers overfit: validation was best after the first epoch and drifted worse while training accuracy kept climbing. The character language model did not help either (its best tuned weight was 0).

Training from scratch, twice

To answer the fairness question, I trained from random weights with the ingredients the top Kaggle solutions used: 192 frames, heavy augmentation, CutMix on half the clips, decoder input masking, a joint CTC loss and weight averaging. One 16-epoch run on a Kaggle T4 took 2 hours 45 minutes.

• It did not beat the deployed model (the CTC head is 0.022 behind, paired interval 0.009 to 0.035, better for only 4 of 14 signers), so nothing was changed in the app.
• I read it as under-trained rather than as a failed recipe: validation CER was still falling when the schedule ended, and the winning solutions trained for about 300 epochs. The CTC head beat the attention decoder for all 14 signers, which fits the attention decoder needing far more epochs to learn to read the encoder.
• The CTC head has no sequence confidence, so the flag rule only applies to the attention decoder.
• Since the curve was still falling, I restarted from those weights with a fresh schedule (3 hours 40 minutes on a T4, early stopping at epoch 26). The CTC head gained 0.017 for every one of the 14 signers and is now 0.008 behind the deployed model, but the paired interval (-0.011 to 0.020) includes zero, so I cannot tell them apart. I only deploy a model that is clearly better, so the app is unchanged.
• The attention decoder barely moved, and I stopped on the wrong metric: early stopping used its noisy, flat validation CER while the CTC head was still improving. It would not have changed the conclusion (about 0.004), but it is the same lesson as the first checkpoint: choose the checkpoint using the metric you will actually serve.

Serving and monitoring

• A FastAPI service takes a video (POST /predict) or raw (T, 84) landmarks (POST /predict/landmarks), reports the model version on /health and exposes Prometheus metrics on /metrics, and writes a structured log line per request.
• Metrics cover request rate and latency, beam search time, prediction confidence, how many frames had a hand and how many videos had no hands at all.
• A drift monitor compares live inputs (clip length, share of frames with each hand, amount of motion) to a reference built from training data using the population stability index over a rolling window. Prometheus alert rules cover drift, a high no-hands rate, slow inference and low confidence, with a provisioned Grafana dashboard.
• I ran the whole docker compose stack (api, MLflow, Prometheus, Grafana): 30 test requests showed up in Prometheus, every dashboard panel returned data and the four alert rules loaded. The test clips were noise, so the drift score was far above the alert level, as it should be.

Making inference faster

• fp32 ONNX is 1.6x faster with identical output, so it is the service default. int8 is smaller but changed the text on one clip in five, so I would not ship it without checking CER on real test data. Quantising the convolutions as well made things worse, so I left them out.
• Beam search now runs all live beams in one decoder call instead of one call per beam: a clip went from about 1.4 s to 0.64 s with identical output on every clip tried, backed by a test against the original implementation.

MLOps and delivery

• DVC chains preprocess, train and evaluate so only stale stages rerun. MLflow logs parameters, per-epoch metrics, the git commit and the checkpoint, and a promotion step only moves the production alias if test CER clears a threshold.
• CI runs on every push: lint, the test suite on Python 3.10 and 3.12, a short training run on generated data, an evaluation with a metric gate that fails if CER or exact match regress, an ONNX export, then builds the Docker image and calls the running container. A release workflow publishes the image to GHCR when a version tag is pushed (I have not tagged one yet, so that workflow has never run, and the README says so).
• The demo runs as a Gradio Space on free ZeroGPU hardware with the model call wrapped for GPU and everything else on CPU. The weights live in a Hugging Face model repo and download at startup, so nothing large is in git. A workflow redeploys the Space when the model or app code changes.
• The demo has an optional LLM step (Llama via Groq) that snaps noisy output onto a fixed word list. It is a demo feature and none of the reported numbers use it.

Where I stopped, and what I would do next

• I stopped at the tuned model (test CER 0.307, with 40% of clips flagged as unreliable). A longer from-scratch run brought the CTC head level with it within noise, so the remaining error is probably the data, not the training recipe.
• Next: rescoring beam search hypotheses with the CTC head or averaging it with the deployed model, pose and lip landmarks for clips where the hands are missed (where most of the remaining error is), z coordinates as extra features, putting the FastAPI service somewhere public (the Space only runs the Gradio demo), and tagging a release.
• Limitations: isolated fingerspelling clips only, no full sentences. It struggles with bad lighting, odd camera angles and signers unlike the training set, and short words tend to get made up. The model was trained on the competition's landmarks but the demo runs MediaPipe on the visitor's own video, so there is a domain gap.