Research prototype / Medical imaging
Uncertain is not negative.
A multi-label chest X-ray classifier over 14 findings, trained across three corpora that do not agree about what they label. The interesting part is not the network. It is the mask that decides which labels are allowed to teach it anything.
Problem
Each dataset labels different findings.
How it was solved
Labels a dataset cannot give are masked, never guessed as negative.
- Three datasets
- Mask
- Focal loss
- Patient split
Result
Masked
missing labels drop out of training
Built with
- PyTorch
- DenseNet-121
- Grad-CAM++
- Apache Kafka
Research prototype / Medical imaging
A chest report that never mentions a finding has not ruled it out. Training a classifier as though it had is the mistake this project is about.
Published performanceNone
How it works
What to do with a label nobody gave
The hard part of this prototype is not the network. Three corpora disagree about what they can label, and the training has to respect that.
Fourteen channels, and the ones a corpus cannot speak to
Problem
Only one of the three sources can speak to every finding. The others are silent on most of them, and silence is not a negative.
Mask
An entry a corpus cannot label is masked, so it contributes nothing to the loss and nothing to the metrics.
Weight
What remains goes through a focal loss, normalised by the mask and the weights, so a heavily masked batch is not scaled as though it had been learned from.
Split
Images are split by patient rather than by image, so two views of one person cannot land on both sides.
Select
The checkpoint is chosen on the minimum class AUC, because a mean would let a strong majority class hide a finding the model cannot do at all.
What this does not show
A research prototype and not a clinical one. The boundaries further down this page are part of the work, and nothing here is a medical claim.
Inspect repository
This diagram in words
- Three chest corpora source — They disagree about which findings they annotate at all.
- Split by patient gate — Not by image. One chest appears more than once.
- Multi-label classifier process — Fourteen finding channels, each decided independently.
- Unlabelled channels masked gate — A finding a corpus cannot label drops out rather than counting as absent.
- Saliency artifact — Where the evidence for a channel was read from.
- No published result sink — No trained weights, no held-out metrics, no clinical validation.
Read the full method
What this is
Three corpora, one label space, and a great deal of silence.
The model predicts 14 findings at once. Only one of its three sources can speak to all 14 of them: NIH ChestX-ray14. CheXpert supplies 7, and the pneumonia set supplies 1. Within the findings CheXpert does carry, an entry can still be explicitly uncertain, or simply not mentioned at all.
The decision that shapes the whole project is what to do with that silence. Treating it as a negative is cheap and wrong: it would teach the model that every CheXpert chest is free of emphysema, fibrosis and hernia, and that every image in the pneumonia set is free of the other thirteen findings. Instead each of those entries is masked, and a masked entry contributes nothing to the loss and nothing to the metrics.
The loss
Focal weighting, and then a switch that can turn the whole term off.
The loss is a focal loss with label smoothing (0.02), alpha 0.75 so positives are weighted three to one, gamma 2 so confident examples stop dominating, and a class weight applied to positives only so that boosting a rare finding does not also boost the flood of negatives around it. All of that is multiplied by the mask, and normalised by the sum of the mask times the weights rather than by the element count — otherwise batches with a lot of masked entries would be scaled as though those entries had been learned from.
Splits and evaluation
Split by patient, and judged on the worst finding.
Both CheXpert and NIH are split 80/10/10 at the level of the patient, not the image, so two views of the same person cannot land on both sides of the split. The NIH split is retried until every finding has at least twenty positives in both validation and test, because a split that leaves a rare finding with three positives produces a metric that means nothing.
ROC-AUC per finding, mask-aware, computed only over samples where that channel is certain. The checkpoint is then selected on the minimum class AUC rather than the mean, with Hernia excluded as too rare to steer on — a mean would let a strong majority class hide a finding the model cannot do at all.
Evaluation pipeline implemented; no validated numbers are published in the repository.
Boundaries
A research prototype, and not a clinical one.
- No trained weightsNothing in the repository can make a prediction.
- No patient dataDatasets are downloaded from their owners under their own terms.
- No held-out metricsThe evaluation code exists; its output is not published.
- No clinical validationNot for diagnosis, triage, or treatment decisions.
The repository states that the implementation must not be used for diagnosis, triage, treatment decisions, or any other clinical purpose. Nothing on this page is a medical claim, no image shown anywhere in it is a real radiograph, and no patient data is used, stored, or published.