Read the label
before the result
The model
We did not train this network. We audited it.
Purino/resnet50-chest-xray-pneumonia, pinned at revision da2a7724bfa5. It is a torchvision ResNet-50 (ImageNet V2 initialisation) fine-tuned for two classes, NORMAL and PNEUMONIA, at 320 × 320 px with aspect-preserving black padding.
Notes from the model card, quoted or paraphrased:
- Final weights were refit on all 5,824 cleaned Kermany images, including the original test folder. “It has no independent test evaluation.”
- Its reported metrics (ROC-AUC 0.998) come from a selection checkpoint on an internal split, and “do not evaluate
model.safetensors.” - Source images are from young children at one centre. Performance on adults, other hospitals and other diseases is unknown.
- Patient grouping was inferred from filenames. “Near-duplicate or undocumented patient overlap may remain.”
- “Softmax outputs are not clinically calibrated probabilities.”
- Research only. Not a clinical diagnostic device.
Why this model? Its card is unusually candid about training scope, leakage and limits. That made it a good subject for an audit. Each warning on the left is something XRayLens measures rather than repeats.
How it runs in your browser
No server sees your image. The model runs on your device.
- The page loads a 66 KB ONNX graph exported from the PyTorch model. Besides the logits, it outputs class-activation maps. For this architecture (last conv block → global average pool → linear), Grad-CAM at the last block equals CAM with the classes’ weight differences exactly, so no gradients are needed in the browser.
- The graph contains no weights. Every tensor points at a byte range inside the model author’s original
model.safetensors, which your browser downloads directly from Hugging Face at a pinned commit (94 MB, cached after the first visit). We never re-host the weights. - ONNX Runtime Web runs the network with WebAssembly, using several threads where the browser allows.
- Preprocessing is a line-by-line port of Pillow: the fixed-point resampling kernels, Python’s banker’s rounding in the letterbox, the 601-2 luma transform and the single-precision blend. On 60 test images the model input is byte-identical to PyTorch’s, and scores agree to within 5×10⁻⁶ (see Reliability §06).
Precomputed versus live. Gallery cases open with results computed in PyTorch, so the page is useful before the weights finish downloading. Once the live model is ready, the page re-runs the case in your browser and prints both scores. The randomisation sanity check needs to re-initialise weights, which the fixed graph can’t do, so it is precomputed for gallery cases and for 120 evaluation images.
The data
Four cohorts, used for different questions. Only redistributable images are served by this site.
| Cohort | Who | Used here for | Licence / terms | On this site? |
|---|---|---|---|---|
| Kermany et al. 2018, test folder (624) | Children aged 1–5, Guangzhou | Showing the leakage gap. Seen in training. | CC BY 4.0 | 3 gallery images |
| RSNA Pneumonia Detection Challenge 2018, sample of 600 | Adults, NIH ChestX-ray14 | External sensitivity and specificity, robustness | RSNA/Kaggle challenge terms, non-commercial research | No. IDs and scores only |
| Montgomery County CXR set (138) | Adults, USA, TB programme | Adult specificity, Grad-CAM vs lung masks | NLM public release, citation required | Gallery images + masks |
| Shenzhen CXR set, sample of 100 | Adults, China, TB programme | Out-of-scope disease probe | NLM public release, citation required | 1 gallery image |
Samples are drawn with a fixed seed (20261020), and every source is pinned to a Hugging Face commit. One dataset we rejected: a Hugging Face set labelled as African patients turned out to be Kermany images under a new name (identical IM-xxxx-xxxx filenames). Testing on it would have repeated the leakage trap.
What XRayLens adds
Our contribution is the instrument, not the classifier.
- Grad-CAM from logits, not softmax. This model’s softmax saturates, so softmax gradients vanish exactly where the model is most confident.
- Area-normalised attribution. Heat on lungs, the frame and padding is always reported against each region’s pixel share.
- Randomisation sanity check (Adebayo et al. 2018), per case and across 120 images.
- Perturbation stress lab. Eight anatomy-preserving edits per image, plus population flip rates.
- An evaluation that refuses the easy number. Leakage made visible, external adult cohorts, Wilson and bootstrap 95% intervals, calibration and a live threshold explorer.
- Private by construction. Uploaded images never leave the device, and the browser pipeline is verified against PyTorch.
Citations
- Kermany D, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018;172(5):1122–1131. Data: Mendeley Data, doi:10.17632/rscbjbr9sj.2 (CC BY 4.0).
- Jaeger S, Candemir S, Antani S, et al. Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 2014;4(6):475–477.
- Shih G, Wu CC, Halabi SS, et al. Augmenting the National Institutes of Health chest radiograph dataset with expert annotations of possible pneumonia. Radiol Artif Intell 2019;1(1):e180041.
- Selvaraju RR, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. ICCV 2017.
- Zhou B, et al. Learning deep features for discriminative localization (CAM). CVPR 2016.
- Adebayo J, et al. Sanity checks for saliency maps. NeurIPS 2018.
- Zech JR, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs. PLOS Medicine 2018;15(11):e1002683.
Third-party software served by this site: ONNX Runtime Web 1.26.0 (MIT, Microsoft), dicom-parser 1.8.21 (MIT), Vega 5 / Vega-Lite 5 (BSD-3-Clause, from jsDelivr).