The problem
Diffusion-weighted MRI of the prostate can suffer geometric distortion. My MSc thesis at UCL (Artificial Intelligence and Medical Imaging, 2026) asked whether that distortion could be detected automatically from the images themselves, using a retrospective dataset of 1,027 acquisitions from 627 patients at a single centre, University College London Hospital. The data are anonymised and cannot be shared.
The more interesting question arrived part-way through. A detector is only useful for triage if it can also say which of its own calls are unreliable, so that those cases can be handed to a person. The most accurate architecture I trained turned out to be poor at exactly this. The thesis became as much about a model’s self-knowledge as about its accuracy.
- Data
- Patient-level folds
- Architectures
- Ensembles
- Uncertainty
- Triage simulation
- Distilled student
- 01
- Data. 1,027 prostate DWI acquisitions from 627 patients; single centre, anonymised; cannot be shared.
- 02
- Patient-level folds. Strict five-fold cross-validation split by patient, so no patient appears in both training and test data.
- 03
- Architectures. Classical baselines, then ResNet-18, ResNet-50, DenseNet-121 and EfficientNet-B0.
- 04
- Ensembles. A three-seed DenseNet-121 ensemble for accuracy; a heterogeneous nine-model ensemble for uncertainty.
- 05
- Uncertainty. MC dropout, deep ensembles, calibration and expected calibration error; ensemble homogeneity and correlated errors examined.
- 06
- Triage simulation. Selective prediction: defer the least-certain cases and measure accuracy on those retained.
- 07
- Distilled student. One model trained to reproduce the ensemble, cutting forward passes from nine to one.
What I did
I built the pipeline end to end in PyTorch, from data handling to statistics.
- Data and validation. Strict patient-level five-fold cross-validation, so that every result is measured on patients the model has never seen, and no acquisition from the same patient leaks between folds.
- Baselines and backbones. Classical baselines first, then ResNet-18, ResNet-50, DenseNet-121 and EfficientNet-B0, compared on the same folds.
- Ensembles and uncertainty. Three-seed ensembles, deep ensembles, MC dropout, selective prediction, expected calibration error and calibration analysis. When the most accurate architecture proved weak at flagging its own errors, I investigated ensemble homogeneity and correlated errors and built a heterogeneous nine-model ensemble instead.
- Interpretation and checks. Grad-CAM, bootstrap confidence intervals, paired Wilcoxon tests and synthetic-distortion validation.
- Distillation. A single student model that reproduces the ensemble’s behaviour in one forward pass instead of nine.
- Also in the public code. SimCLR and BYOL pretraining, DWI plus T2 fusion, an auxiliary b-value head, and a CPU-only pytest suite that runs on synthetic NIfTI data, so the repository can be tested without any patient data.
Decisions
- Split by patient, not by acquisition. Several acquisitions can come from one patient. Acquisition-level folds would have flattered every number; patient-level folds are the honest measure.
- Report the spread across folds. The headline AUROC of 0.819 carries an SD of 0.049 across the five folds. That is fold-to-fold variation, not seed-to-seed variation, and it is not a single-model result.
- Prefer diversity to raw accuracy for the uncertainty ensemble. Models that make the same mistakes cannot warn each other. The heterogeneous nine-model ensemble was built to disagree usefully.
- Frame the result as triage. The question a clinic would ask is not “how accurate is it” but “how many cases can it safely take, and how accurate is it on those”. Selective prediction at 70% coverage answers that.
- Distil for cost. Nine forward passes per case is a research budget. One is a deployable one.
- Test without the data. Synthetic-distortion validation as a check on the method, and a test suite on synthetic NIfTI volumes, so the code is verifiable by anyone.
Evidence
| Figure | Context | As of |
|---|---|---|
| 1,027 | MRI acquisitions from 627 patients; single centre, anonymised; strict patient-level five-fold cross-validation | Thesis, 2026 |
| 0.819 | Mean AUROC of a three-seed DenseNet-121 ensemble; SD 0.049 across the five folds, not across seeds | Thesis, 2026 |
| 0.685 | Uncertainty-error AUROC of a heterogeneous nine-model ensemble: how well its uncertainty identified its own mistakes | Thesis, 2026 |
| 83.6% | Retained-case accuracy at 70% coverage in a retrospective triage simulation, deferring the least-certain 30% of cases | Thesis, 2026 |
| 9 to 1 | Forward passes per case after distilling the nine-model ensemble into a single student model | Thesis, 2026 |
Read together: the accurate ensemble is accurate; the diverse ensemble is better at knowing when it is wrong (uncertainty-error AUROC 0.685); and using that uncertainty to defer 30% of cases raised accuracy on the remaining 70% to 83.6% in a retrospective simulation. The distilled model makes the approach affordable to run.
Placeholder
Reliability diagram, ROC curves and Grad-CAM examples, to be supplied by Michal as anonymised or synthetic images. No patient images will be shown.
Limitations and status
This is a retrospectively validated proof of concept. The dataset comes from a single centre, the triage figure comes from a retrospective simulation rather than a prospective study, and nothing here has been deployed or validated in a clinic. An uncertainty-error AUROC of 0.685 is a moderate signal, not a solved problem. The data cannot be shared, so the public repository ships with synthetic data and tests rather than the thesis dataset.
What I learned
- Accuracy and self-knowledge are different properties of a model, and the second is the one that decides whether it can be trusted with a queue.
- Ensemble diversity is a design choice, not a by-product; seeds alone do not buy it.
- Say what a standard deviation is across. Folds and seeds tell different stories.
- Write the tests so they run without the data. It is the only way anyone else can check the work.
Links
- Code on GitHub, MIT licence (external link)The full pipeline, the uncertainty methods and a CPU-only test suite that runs on synthetic NIfTI data.