Go back     Home Solutions Claims Reloaded

Claims Reloaded

Authors: Evangelia Christodoulou, Carlos Aumente-Maestro, Annika Reinke, A. Emre Kavur, Charles Heitz, Pascaline Andre, Kris Dreher, Lena Maier-Hein, Gaël Varoquaux, Olivier Colliot

The mission of Claims Reloaded is to provide an open-source toolkit for assessing the validity of claims of superior performance in biomedical segmentation and classification tasks, enabling researchers and reviewers to detect unsupported claims.

Performance comparisons are fundamental in medical imaging AI research, often driving claims of superiority based on relative improvements in common performance metrics. However, such claims frequently rely on empirical mean performance. Claims Reloaded quantifies the probability of false claims based on a Bayesian approach that leverages reported results alongside empirically estimated model congruence to estimate whether the relative ranking of methods is likely to have occurred by chance.


Publications

False Promises in Medical Imaging AI? Assessing Validity of Outperformance Claims

Christodoulou E, Reinke A, Andrè P, Godau P, Kalinowski P, Houhou R, Erkan S, Sudre C, Burgos N, Boutaj S, Loizillon S, Solal M, Cheplygina V, Heitz C, Kozubek M, Antonelli M, Rieke N, Gilson A, Mayer L, Tizabi M, Cardoso M, Simpson A, Kopp-Schneider A, Varoquaux G, Colliot O, Maier-Hein L - arXiv - 2025



Helmholtz Imaging spinning wheel

Please wait, your data is processed