Go back    Home Solutions challengeR: Methods and open-source toolkit for analyzing and visualizing challenge results

challengeR: Methods and open-source toolkit for analyzing and visualizing challenge results (challengeR)

Authors: Wiesenfarth, M., Reinke, A., Landman, B.A., Eisenmann, M., Aguilera Saiz, L., Cardoso, M.J., Maier-Hein, L. and Kopp-Schneider, A., Kavur A.E

Keywords: Computer science, Image processing, Machine learning, Validation, (Ranking) Uncertainties

challengeR is a tool for analyzing and visualizing challenge results in the field of image analysis and beyond.

Challenges have become the de facto standard for benchmarking image analysis algorithms. While the number of challenges is steadily increasing, surprisingly little effort has been invested in ensuring high quality design, execution and reporting for these international competitions. Specifically, results analysis and visualization in the event of uncertainties have been given almost no attention in the literature.

Given these shortcomings, challengeR aims to catalyze fast and widespread adoption of comprehensive analysis & visualization of competition results. challengeR offers an intuitive way to gain important insights into the relative and absolute performance of algorithms, including ranking uncertainties, which cannot be revealed by commonly applied visualization techniques.


Publications

Methods and open-source toolkit for analyzing and visualizing challenge results

Wiesenfarth M, Reinke A, Landman B, Eisenmann M, Saiz L, Cardoso M, Maier-Hein L, Kopp-Schneider A - Scientific Reports - 2021


challengeR: Methods and open-source toolkit for analyzing and visualizing challenge results Image
License
GPLv2 or any later version

Helmholtz Imaging spinning wheel

Please wait, your data is processed

Please wait, your data is processed