Robust Detection Outcome: A Metric for Pathology Detection in Medical Images
Abstract
Detection of pathologies is a fundamental task in medical imaging and the evaluation of algorithms that can perform this task automatically is crucial. However, current object detection metrics for natural images do not reflect the specific clinical requirements in pathology detection sufficiently. To tackle this problem, we propose Robust Detection Outcome (RoDeO); a novel metric for evaluating algorithms for pathology detection in medical images, especially in chest X-rays. RoDeO evaluates different errors directly and individually, and reflects clinical needs better than current metrics. Extensive evaluation on the ChestX-ray8 dataset shows the superiority of our metrics compared to existing ones. We released the code at this https URL and published RoDeO as pip package (rodeometric).
Find out more
Installation
RoDeO is available as a python package for python 3.7+ as rodeometric. To install, simply install it with pip:
python -m pip install rodeometric
Usage
import numpy as np
from rodeo import RoDeO
# Init RoDeO with two classes
rodeo = RoDeO(class_names=['a', 'b'])
# Add some predictions and targets
pred = [np.array([[0.1, 0.1, 0.2, 0.1, 0.0],
[0.0, 0.3, 0.1, 0.1, 1.0],
[0.2, 0.2, 0.1, 0.1, 0.0]])]
target = [np.array([[0.0, 0.0, 0.1, 0.1, 0.0],
[0.0, 0.2, 0.1, 0.1, 1.0]])]
rodeo.add(pred, target)
# Compute the score
score = rodeo.compute()
for key, val in score.items():
print(f'{key}: {val}')
Felix Meissen
Ph.D. Student for AI in Medicine
My research interests include anomaly detection, object detection, and anything related to medical images