Detection Benchmark

See Detection2 for the currently running Lesion Detection Benchmark.

 

This page describes the VISCERALdetection Benchmark as it was run in 2014-2015.

 

VISCERALdetection Lesion Detection Benchmark

News (November 2014): submission deadline has been extended to match with the Anatomy3 challenge and the ISBI challenge workshop.

Objective

We distribute medical imaging data (CT, MRI, w & w/o contrast enhancement) that contains various lesions in anatomical regions such as the bones, liver, brain, lung, or lymph nodes. There are overall 308 annotated lesions in the data-set.

Task

During the training phase, you will get imaging data and annotations in the form of lesion center position, and for large lesions, annotations that indicate the radius.

During the benchmark phase, we will run your algorithm on new data, and evaluate its detection performance against expert annotations of lesions.

Important Dates

Mid November	Registration opens.
Mid November	Training dataset is available.
March 30th 2015	Submission deadline for results that will be presented at the ISBI event.

→ If you wish to present your results at the ISBI 2015 challenge event: A short abstract/PDF will be required to contribute to the challenge proceedings. Details on the submission method are given on the Workshop page.

How to participate

The benchmark is run via virtual machines. That is, we provide you with a powerful virtual machine, on which you make your algorithm run. You can use the VMs for training and experimenting, or download the training data and work with it locally. In any case, for the benchmark, i.e., the evaluation of your algorithm on a test set, you have to get it to run on the VM. We will run your algorithm in a specified way, using a few scripts that we will publish for your guidance.

Documents and Resources

Register for a benchmark account at the VISCERAL registration website.  Choose "VISCERALdetection" and your choice of operating system (Windows, Ubuntu, openSUSE,centOS) for your virtual machine (VM).

The participants have access through the the VISCERAL registration website to a data handling tutorial. It explains how to access the lesion annotations and uploading of the data.   

The Detection Benchmark Specifications can be downloaded (currently v1.2 of 20141118)

The latest version of the EvaluateSegmentation tool can always be downloaded from: https://github.com/codalab/EvaluateSegmentation

Questions?

If you have any questions, please feel free to get in touch: [email protected]