Construction of a Machine Learning Dataset through Collaboration: The RSNA 2019 Brain CT Hemorrhage Challenge

Abstract

I ntracranial hemorrhage is a potentially life-threatening problem that has many direct and indirect causes. Accuracy in diagnosing the presence and type of intracranial hemorrhage is a critical part of effective treatment. Diagnosis is often an urgent procedure requiring review of medical images by highly trained specialists and sometimes necessitating confirmation through clinical history, vital signs, and laboratory examinations. The process is complicated and requires immediate identification for optimal treatment.Intracranial hemorrhage is a relatively common condition that has many causes, including trauma, stroke, aneurysm, vascular malformation, high blood pressure, illicit drugs, and blood clotting disorders (1). Neurologic consequences can vary extensively from headache to death depending upon the size, type, and location of the hemorrhage. The role of the radiologist is to detect the hemorrhage, characterize the type and cause of the hemorrhage, and to determine if the hemorrhage could be jeopardizing critical areas of the brain that might require immediate surgery.While all acute hemorrhages appear attenuated on CT images, the primary imaging features that help radiologists determine the cause of hemorrhage are the location, shape, and proximity to other structures. Intraparenchymal hemorrhage is blood that is located completely within the brain itself. Intraventricular or subarachnoid hemorrhage is blood that has leaked into the spaces of the brain that normally contain cerebrospinal fluid (the ventricles or subarachnoid cisterns, respectively). Extra-axial hemorrhage is blood that collects in the tissue coverings that surround the brain (eg, subdural or epidural subtypes). It is important to note that patients may exhibit more than one type of cerebral hemorrhage, which may appear on the same image or imaging study. Although small hemorrhages are typically less morbid than large hemorrhages, even a small hemorrhage can lead to death if it is in a critical location. Small hemorrhages also may herald future hemorrhages that could be fatal (eg, ruptured cerebral aneurysm). The presence or absence of hemorrhage may guide specific treatments (eg, stroke).Detection of cerebral hemorrhage with brain CT is a popular clinical use case for machine learning (2-5). Many of these early successful investigations were based upon relatively small datasets (hundreds of examinations) from single institutions. Chilamkurthy et al created a diverse brain CT dataset that was selected from 20 geographically distinct centers in India (more than 21 000 unique examinations). This was used to create smaller randomly selected subsets for validation and testing on common acute brain abnormalities (6). The ability for machine learning algorithms to generalize to "real-world" clinical imaging data from disparate institutions is paramount to successful use in the clinical environment.The intent for this challenge was to provide a large multiinstitutional and multinational dataset to help develop machine learning algorithms that ca...

Accurate and Efficient Intracranial Hemorrhage Detection and Subtype Classification in 3D CT Scans with Convolutional and Long Short-Term Memory Neural Networks

“…The Radiological Society of North America (RSNA) 2019 Brain CT Hemorrhage dataset [ 28 ] was built from scratch for the 2019 RSNA Intracranial Hemorrhage Detection challenge held on Kaggle ( ). The dataset is comprised of CT scans from three institutions: Stanford University, Universidade Federal de Sao Paulo and Thomas Jeffereson University Hospital.…”

“…Considering that there are 22 mistakes in common, we have strong reasons to suspect that the ground-truth labels might actually be wrong. As the ground-truth labels are also given by specialists [ 28 ], this is a likely explanation for the high overlap between the mistakes of doctor #2 and those of our CNN model. After seeing the ground-truth labels and making a careful reassessment, our team of doctors found at least 25 ground-truth labels that are wrong and another 5 that are disputable.…”

“…The common format for storing and transmitting medical CT images is DICOM ® (Digital Imaging and Communications in Medicine), which contains metadata alongside the pixel data. The RSNA dataset [ 28 ] contains 16-bit images, meaning that each image is represented on a scale with 65,536 levels of gray. The value of each pixel represents the density of the tissue measured in Hounsfield units (HU).…”

“…The images resulted after applying the three intensity windows recommended by radiologists are combined into a single three-channel image, as shown in Figure 1 . While the original CT slices are formed of pixels, we reduce their spatial dimension to cope with the large training set of over 700 thousand slices [ 28 ]. The CT slices are reduced to pixels using linear interpolation.…”

“…After the initial success of deep learning [ 10 ] in object recognition from images [ 3 , 11 ], deep neural networks have been adopted for a broad range of tasks in medical imaging, ranging from cell segmentation [ 12 ] and cancer detection [ 13 , 14 , 15 , 16 , 17 ] to intracranial hemorrhage detection [ 5 , 8 , 18 , 19 , 20 , 21 , 22 ] and CT/MRI super-resolution [ 23 , 24 , 25 , 26 ]. Since we address the task of intracranial hemorrhage detection, we consider related works that are focused on the same task as ours [ 5 , 6 , 7 , 8 , 18 , 19 , 20 , 21 , 22 , 27 , 28 , 29 , 30 ], as well as works that study intracranial hemorrhage segmentation [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 ].…”