Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines

Rogers, Thomas W.; Jaccard, Nicolas; Protonotarios, Emmanouil D.; Ollier, James; Morton, Edward J.; Griffin, Lewis D.

doi:10.1109/ccst.2016.7815717

Cited by 25 publications

(26 citation statements)

References 17 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This includes: object volume scaling by jointly scaling the in-plane area and the object attenuation; object density scaling by scaling the object attenuation; object flips, formation of composite threat objects; addition of noise; and varying the background appearance. More recently Rogers et al [67] have introduced a method for magnifying the object according to the depth of the object in the scene. Since most X-ray scanner use a divergent fan-beam the object appears taller as it is moved closer to the source.…”

Section: Threat Image Projection (Tip)mentioning

confidence: 99%

Automated X-ray image analysis for cargo security: Critical review and future promise

Rogers

Jaccard

Morton³

et al. 2017

XST

Self Cite

View full text Add to dashboard Cite

We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs-and securityrelated threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo.

show abstract

Section: Threat Image Projection (Tip)mentioning

confidence: 99%

Automated X-ray image analysis for cargo security: Critical review and future promise

Rogers

Jaccard

Morton³

et al. 2017

XST

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another possibility is that the CNN, when applied to cargo is exploiting projection artefacts which are not present in the staged parcel data. Although the analyses in [21,22] seemed to rule this out. …”

Section: Testing Transfer: Cargo To Parcelsmentioning

confidence: 99%

“…These threats were extracted manually and used for threat image projection as previously described. 21 Images were obtained as dual energies (4MeV and 6MeV) with a spatial resolution of the order of a few mm and 16-bit precision.…”

Section: Examples: Cargo Containers Vs Parcelsmentioning

confidence: 99%

Transferring x-ray based automated threat detection between scanners with different energies and resolution

Caldwell

Ransley

Rogers

et al. 2017

Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies

Self Cite

View full text Add to dashboard Cite

A significant obstacle to developing high performance Deep Learning algorithms for Automated Threat Detection (ATD) in security X-ray imagery, is the difficulty of obtaining large training datasets. In our previous work, we circumvented this problem for ATD in cargo containers, using Threat Image Projection and data augmentation. In this work, we investigate whether data scarcity for other modalities, such as parcels and baggage, can be ameliorated by transforming data from one domain so that it approximates the appearance of another. We present an ontology of ATD datasets to assess where transfer learning may be applied. We define frameworks for transfer at the training and testing stages, and compare the results for both methods against ATD where a common data source is used for training and testing. Our results show very poor transfer, which we attribute to the difficulty of accurately matching the blur and contrast characteristics of different scanners.

show abstract

“…To generate de-novo examples for training an SMT was randomly selected from the TIP library and projected into the background patch. 24 Projecting the same SMT instance into different images results in vastly different appearances due to the translucency property of X-ray images and has recently been shown to be indistinguishable from real threat imagery. 24 The dataset is made more diverse by the injection of realistic variations including translations, intensity scaling and flipping.…”

Section: Data Pre-processing and Augmentationmentioning

confidence: 99%

“…24 Projecting the same SMT instance into different images results in vastly different appearances due to the translucency property of X-ray images and has recently been shown to be indistinguishable from real threat imagery. 24 The dataset is made more diverse by the injection of realistic variations including translations, intensity scaling and flipping. SMT instances were kept disjoint between the training, validation, and testing datasets.…”

Section: Data Pre-processing and Augmentationmentioning

confidence: 99%

A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery

2017

Self Cite

View full text Add to dashboard Cite

Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called Small Metallic Threats (SMTs) hidden amongst legitimate goods inside a cargo container. We trained CNNs from scratch on data produced by a Threat Image Projection (TIP) framework that generates images with realistic variation to robustify performance. We achieved 90% detection of containers that contained a single SMT, while raising 6% false positives on benign containers. The best CNN architecture used the raw high energy image (single-energy) and its logarithm as input channels. Use of the logarithm improved performance, thus echoing studies on human operator performance. However, it is an unexpected result with CNNs.In this work, we (i) investigate methods to exploit material information captured in dual-energy images, and (ii) introduce a new CNN training scheme that generates 'spot-the-difference' benign and threat pairs on-the-fly. To the best of our knowledge, this is the first time that CNNs have been applied directly to raw dual-energy X-ray imagery, in any field. To exploit dual-energy, we experiment with adapting several physics-derived approaches to material discrimination from the cargo literature, and introduce three novel variants. We hypothesise that CNNs can implicitly learn about the material characteristics of objects from the raw dual-energy images, and use this to suppress false positives. The best performing method is able to detect 95% of containers containing a single SMT, while raising 0.4% false positives on benign containers. This is a step change improvement in performance over our prior work.

show abstract

Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines

Cited by 25 publications

References 17 publications

Automated X-ray image analysis for cargo security: Critical review and future promise

Automated X-ray image analysis for cargo security: Critical review and future promise

Transferring x-ray based automated threat detection between scanners with different energies and resolution

A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery

Contact Info

Product

Resources

About