Computational strategies for understanding underwater optical image datasets

Abstract

A fundamental problem in autonomous underwater robotics is the high latency between the capture of image data and the time at which operators are able to gain a visual understanding of the survey environment. Typical missions can generate imagery at rates hundreds of times greater than highly compressed images can be transmited acoustically, delaying that understanding until after the vehicle has been recovered and the data analyzed. While automated classification algorithms can lessen the burden on human annotators after a mission, most are too computationally expensive or lack the robustness to run in situ on a vehicle. Fast algorithms designed for mission-time performance could lessen the latency of understanding by producing low-bandwidth semantic maps of the survey area that can then be telemetered back to operators during a mission.This thesis presents a lightweight framework for processing imagery in real time aboard a robotic vehicle. We begin with a review of pre-processing techniques for correcting illumination and attenuation artifacts in underwater images, presenting our own approach based on multi-sensor fusion and a strong physical model. Next, we construct a novel image pyramid structure that can reduce the complexity necessary to compute features across multiple scales by an order of magnitude and recommend features which are fast to compute and invariant to underwater artifacts. Finally, we implement our framework on real underwater datasets and demonstrate how it can be used to select summary images for the purpose of creating low-bandwidth semantic maps capable of being transmitted acoustically.

Visual summaries for low-bandwidth semantic mapping with autonomous underwater vehicles

“…Our approach is unique because it demonstrates that a simple, forward-mapped pattern vocabulary can be used to produce meaningful results without relying on complex descriptors that must be first learned and then subsequently quantized into a dictionary [23]. Furthermore, this work augments the visual summary literature under the assumption that summary images, navigation data, and classification masks can all be transmitted back at some rate during a mission.…”

“…However, due to ship motion and unpredictable topography, there is a large variation in the altitude, much more than an AUV usually might experience. We truncated the data to only include about 2500 images captured at altitudes between 1.5 and 4 meters, approximately the range within which additive scattering can be neglected [23].…”

“…For each image, we computed 1000 keypoints and quantized each to one of 14 binary QuAHOG patterns [23]. A global histogram was then computed for the entire image.…”

Extremely Low and Variable Bandwidth Image Compression with Region of Interest Applied to Real Time Underwater Robotic Interventions

“…According to Kaeli [22] in his MIT PhD thesis, the current standard state-of-the-art image compression algorithm Joint Photographic Experts Group 2000 (JPEG-2000) [23] [9] DCT and JPEG 16 kbps ASIMOV [10] 2 Frames/s 30 kps Japan [9] MPEG4 10Frames/s Hong [11] DWT 30Frames/s Khamene [12] DWT 128x128 pixel frames Konstantinos [13] DCT 150 kbps Eastwwod [14] DCT and JPEG EPIC Walter [15] WDR/DWT ROI Pearlman [16] DWT Low complexity Murphy [17] Telemetry Acoustic channel Zheng [18] Telemetry Acoustic channel Senapati [19] WBTC DCT/DWT Zhang [20] Hybrid wavelets Directional filter banks Mohammed [21] SPIHT Reducing PAPR Kaeli [22] JPEG-2000 Larger packets employs variable compression rates using progressive encoding, meaning that a compressed image can be transmitted in pieces or packets that independently add finer detail to the received image. This is particularly well suited to underwater applications where acoustic channels are noisy and subject to high packet loss, however, JPEG-2000 [23] is optimized for larger packets that are unrealistic for underwater acoustic or radio frequency (RF) transmissions [22].…”

“…Kaeli [22] also compares Joint Photographic Experts Group (JPEG) [24], Joint Photographic Experts Group 2000 (JPEG-2000) [23] and Set Partitioning in Hierarchical Trees (SPIHT) [25]. JPEG [24] is a common example of a lossy compression format which uses the DCT (Discrete Cosine Transform) for each 8x8 block to achieve roughly 10:1 compression without major perceptual changes in the image.…”