With the continuous development of space and sensor technologies during the recent 40 years, ocean remote sensing has entered into the Big Data era with typical Five-V (volume, variety, value, velocity, and veracity) characteristics. Ocean remote sensing data archives reach several tens of petabytes, and massive satellite data are acquired worldwide daily. To precisely, efficiently and intelligently mining the useful information submerged in such ocean remote sensing data sets is a big challenge. Deep learning, a powerful technology recently emerging in the machine-learning field, has demonstrated its more significant superiority over traditional physical- or statistical-based algorithms for image information extraction in many industrial-field applications and starts to draw interest in ocean remote sensing applications. In this review paper, we first systematically reviewed two deep learning frameworks that carry out ocean remote sensing image classifications and then presented eight typical applications in ocean internal wave/eddy/oil spill/coastal inundation/sea-ice/green algae/ship/coral reef mapping from different types of ocean remote sensing imagery to show how effective of these deep learning frameworks. Researchers can also readily modify these existing frameworks for information mining of other kinds of remote sensing imagery.