On-Board Ship Detection in Micro-Nano Satellite Based on Deep Learning and COTS Component

Abstract

Micro-nano satellites have provided a large amount of remote sensing images for many earth observation applications. However, the hysteresis of satellite-ground mutual communication of massive remote sensing images and the low efficiency of traditional information processing flow have become the bottlenecks for the further development of micro-nano satellites. To solve this problem, this paper proposes an on-board ship detection scheme based on deep learning and Commercial Off-The-Shelf (COTS) component, which can be used to achieve near real-time on-board processing by micro-nano satellite computing platform. The on-board ship detection algorithm based on deep learning consists of a feature extraction network, Region Proposal Network (RPN) with square anchors, Global Average Pooling (GAP), and Bigger-Left Non-Maximum Suppression (BL-NMS). With the help of high performance COTS components, the proposed scheme can extract target patches and valuable information from remote sensing images quickly and accurately. A ground demonstration and verification system is built to verify the feasibility and effectiveness of our scheme. Our method achieves the performance with 95.9% recall and 80.5% precision in our dataset. Experimental results show that the scheme has a good application prospect in micro-nano satellites with limited power and computing resources.The traditional processing flow of ship detection from remote sensing images consists of mission planning, satellite imaging, satellite-earth data transmission, and ground processing. The ground processing scheme has time delay for ground users, and the satellite-earth data transmission system is under tremendous pressure. There are two bottlenecks for the further development of micro-nano satellites, i.e., the hysteresis of satellite-ground mutual communication of massive remote sensing images and the low efficiency of traditional information processing. On-board processing in micro-nano satellites is an effective way to improve the response speed to emergencies and provide immediate products for users. Figure 1 shows the comparison between on-board processing flow and the traditional ground processing flow. On-board image processing analyzes the remote sensing images directly on the satellite, and then only transmits the effective information to the ground receiver immediately. Compared with the traditional ground processing flow of ship detection, the advantages of on-board processing flow of ship detection are as follows: (1) the gigabyte level images are simplified to megabyte level target patches and information by on-board ship detection. It can reduce the compression, transmission and storage pressure for emergencies. (2) It simplifies the process of ground equipment and directly delivers the concerned information to users from the satellite. (3) Users can acquire information in near real-time while traditional processing flow needs several hours or days.

Counting Cattle in UAV Images—Dealing with Clustered Animals and Animal/Background Contrast Changes

“…Although the algorithm described in this article was developed having cattle counting in mind, the methodology can be adapted to some other applications such as ship detection [27], tent detection in refugee camps [28], among others.…”

On-Board Ship Detection for Medium Resolution Optical Sensors

“…While in [ 15 , 16 ] the authors exploited across-band analysis toward static/in-transit ships using a multispectral camera. Both these types of cameras could be used from airborne [ 7 , 17 ] as well as space-borne [ 18 , 19 , 20 , 21 ] systems during the daytime. These systems can provide very good accuracy in detection of ships and also enable classification to a certain extent of vessels into subcategories.…”

“…Although a real-time on-board ship detection method for an optical remote sensing satellite with 15 m resolution is proposed in [ 32 ], the implementations and results presented are for ground simulations only. Yao proposed [ 19 ] the use of COTS components to apply deep learning for ship detection for micro-nano satellites with a tradeoff between accuracy and model size. Yu proposed ship identification in the wavelet domain with Ostu’s threshold based on multi-scale salience enhancement [ 20 ].…”

On-Board Fast and Intelligent Perception of Ships With the “Jilin-1” Spectrum 01/02 Satellites

“…Therefore, relevant algorithm design should be carried out according to the actual requirements. Reference [14] combined constraints such as the length and width of the ship and realized the ship detection using a multispectral Sentinel-2 image and compared the results with the AIS information. Although high-quality standard images are still used in the literature, the algorithm complexity is more suitable for on-board application.…”