Color-Ratio Maps Enhanced Optical Filter Design and Its Application in Green Pepper Segmentation

Abstract

There is a growing demand for developing image sensor systems to aid fruit and vegetable harvesting, and crop growth prediction in precision agriculture. In this paper, we present an end-to-end optimization approach for the simultaneous design of optical filters and green pepper segmentation neural networks. Our optimization method modeled the optical filter as one learnable neural network layer and attached it to the subsequent camera spectral response (CSR) layer and segmentation neural network for green pepper segmentation. We used not only the standard red–green–blue output from the CSR layer but also the color-ratio maps as additional cues in the visible wavelength and to augment the feature maps as the input for segmentation. We evaluated how well our proposed color-ratio maps enhanced optical filter design methods in our collected dataset. We find that our proposed method can yield a better performance than both an optical filter RGB system without color-ratio maps and a raw RGB camera (without an optical filter) system. The proposed learning-based framework can potentially build better image sensor systems for green pepper segmentation.

Wide-Angular Tolerance Optical Filter Design and Its Application to Green Pepper Segmentation

“…Although their approach had compelling results under specific illumination sources, it is redistricted under controlled environments and vulnerable to intense outdoor sunlight. Yu et al [ 3 , 4 ] proposed an optical-filter-based method for distinguishing near-color objects, e.g., green leaves and peppers. Their design method does not require any specific neural network; a latest neural network, such as that of [ 5 ], can be applied, and better performance is expected by using an optical filter for distinguishing near-color objects.…”