From RGB to Spectrum for Natural Scenes via Manifold-Based Mapping

Jia, Yan; Zheng, Yinqiang; Gu, Lin; Subpa-asa, Art; Lam, Antony; Sato, Yoichi; Sato, Iwao

doi:10.1109/iccv.2017.504

Cited by 89 publications

(43 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jia et. al (Jia et al 2017) considered the manifold structure of HSIs in a low-dimensional space. Recently, most methods turn to learning a deep mapping function from the RGB image to an HSI.…”

Section: Related Workmentioning

confidence: 99%

“…Different from convnetioanl HSIs super-resolution (Mei et al 2017;Zhang et al 2018a) that directly improves the spatial resolution of a given HSI, spectral super-resolution (SSR) (Arad and Ben-Shahar 2016;Xiong et al 2017) adopts an alternative way and attempts to produce an HSR HSI by increasing the spectral resolution of a given RGB image with satisfactory spatial resolution. Early SSR methods (Arad and Ben-Shahar 2016;Aeschbacher, Wu, and Timofte 2017;Jia et al 2017) often formulate SSR as a linear inverse problem, and exploit the inherent low-level statistic of HSR HSIs as priors. However, due to the limited expressive capacity of their handcrafted prior models, these methods fail to well generalize to challenging cases.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Zhang

Lang

Wang

et al. 2020

AAAI

View full text Add to dashboard Cite

Spectral super-resolution (SSR) aims at generating a hyperspectral image (HSI) from a given RGB image. Recently, a promising direction is to learn a complicated mapping function from the RGB image to the HSI counterpart using a deep convolutional neural network. This essentially involves mapping the RGB context within a size-specific receptive field centered at each pixel to its spectrum in the HSI. The focus thereon is to appropriately determine the receptive field size and establish the mapping function from RGB context to the corresponding spectrum. Due to their differences in category or spatial position, pixels in HSIs often require different-sized receptive fields and distinct mapping functions. However, few efforts have been invested to explicitly exploit this prior.To address this problem, we propose a pixel-aware deep function-mixture network for SSR, which is composed of a new class of modules, termed function-mixture (FM) blocks. Each FM block is equipped with some basis functions, i.e., parallel subnets of different-sized receptive fields. Besides, it incorporates an extra subnet as a mixing function to generate pixel-wise weights, and then linearly mixes the outputs of all basis functions with those generated weights. This enables us to pixel-wisely determine the receptive field size and the mapping function. Moreover, we stack several such FM blocks to further increase the flexibility of the network in learning the pixel-wise mapping. To encourage feature reuse, intermediate features generated by the FM blocks are fused in late stage, which proves to be effective for boosting the SSR performance. Experimental results on three benchmark HSI datasets demonstrate the superiority of the proposed method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Zhang

Lang

Wang

et al. 2020

AAAI

View full text Add to dashboard Cite

show abstract

“…Some methods use image priors such as Yan et al introduce category and location information into the network [48]. In [49], the authors adopt dictionary learning and in [50] manifold learning is used for the HSI generating. The most common RGB to HSI reconstruction methods are based on CNN [51,52].…”

Section: B Spectral Super-resolutionmentioning

confidence: 99%

“…Several methods [57,58] use a GAN discriminator to ensure the reality of generated HSI. These methods are all based on natural HSI data such as ICVL [49,50], Bgu HS [51,53], and Arad HS [54,59]. They mainly generate 31 bands from RGB images, which different from the remote sensing HSI generating to 150 bands.…”

Section: B Spectral Super-resolutionmentioning

confidence: 99%

Hyperspectral Remote Sensing Imagery Generation From RGB Images Based on Joint Discrimination

Liu

Lei

Shi

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Sparse reconstruction methods exploit the sparsity of spectra, recover spectra for every pixels using an learnt overcomplete dictionary [23,24,25]. Based on sparse reconstruction, some literature has proved that the reconstruction accuracy would benefit from combining the prior of local manifold structure [16,26]. To further improve reconstruction quality, the convolutional neural network based learning methods which utilize the spatial information of images have begun to appear [27,28].…”

Section: Related Workmentioning

confidence: 99%

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Xie

et al. 2019

Sensors

View full text Add to dashboard Cite

Multispectral filter array (MSFA)-based imaging is a compact, practical technique for snapshot spectral image capturing and reconstruction. The imaging and reconstruction quality is highly influenced by the spectral sensitivities and spatial arrangement of channels on MSFAs, and the used reconstruction method. In order to design a MSFA with high imaging capacity, we propose a sparse representation based approach to optimize spectral sensitivities and spatial arrangement of MSFAs. The proposed approach first overall models the various errors associated with spectral reconstruction, and then uses a global heuristic searching method to optimize MSFAs via minimizing the estimated error of MSFAs. Our MSFA optimization method can select filters from off-the-shelf candidate filter sets while assigning the selected filters to the designed MSFA. Experimental results on three datasets show that the proposed method is more efficient, flexible, and can design MSFAs with lower spectral construction errors when compared with existing state-of-the-art methods. The MSFAs designed by our method show better performance than others even using different spectral reconstruction methods.

show abstract

From RGB to Spectrum for Natural Scenes via Manifold-Based Mapping

Cited by 89 publications

References 22 publications

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Hyperspectral Remote Sensing Imagery Generation From RGB Images Based on Joint Discrimination

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Contact Info

Product

Resources

About