Membranous nephropathy classification using microscopic hyperspectral imaging and tensor patch-based discriminative linear regression

2021

Sensors

Spectral reconstruction (SR) algorithms attempt to recover hyperspectral information from RGB camera responses. Recently, the most common metric for evaluating the performance of SR algorithms is the Mean Relative Absolute Error (MRAE)—an ℓ1 relative error (also known as percentage error). Unsurprisingly, the leading algorithms based on Deep Neural Networks (DNN) are trained and tested using the MRAE metric. In contrast, the much simpler regression-based methods (which actually can work tolerably well) are trained to optimize a generic Root Mean Square Error (RMSE) and then tested in MRAE. Another issue with the regression methods is—because in SR the linear systems are large and ill-posed—that they are necessarily solved using regularization. However, hitherto the regularization has been applied at a spectrum level, whereas in MRAE the errors are measured per wavelength (i.e., per spectral channel) and then averaged. The two aims of this paper are, first, to reformulate the simple regressions so that they minimize a relative error metric in training—we formulate both ℓ2 and ℓ1 relative error variants where the latter is MRAE—and, second, we adopt a per-channel regularization strategy. Together, our modifications to how the regressions are formulated and solved leads to up to a 14% increment in mean performance and up to 17% in worst-case performance (measured with MRAE). Importantly, our best result narrows the gap between the regression approaches and the leading DNN model to around 8% in mean accuracy.

Section: Introductionmentioning

confidence: 99%

On the Optimization of Regression-Based Spectral Reconstruction

2021

Sensors

“…This almost ubiquitous imaging practice sacrifices a significant portion of spectral information for faster, light weight and more affordable image captures. On the other hand, there are hyperspectral cameras that measure the light radiances with high “spectral” resolution, and these devices are found to be more useful (compared with RGB cameras) for many industrial applications including, remote sensing, 2,3 medical imaging, 4,5 anomaly detection, 6,7 artwork conservation, 8,9 and food processing 10,11 . However, the high price tag of hyperspectral imaging technologies, as well as the trade‐offs among spatial, spectral and temporal resolutions limit their usefulness.…”

Section: Introductionmentioning

confidence: 99%

“…radiances with high "spectral" resolution, and these devices are found to be more useful (compared with RGB cameras) for many industrial applications including, remote sensing, 2,3 medical imaging, 4,5 anomaly detection, 6,7 artwork conservation, 8,9 and food processing. 10,11 However, the high price tag of hyperspectral imaging technologies, as well as the trade-offs among spatial, spectral and temporal resolutions limit their usefulness.…”

mentioning

confidence: 99%

An investigation on worst‐case spectral reconstruction from RGB images via Radiance Mondrian World assumption

Color Research & Application

2023

Spectral reconstruction (SR) algorithms recover hyperspectral measurements from RGB camera responses. Statistical models at different levels of complexity are used to solve the SR problem-from the simplest closed-form regression, to sparse coding, to the complex deep neural networks (DNN). Recently, these methods were benchmarked based on the mean performance of the models and on a fixed set of real-world scenes, suggesting that more complex (more non-linear) models generally deliver better SR. In this paper, we investigate the relative performances of these models in terms of a real-world worst-case imaging condition called the Radiance Mondrian World (RMW) assumption. Under the RMW, testing hyperspectral images are composed of randomlyarranged and overlapping rectangular patches, where each patch is filled with one random radiance spectrum uniformly sampled from the convex closure of all natural radiances (i.e., all spectra in the concerned hyperspectral image dataset). Surprisingly, we show that all compared algorithms-regardless of their model complexity-degrade to broadly the same level of performance on our RMW testing set. Further, by retraining all models with an RMW training set, we show that increasing model complexity also does not help learning better SR mappings from the RMW images. That is, using simple regression is as good as using a DNN. This similarity of performance is also shown to hold for images adhering to the conventional Mondrian World assumption (random reflectances lit by a single, per scene, randomly selected light source).

“…However, compared with RGBs, the spectrum (from which the RGB is formed [1]) conveys significantly more information about an object's material properties. Consequently, in many computer vision tasks, it is useful to deploy hyperspectral cameras where finely-sampled light spectrum is captured at every pixel of the scene, including remote sensing [2][3][4][5], anomaly detection [6][7][8][9], medical imaging [10,11], food processing [12][13][14] and artwork preservation [15,16].…”

Section: Introductionmentioning

confidence: 99%

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

2023

Sensors

Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.