Light Field Image Super-Resolution via Mutual Attention Guidance

Abstract

Deep learning-based methods have prompted light field image super-resolution to achieve significant progress. However, most of them ignore aligning different sub-aperture features of light field image before aggregation, resulting in sub-optimal super-resolution results. We aim to propose an efficient feature alignment method for sub-aperture feature aggregation. To this end, we develop a mutual attention mechanism for sub-aperture feature alignment and propose a mutual attention guidance block (MAG). MAG achieves the mutual attention mechanism between the center feature and surrounding feature with the center attention guidance module (CAG) and the surrounding attention guidance module (SAG). CAG aligns the center-view feature with the surrounding-view feature and generates the refined surrounding-view feature, while SAG aligns the refined surrounding-view feature with the original surrounding-view feature to implement bidirectional center-view, and surrounding view features alignment. Based on MAG, we build a Light Field Mutual Attention Guidance Network (LF-MAGNet) constructed by multiple MAGs in a cascade manner. Experiments are performed on commonly-used light field image super-resolution benchmarks. Our method achieves superior qualitative and quantitative results to other state-of-the-art methods, which demonstrate the effectiveness of our LF-MAGNet.INDEX TERMS Light-field image super-resolution, Visual attention mechanism, Feature alignment.