Kornia: an Open Source Differentiable Computer Vision Library for PyTorch

Abstract

This work presents Kornia -an open source computer vision library which consists of a set of differentiable routines and modules to solve generic computer vision problems. The package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by OpenCV, Kornia is composed of a set of modules containing operators that can be inserted inside neural networks to train models to perform image transformations, camera calibration, epipolar geometry, and low level image processing techniques, such as filtering and edge detection that operate directly on high dimensional tensor representations. Examples of classical vision problems implemented using our framework are provided including a benchmark comparing to existing vision libraries.

Recovering compressed images for automatic crack segmentation using generative models

“…( 9), 𝐀 is the motion blurring kernel and ⊙ denotes the operation of filtering clear images with kernel 𝐀. Since the parameters of motion blurring kernels can be estimated from motion blurring images by preprocessing methods [45], we directly give motion blurring kernels and generate different motion blurring images for subsequent experiments using Kornia [46], which is a computer vision library for PyTorch [36]. In Fig.…”

Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification

“…We report R-PSNR and SSIM metrics in this work. For R-PSNR (registered-PSNR), as also investigated in Yin et al (2021), we register the reconstructed image to the ground truth reference data by direction optimization using kornia (Riba et al, 2019). For SSIM we also compute a translation-invariant SSIM score by implementinging complex-wavelet SSIM as originally proposed in Wang & Simoncelli (2005).…”

TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning

“…every time an image is loaded from disk during training. Multiple libraries with support for computer vision data augmentation have appeared in the last years, such as Albumentations [Buslaev et al, 2020], Augmentor [Bloice et al, 2019], Kornia [Riba et al, 2019], or imgaug [Jung et al, 2020]. PyTorch also includes some computer vision transforms, mostly implemented as Pillow wrappers [wiredfool et al, 2016].…”