Fluorescence microscopy is a key driver of discoveries in the life-sciences, with observable phenomena being limited by the optics of the microscope, the chemistry of the fluorophores, and the maximum photon exposure tolerated by the sample. These limits necessitate trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. In this work we show how image restoration based on deep learning extends the range of biological phenomena observable by microscopy. On seven concrete examples we demonstrate how microscopy images can be restored even if 60-fold fewer photons are used during acquisition, how near isotropic resolution can be achieved with up to 10-fold under-sampling along the axial direction, and how tubular and granular structures smaller than the diffraction limit can be resolved at 20-times higher frame-rates compared to state-of-the-art methods. All developed image restoration methods are freely available as open source software in Python, FIJI, and KNIME.

Fluorescence microscopy is a key driver of discoveries in the life-sciences, with observable phenomena being limited by the optics of the microscope, the chemistry of the fluorophores, and the maximum photon exposure tolerated by the sample. These limits necessitate tradeoffs between imaging speed, spatial resolution, light exposure, and imaging depth. In this work we show how deep learning enables biological observations beyond the physical limitations of microscopes. On seven concrete examples we illustrate how microscopy images can be restored even if 60-fold fewer photons are used during acquisition, how isotropic resolution can be achieved even with a 10-fold under-sampling along the axial direction, and how diffraction-limited structures can be resolved at 20-times higher frame-rates compared to state-of-the-art methods. All developed image restoration methods are freely available as open source software.

Graphics processing units (GPU) allow image processing at unprecedented speed. We present CLIJ, a Fiji plugin enabling end-users with entry level experience in programming to benefit from GPU-accelerated image processing. Freely programmable workflows can speed up image processing in Fiji by factor 10 and more using high-end GPU hardware and on affordable mobile computers with built-in GPUs.Modern microscopy generates staggering amounts of multidimensional image data that place increasing demands on processing flexibility and efficiency. One way to speed up image processing is to exploit the parallel processing capabilities of graphics processing units (GPU).Recently, GPU-acceleration was used in specific image processing tasks such as reconstruction 1,2 , image quality determination 3 , image restoration 4 , segmentation 5 and visualisation 6 . However, in these tools, GPU code is fulfilling one specific purpose and is not intended to be reused in other contexts. By contrast, most common image processing tasks are solved by building flexible workflows consisting of simple operations in widely used tools such as ImageJ 7 and Fiji 8 . Most of these operations were however programmed at a time when GPUs were not commonly used for general purpose processing. Therefore, typical workflows consisting of core ImageJ operations do not take advantage of GPUs. To address this issue we developed a flexible and reusable platform for GPU-acceleration in Fiji.

Convolutional neural networks have become state-of-the-art in a wide range of image recognition tasks. The interpretation of their predictions, however, is an active area of research. Whereas various interpretation methods have been suggested for image classification, the interpretation of image segmentation still remains largely unexplored. To that end, we propose seg-grad-cam, a gradient-based method for interpreting semantic segmentation. Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic segmentation.

Graphics processing units (GPU) allow image processing at unprecedented speed. We present CLIJ, a Fiji plugin enabling end-users with entry level experience in programming to benefit from GPU-accelerated image processing. Freely programmable workflows can sped up image processing in Fiji by factor 10 and more using high-end GPU hardware and on affordable mobile computers with built-in GPUs.

Owing to decreased costs over time, the genomes of an increasing number of species are being sequenced. However, chromosome-scale genome assembly is affected by scaffolding errors, resulting in incorrect chromosome sizes. Here, we present KICS, a semi-automated and cost-efficient approach for examining the validity of the assembly by estimating relative chromosome sizes from karyotype images. The method employs threshold-based image segmentation and uses the extracted chromosome areas as proxies for the actual chromosome sizes. A strong correlation between chromosomal area and DNA confirmed the suitability of our approach, as assessed from karyotype images of multiple species. KICS can be applied to microchromosomes, and we identified assembly errors made by HiC sequencing in the horseshoe bat genome. By using the human genome as a reference, for which telomere-to-telomere data are available, we estimate an error of our tool of ∼6Mb. We foresee that KICS will be routinely used as an inexpensive and intuitive tool to validate the de novo assembly of new genomes.

Highly contiguous genome assemblies are essential for genomic research. Chromosome-scale assembly is feasible with the modern sequencing techniques in principle, but in practice, scaffolding errors frequently occur, leading to incorrect number and sizes of chromosomes. Relating the observed chromosome sizes from karyotype images to the generated assembly scaffolds offers a method for detecting these errors.Here, we present KICS, a semi-automated approach for estimating relative chromosome sizes from karyotype images and their subsequent comparison to the corresponding assembly scaffolds. The method relies on threshold-based image segmentation and uses the computed areas of the chromosome-related connected components as a proxy for the actual chromosome size. We demonstrate the validity and practicality of our approach by applying it to karyotype images of humans and various amphibians, birds, fish, insects, mammals, and plants. We found a strong linear relationship between pixel counts and the DNA content of chromosomes. Averaging estimates from eight human karyotype images, KICS predicts most of the chromosome sizes within an error margin of just 6 Mb.Our method provides additional means of validating genome assemblies at low costs. An interactive implementation of KICS is available at https://github.com/mpicbg-csbd/napari-kics.