Graph Laplacian for spectral clustering and seeded image segmentation

Abstract

I mage segmentation is an essential tool to enhance the ability of computer systems to efficiently perform elementary cognitive tasks such as detection, recognition and tracking. In this thesis we concentrate on the investigation of two fundamental topics in the context of image segmentation: spectral clustering and seeded image segmentation. We introduce two new algorithms for those topics that, in summary, rely on Laplacian-based operators, spectral graph theory, and minimization of energy functionals. The effectiveness of both segmentation algorithms is verified by visually evaluating the resulting partitions against state-of-the-art methods as well as through a variety of quantitative measures typically employed as benchmark by the image segmentation community. Our spectral-based segmentation algorithm combines image decomposition, similarity metrics, and spectral graph theory into a concise and powerful framework. An image decomposition is performed to split the input image into texture and cartoon components. Then, an affinity graph is generated and weights are assigned to the edges of the graph according to a gradient-based inner-product function. From the eigenstructure of the affinity graph, the image is partitioned through the spectral cut of the underlying graph. Moreover, the image partitioning can be improved by changing the graph weights by sketching interactively. Visual and numerical evaluation were conducted against representative spectral-based segmentation techniques using boundary and partition quality measures in the well-known BSDS dataset. vii Unlike most existing seed-based methods that rely on complex mathematical formulations that typically do not guarantee unique solution for the segmentation problem while still being prone to be trapped in local minima, our segmentation approach is mathematically simple to formulate, easy-to-implement, and it guarantees to produce a unique solution. Moreover, the formulation holds an anisotropic behavior, that is, pixels sharing similar attributes are preserved closer to each other while big discontinuities are naturally imposed on the boundary between image regions, thus ensuring better fitting on object boundaries. We show that the proposed approach significantly outperforms competing techniques both quantitatively as well as qualitatively, using the classical "GrabCut" dataset from Microsoft as a benchmark. While most of this research concentrates on the particular problem of segmenting an image, we also develop two new techniques to address the problem of image inpainting and photo colorization. Both methods couple the developed segmentation tools with other computer vision approaches in order to operate properly.

On the Graph Laplacian for Spectral Image Segmentation and Energy Minimization on Graphs

“…To accomplish the numerical evaluation, we make use of Success Rate (Precision) and F-Score measures [3] on the well-known Berkeley Segmentation Dataset (BSDS), which provides 300 natural images with their human-drawn ground-truth segmentations. For a few illustrations and the measurements accomplished on BSDS dataset, see Figure 2.…”

“…We also propose a novel energy minimization clustering technique, first reported in [3,5], that addresses many of the undesirable traits present in state-of-the-art methods such as non-uniqueness of solution to the associated optimization problem, use of computationally costly tools and the absence of an accurate and well-behaved (smoother) solution. The new approach, called Laplacian Coordinates (LC), guarantees uniqueness of solution for the segmentation problem, presenting anisotropic behavior to ensure contour adherence on image boundaries.…”

“…Moreover, the method allows for user intervention while leading to smoother and accurate solutions. Other important characteristic of Laplacian Coordinates is that the minimizer of the cost function is given by the solution of a constrained system of linear equations, making the algorithm quite simple to be used and coded [3].…”

“…In this context, the use of interact mechanisms to properly settle Laplacian-based operators on image graph representations have proven to be an effective alternative [2,3].…”

“…This step is performed by combining the texture component T initially generated in the Cartoon-Texture decomposition stage with a recent technique of harmonic analysis [24] aiming at incorporating the remaining high-level oscillatory information into the spectral cut process. See [3] for details. …”