Manifold Alignment Aware Ants: A Markovian Process for Manifold Extraction

Abstract

The presence of manifolds is a common assumption in many applications, including astronomy and computer vision. For instance, in astronomy, low-dimensional stellar structures, such as streams, shells, and globular clusters, can be found in the neighborhood of big galaxies such as the Milky Way. Since these structures are often buried in very large data sets, an algorithm, which can not only recover the manifold but also remove the background noise (or outliers), is highly desirable. While other works try to recover manifolds either by pushing all points toward manifolds or by downsampling from dense regions, aiming to solve one of the problems, they generally fail to suppress the noise on manifolds and remove background noise simultaneously. Inspired by the collective behavior of biological ants in food-seeking process, we propose a new algorithm that employs several random walkers equipped with a local alignment measure to detect and denoise manifolds. During the walking process, the agents release pheromone on data points, which reinforces future movements. Over time the pheromone concentrates on the manifolds, while it fades in the background noise due to an evaporation procedure. We use the Markov chain (MC) framework to provide a theoretical analysis of the convergence of the algorithm and its performance. Moreover, an empirical analysis, based on synthetic and real-world data sets, is provided to demonstrate its applicability in different areas, such as improving the performance of t-distributed stochastic neighbor embedding (t-SNE) and spectral clustering using the underlying MC formulas, recovering astronomical low-dimensional structures, and improving the performance of the fast Parzen window density estimator.

Swarm Intelligence-based Extraction and Manifold Crawling Along the Large-Scale Structure

“…hood radius centered at the different points, the effect of modifying the learning rate 𝜂 (refer to equation ( 13)) should also be specified. Mohammadi et al (2022) have shown that selecting a smaller value for 𝜂 leads to less variation in the results across different runs but requires a larger number of iterations for one run. This highlights the trade-off effect between the number of iterations specified for the random walk and between the quality of the results.…”

“…The deposited pheromone amount can be interpreted as a measure of faintness of the structures, and thresholding is used to extract the detected structures. EM3A (Mohammadi et al 2022): Evolutionary Manifold Alignment Aware Agents moves particles belonging to the manifolds towards their central axis, thus further enhancing the contrast between under-dense and over-dense regions in the data. This algorithm together with LAAT is said to "denoise" the data, as in it uncovers the manifolds embedded within their scattered or noisy environments.…”

“…To limit this dependence on the radius, EM3A implements methods from evolutionary game theory by representing a range of calculated radii as evolutionary strategies and afterwards trying to compute the "fittest" strategy between them. We refer the reader to Canducci et al (2022a) and Mohammadi et al (2022) for a more detailed description of the algorithm.…”

“…The effect of the parameters on changes in the random walks have been studied in the initial propositions of LAAT and EM3A, namely the work presented in Taghribi et al (2022a) and Mohammadi et al (2022). We therefore recount briefly the results of the parameter analysis presented in those works.…”