Convergence versus Divergence Behaviors of Asynchronous Iterations, and Their Applications in Concrete Situations

Abstract

Asynchronous iterations have long been used in distributed computing algorithms to produce calculation methods that are potentially faster than a serial or parallel approach, but whose convergence is more difficult to demonstrate. Conversely, over the past decade, the study of the complex dynamics of asynchronous iterations has been initiated and deepened, as well as their use in computer security and bioinformatics. The first work of these studies focused on chaotic discrete dynamical systems, and links were established between these dynamics on the one hand, and between random or complex behaviours in the sense of the theory of the same name. Computer security applications have focused on pseudo-random number generation, hash functions, hidden information, and various security aspects of wireless sensor networks. At the bioinformatics level, this study of complex systems has allowed an original approach to understanding the evolution of genomes and protein folding. These various contributions are detailed in this review article, which is an extension of the paper “An update on the topological properties of asynchronous iterations” presented during the Sixth International Conference on Parallel, Distributed, GPU and Cloud Computing (Pareng 2019).

A Review of the Parallelization Strategies for Iterative Algorithms

“…It is shown that [75] if the function f satisfies the pseudo-compression under the maximum block norm, the asynchronous algorithm can converge to the fixed point and can tolerate communication and calculation delay at any size. In particular, for partial asynchrony, as long as the asynchronous factor B is reasonably set, the iterative algorithm is convergent in most cases [34].…”

“…The asynchronous iteration algorithm [33] entails the independent execution of iterations by all processors. Prior to the initiation of each iteration, if the required data has not been updated from other threads, and assuming Thread 1 requires data from Thread 2, but if Thread 2, due to a high computational load or weak computational capability, results in j 2 < j 1 , then the thread will resort to using outdated (data(2, j 2 )) data to proceed with the computation.Asynchronous algorithms are generally utilized for parallel computing in distributed systems, encompassing the optimization of large-scale linear algebra to the distributed coordination of small embedded devices [34]. The asynchronous algorithm allows computing nodes to operate asynchronously, facilitating the implementation of distributed algorithms and eliminating the communication waiting overhead associated with synchronization.…”