Additional Applications

Andrilli, Stephen

doi:10.1016/b978-0-12-374751-8.00019-6

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The product of the stochastic matrices is a stochastic matrix. 34 Theorem 1. The convergence result of the product of the random doubly stochastic matrices is equal to the AllReduce network topology.…”

Section: Random Topology With Fair Peer Selectionmentioning

confidence: 99%

Crossover‐SGD: A gossip‐based communication in distributed deep learning for alleviating large mini‐batch problem and enhancing scalability

Yeo

Bae

Jeong

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary Distributed deep learning is an effective way to reduce the training time for large datasets as well as complex models. However, the limited scalability caused by network‐overheads makes it difficult to synchronize the parameters of all workers and gossip‐based methods that demonstrate stable scalability regardless of the number of workers have been proposed. However, to use gossip‐based methods in general cases, the validation accuracy for a large mini‐batch needs to be verified. For this, we first empirically study the characteristics of gossip methods in a large mini‐batch problem and observe that gossip methods preserve higher validation accuracy than AllReduce‐SGD (stochastic gradient descent) when the number of batch sizes is increased, and the number of workers is fixed. However, the delayed parameter propagation of the gossip‐based models decreases validation accuracy in large node scales. To cope with this problem, we propose Crossover‐SGD that alleviates the delay propagation of weight parameters via segment‐wise communication and random network topology with fair peer selection. We also adapt hierarchical communication to limit the number of workers in gossip‐based communication methods. To validate the effectiveness of our method, we conduct empirical experiments and observe that our Crossover‐SGD shows higher node scalability than stochastic gradient push.

show abstract

Section: Random Topology With Fair Peer Selectionmentioning

confidence: 99%

Crossover‐SGD: A gossip‐based communication in distributed deep learning for alleviating large mini‐batch problem and enhancing scalability

Yeo

Bae

Jeong

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…The Markov model generated the probability transition matrix for land use/cover maps of 1993 and 2006, as shown in Table 7. The transition matrix areas show the total cells to be transformed into another category in the next time span (Eastman 2012), it is a stochastic matrix whose (i, j) entry gives the probability that an element moves from the jth category to the ith category during the next step of the process (Andrilli and Hecker 2010). This probability transition matrix was used to simulate LULC for the year 2018.…”

Section: The Transition Matrix Of Land-use/cover Change On the Slopes Of Kilimanjaromentioning

confidence: 99%

Predicting land use/cover changes and its association to agricultural production on the slopes of Mount Kilimanjaro, Tanzania

et al. 2021

View full text Add to dashboard Cite

Increasing demand for food production results in Land use and land cover (LULC) changes, which afflicts the provision of ecosystem services in high mountain areas. This work used time-series LULC and selected spatial metrics to predict the LULC changes for Kikafu-Weruweru-Karanga (KWK) watershed (on the southern slopes of Mt. Kilimanjaro) for the next decade. LULC maps were generated by classifying time-series satellite images. We further predicted the implications for selected staple crop production over the next decade. The simulated LULC shows expansion in built-up (by 32.55%/27.04 km 2 ) and agriculture (by 39.52%/52.0 km 2 ) areas from 2018 to 2030. These results suggest that urbanization is likely the next biggest threat to water availability and food production. Grasslands and wetlands are expected to decrease by 57.24% and 39.29%, respectively. The forest area is likely to shrink by 6.37%, about 9.82 km 2 , and 1.26 km 2 being converted to agriculture and built-up areas, respectively. However, expansion in agricultural land shows very little increase in staple food crop production records, suggesting that farm size plays a minor role in increasing crop production. Predicting the near future LULC around KWK is useful for evaluating the likelihood of achieving development and conservation targets that are set locally, nationally and internationally.

show abstract

“…The system states and transition rates can be summarized into a transition matrix , which allows to calculate the specific probability that the process, in its next step, switches from one particular state to another [11]. Upon specific system conditions and assumptions (e.g., thermodynamic equilibrium), and by analyzing the eigendecomposition of the matrix , we can furthermore obtain additional information about the system dynamics [12].…”

Section: Introductionmentioning

confidence: 99%

Spectral clustering of Markov chain transition matrices with complex eigenvalues

Frank¹,

Sikorski²,

Röblitz³

2022

Preprint

View full text Add to dashboard Cite

The Robust Perron Cluster Analysis (PCCA+) has become a popular algorithm for coarsegraining transition matrices of nearly decomposable Markov chains with transition states. Though originally developed for reversible Markov chains, it has been shown previously that PCCA+ can also be applied to cluster non-reversible Markov chains. However, the algorithm was implemented by assuming the dominant (target) eigenvalues to be real numbers. Therefore, the generalized Robust Perron Cluster Analysis (G-PCCA+) has recently been developed. G-PCCA+ is based on real Schur vectors instead of eigenvectors and can therefore be used to also coarse-grain transition matrices with complex eigenvalues. In its current implementation, however, G-PCCA+ is computationally expensive, which limits its applicability to large matrix problems.In this paper, we demonstrate that PCCA+ works in fact on any dominant invariant subspace of a nearly decomposable transition matrix, including both Schur vectors and eigenvectors. In particular, by separating the real and imaginary parts of complex eigenvectors, PCCA+ also works for transition matrices that have complex eigenvalues, including matrices with a circular transition pattern. We show that this separation maintains the invariant subspace, and that our version of the PCCA+ algorithm results in the same coarse-grained transition matrices as G-PCCA+, whereby PCCA+ is consistently faster in runtime than G-PCCA+. The analysis is performed in the Matlab programming language and codes are provided.

show abstract

Additional Applications

Cited by 3 publications

References 0 publications

Crossover‐SGD: A gossip‐based communication in distributed deep learning for alleviating large mini‐batch problem and enhancing scalability

Crossover‐SGD: A gossip‐based communication in distributed deep learning for alleviating large mini‐batch problem and enhancing scalability

Predicting land use/cover changes and its association to agricultural production on the slopes of Mount Kilimanjaro, Tanzania

Spectral clustering of Markov chain transition matrices with complex eigenvalues

Contact Info

Product

Resources

About