Holarchic structures for decentralized deep learning: a performance analysis

Abstract

Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and selfmanagement is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.

Decentralized Collective Learning for Self-managed Sharing Economies

“…Although the learning process of I-EPOS relies so far on a tree topology to perform the aggregation of the selected plans, more densely connected hierarchical structures as the ones of neural networks are worth further investigation. Furthermore, the impact of failures on the learning performance is out of the scope of this article but addressed in related and ongoing work (Pournaras et al 2018).…”

Impact of Smart Economy on Smart Areas and Mediation Effect of National Economy

“…The researches dealing with Smart People suppose them to be creative, flexible in all areas of their lives, and integrated into society, having highly developed professional competences and skills, actively participating in all initiatives, and always ready to learn [24][25][26][27][28][29]. These features can be implemented only in the case when the economy of the country really requires them, and we can assume that the smart economy will have a specific impact on the area.…”

On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework

“…An alternative approach is the introduction of learning mechanisms that are by design self-adaptive to network uncertainties. Self-adaptation may refer to the localisation of the learning process, applying the concept of dropout in the communication network to improve performance, and balance exploration and exploitation [92].…”