Byzantine Fault-Tolerance in Decentralized Optimization under Minimal Redundancy

Abstract

This paper considers the problem of Byzantine fault-tolerance in multi-agent decentralized optimization. In this problem, each agent has a local cost function. The goal of a decentralized optimization algorithm is to allow the agents to cooperatively compute a common minimum point of their aggregate cost function. We consider the case when a certain number of agents may be Byzantine faulty. Such faulty agents may not follow a prescribed algorithm, and they may share arbitrary or incorrect information with other non-faulty agents. Presence of such Byzantine agents renders a typical decentralized optimization algorithm ineffective. We propose a decentralized optimization algorithm with provable exact fault-tolerance against a bounded number of Byzantine agents, provided the non-faulty agents have a minimal redundancy.

Byzantine-Resilient Multi-Agent Distributed Exact Optimization with Less Data

“…From this point of view, this redundancy condition seems to be strict. However, the redundancy in cost functions often happens in the actual scenarios, such as the distributed sensing [25,26] and the homogeneous distributed learning [27,28].…”

A Survey on Fault-tolerance in Distributed Optimization and Machine Learning

“…Gupta et al [48] studied the 2f -redundancy property in decentralized system. Specifically, the authors proposed Comparative Elimination (abbr.…”

“…Peer-to-peer network The majority of Byzantine fault-tolerant algorithms are built under the server-based architecture, e.g., [6,18,49], etc. ; while studies under peer-to-peer architectures, although exist [47,48], are rather rare, and the results are not as systematic. It would be interesting to explore further how the communication network structure is related to solvability of the fault-tolerance problem, and also practical algorithms to achieve fault-tolerance.…”