Distributed Support Vector Machines Over Dynamic Balanced Directed Networks

Abstract

In this paper, we consider the binary classification problem via distributed Support Vector Machines (SVMs), where the idea is to train a network of agents, with limited share of data, to cooperatively learn the SVM classifier for the global database. Agents only share processed information regarding the classifier parameters and the gradient of the local loss functions instead of their raw data. In contrast to the existing work, we propose a continuoustime algorithm that incorporates network topology changes in discrete jumps. This hybrid nature allows us to remove chattering that arises because of the discretization of the underlying CT process. We show that the proposed algorithm converges to the SVM classifier over time-varying weight balanced directed graphs by using arguments from the matrix perturbation theory.

Distributed Finite-Sum Constrained Optimization subject to Nonlinearity on the Node Dynamics

“…For example, this may represent the entire convex area covered by a robotic network or the overall load demand and the amount of power allocated for generator coordination over power grids. For different applications, the problem could be subject to some additional constraints 1 . For example, the socalled box constraints in the form,…”

“…It can be proved that the solution of this penalized case can become arbitrary close to the exact optimizer by choosing sufficiently small. This nonsmooth function can be substituted by the following smooth equivalents [1], [37],…”

“…Using the KKT condition, the unique optimal solution x * for (1) is in the form ∇F (x * ) = ψ * 1 n , where ∇F (x * ) = 1 Recall that having multiple constraints in the form Ax = b, the problem (1) is not necessarily solvable locally. In this case, one can find the unique intersection of constraint faucets (if any) and reduce the constraints to a single lower-order one by presenting some states say xa as a linear combination of other complementary states x −a , say x j 1 , x j 2 , .…”

“…This implies that the bidirectional links over homogeneous MAN is well-justified. Moreover, for protocol (10) the solution can be further extended to strongly-connected with directed communications over MAN under certain design of the weighting (consensus) matrix W , referred to as doubly-stochastic condition or balanced network design [1].…”

“…D ISTRIBUTED and localized solutions are widespread in signal-processing, machine-learning, and control literature [1], and are getting more attention due to recent advancements in wireless communication devices, green networking [2], Internet-of-Things (IoT), cloud computing, and high-performance central processing units (CPUs). These techniques further found their way into vehicular technology, and intelligent transportation systems, e.g., Internet-of-Connected-Vehicles (IoCV) [3], [4] and platoons of autonomous cars [5].…”