Learning by Message Passing in Networks of Discrete Synapses

Braunstein, Alfredo; Zecchina, Riccardo

doi:10.1103/physrevlett.96.030201

Cited by 134 publications

(222 citation statements)

References 37 publications

Supporting

Mentioning

211

Contrasting

Order By: Relevance

“…Additionally, it is the most commonly used measure in recent publications in this field (e.g., in Ref. [27]). We use the cost function (2) as a measure used for the performance of the studied algorithms; however, the algorithms themselves have been derived by statistical physics methods and rely on minimization of an extensive energy given by the number of misclassified patterns…”

Section: Exemplar Problem: the Binary Ising Perceptronmentioning

confidence: 99%

“…The reason for choosing PT is that it is a well established parallel algorithm with good performance in searching for solutions in the BIP capacity problem. Other derivatives of BP-based algorithms have been used to solve the BIP capacity problem, for instance, survey propagation [6,27]; the latter also aims to address the fragmentation of solution space but employs a different approach. The results reported [6,27] show that solutions can be found very close to the theoretical limits even for large systems, but additional practical techniques and considerations should be used to successfully obtain solutions.…”

Section: Performancementioning

confidence: 99%

“…Generally, like most MP-based algorithms, the approach is based on solutions being calculated iteratively using a pair of coupled self-consistent equations. We will study the properties of this alternative algorithm, its advantages and limitations, on an exemplar problem of the binary Ising perceptron (BIP) [25,26] that has been used as a benchmark also in other works on advanced inference methods [27].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Replication-based inference algorithms for hard computational problems

2013

View full text Add to dashboard Cite

Inference algorithms based on evolving interactions between replicated solutions are introduced and analyzed on a prototypical NP-hard problem: the capacity of the binary Ising perceptron. The efficiency of the algorithm is examined numerically against that of the parallel tempering algorithm, showing improved performance in terms of the results obtained, computing requirements and simplicity of implementation.

show abstract

Section: Exemplar Problem: the Binary Ising Perceptronmentioning

confidence: 99%

Section: Performancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Replication-based inference algorithms for hard computational problems

2013

View full text Add to dashboard Cite

show abstract

“…25-27, which have considered similar graph-theoretic problems † † . BP allows one to derive results on ensembles of graphs in the limit jNj → ∞ (18) but it also provides efficient heuristic algorithms to find collaborative equilibria on given graph instances (28,29). The messages which are exchanged in the BP algorithm are the probabilities μ i→j ¼ Pfi ∈ Γ j ⋂ Cg that player i collaborates and punishes j, in the collaborative equilibrium ‡ ‡ .…”

Section: The Complexity Of Collaboration On Networkmentioning

confidence: 99%

Collaboration in social networks

Dall’Asta

Marsili²,

Pin³

2012

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The very notion of social network implies that linked individuals interact repeatedly with each other. This notion allows them not only to learn successful strategies and adapt to them, but also to condition their own behavior on the behavior of others, in a strategic forward looking manner. Game theory of repeated games shows that these circumstances are conducive to the emergence of collaboration in simple games of two players. We investigate the extension of this concept to the case where players are engaged in a local contribution game and show that rationality and credibility of threats identify a class of Nash equilibria-that we call "collaborative equilibria"-that have a precise interpretation in terms of subgraphs of the social network. For large network games, the number of such equilibria is exponentially large in the number of players. When incentives to defect are small, equilibria are supported by local structures whereas when incentives exceed a threshold they acquire a nonlocal nature, which requires a "critical mass" of more than a given fraction of the players to collaborate. Therefore, when incentives are high, an individual deviation typically causes the collapse of collaboration across the whole system. At the same time, higher incentives to defect typically support equilibria with a higher density of collaborators. The resulting picture conforms with several results in sociology and in the experimental literature on game theory, such as the prevalence of collaboration in denser groups and in the structural hubs of sparse networks.T he social network influences and constrains in nontrivial ways the behavior of individuals (1) but also contributes to aspects generically referred to as social capital (2-4), which favor the emergence of coordinated actions or collaboration † . The theoretical investigation about the emergence of collaboration and the fate of repeated actions on networks has, up to now, mostly focused on adaptive learning, imitation, and evolutionary game theory. The corresponding theoretical models (6-9) seem to suggest that sustaining collaboration is easier on sparse networks and spatial structures than on dense groups. On the contrary, there is no clear experimental evidence of such an effect. Results of recent experiments on voluntary contribution suggest instead that the establishment and maintenance of collaboration is easier on well connected groups than on sparse graphs (10-12), in some ways confirming the intuition behind the classical Coleman's closure argument (13). The picture becomes more complex as we move to analyze individual positions in the network: Experiments performed on star-shaped groups suggest that central nodes are fostered to collaborate more than peripherals (12), corroborating another well known idea; i.e., Burt's argument on the importance of structural holes (14).However, the network is not only the channel for the achievement of payoffs or the exchange of information, which are the key ingredients of learning and imitative behavior. The network al...

show abstract

“…It was shown empirically that SP rarely makes any mistakes in its decimation, and SP solves very large 3-SAT instances that are very hard for local search algorithms. Recently, Braunstein and Zecchina (2006) have shown that by modifying BP and SP updates with a reinforcement term, the effectiveness of these algorithms as solvers can be further improved.…”

Section: Related Work On Satmentioning

confidence: 99%

Relaxed Survey Propagation for The Weighted Maximum Satisfiability Problem

Hutter

Hoos

Leyton‐Brown

et al. 2009

jair

View full text Add to dashboard Cite

The survey propagation (SP) algorithm has been shown to work well on large instances of the random 3-SAT problem near its phase transition. It was shown that SP estimates marginals over covers that represent clusters of solutions. The SP-y algorithm generalizes SP to work on the maximum satisfiability (Max-SAT) problem, but the cover interpretation of SP does not generalize to SP-y. In this paper, we formulate the relaxed survey propagation (RSP) algorithm, which extends the SP algorithm to apply to the weighted Max-SAT problem. We show that RSP has an interpretation of estimating marginals over covers violating a set of clauses with minimal weight. This naturally generalizes the cover interpretation of SP. Empirically, we show that RSP outperforms SP-y and other state-of-the-art Max-SAT solvers on random Max-SAT instances. RSP also outperforms state-of-the-art weighted Max-SAT solvers on random weighted Max-SAT instances.

show abstract

Learning by Message Passing in Networks of Discrete Synapses

Cited by 134 publications

References 37 publications

Replication-based inference algorithms for hard computational problems

Replication-based inference algorithms for hard computational problems

Collaboration in social networks

Relaxed Survey Propagation for The Weighted Maximum Satisfiability Problem

Contact Info

Product

Resources

About