Critical branching processes in digital memcomputing machines

Bearden, Sean R. B.; Sheldon, Forrest; Ventra, Massimiliano Di

doi:10.1209/0295-5075/127/30005

Cited by 8 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the larger MNIST case, mode sampling was done via simulation of a digital memcomputing machine based on ref. 21 . The specific details of our implementation can be found in Supplementary Note 3.…”

Section: Resultsmentioning

confidence: 99%

Mode-assisted unsupervised learning of restricted Boltzmann machines

et al. 2020

View full text Add to dashboard Cite

Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gradient direction, constructed from samples of the RBM ground state (mode), improves training dramatically over traditional gradient methods. This approach, which we call 'mode-assisted training', promotes faster training and stability, in addition to lower converged relative entropy (KL divergence). We demonstrate its efficacy on synthetic datasets where we can compute KL divergences exactly, as well as on a larger machine learning standard (MNIST). The proposed mode-assisted training can be applied in conjunction with any given gradient method, and is easily extended to more general energybased neural network structures such as deep, convolutional and unrestricted Boltzmann machines.

show abstract

Section: Resultsmentioning

confidence: 99%

Mode-assisted unsupervised learning of restricted Boltzmann machines

et al. 2020

View full text Add to dashboard Cite

show abstract

“…For the larger MNIST case, mode sampling was done via simulation of a digital memcomputing machine based on Ref. 25. The specific details of our implementation can be found in the appendix.…”

Section: Resultsmentioning

confidence: 99%

“…Our implementation is based on the approach used in Ref. 25 for the satisfiability (SAT) problem, appropriately modified for the MAX-2-SAT optimization problem. For a MAX-2-SAT with N variables, M 1 1-SAT clauses, and…”

Section: Memory Dynamicsmentioning

confidence: 99%

Mode-Assisted Unsupervised Learning of Restricted Boltzmann Machines

Manukian,

Pei,

Bearden

et al. 2020

Preprint

View full text Add to dashboard Cite

Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gradient direction, constructed from samples of the RBM ground state (mode), improves their training dramatically over traditional gradient methods. This approach, which we call mode training, promotes faster training and stability, in addition to lower converged relative entropy (KL divergence). Along with the proofs of stability and convergence of this method, we also demonstrate its efficacy on synthetic datasets where we can compute KL divergences exactly, as well as on a larger machine learning standard, MNIST. The mode training we suggest is quite versatile, as it can be applied in conjunction with any given gradient method, and is easily extended to more general energy-based neural network structures such as deep, convolutional and unrestricted Boltzmann machines.

show abstract

“…XI of SM). Simulations found the DMMs then self-tune into a critical ( collective ) state which persists for the whole transient dynamics until a solution is found 16 . It is this critical branching behavior that allows DMMs to explore collective updates of variables during the solution search, without the need to check an exponentially-growing number of states.…”

Section: The Digital Memcomputing Approachmentioning

confidence: 99%

“…In Eq. ( 2 ), the first term in the summation is a “gradient-like” term, the second term is a “rigidity” term 16 . The gradient-like term attempts to influence the voltage in a clause based on the value of the other two voltages in the associated clause.…”

Section: Dmm For 3-satmentioning

confidence: 99%

Efficient solution of Boolean satisfiability problems with digital memcomputing

Bearden

Pei

Ventra

2020

Sci Rep

View full text Add to dashboard Cite

Boolean satisfiability is a propositional logic problem of interest in multiple fields, e.g., physics, mathematics, and computer science. Beyond a field of research, instances of the SAT problem, as it is known, require efficient solution methods in a variety of applications. It is the decision problem of determining whether a Boolean formula has a satisfying assignment, believed to require exponentially growing time for an algorithm to solve for the worst-case instances. Yet, the efficient solution of many classes of Boolean formulae eludes even the most successful algorithms, not only for the worst-case scenarios, but also for typical-case instances. Here, we introduce a memory-assisted physical system (a digital memcomputing machine) that, when its non-linear ordinary differential equations are integrated numerically, shows evidence for polynomially-bounded scalability while solving “hard” planted-solution instances of SAT, known to require exponential time to solve in the typical case for both complete and incomplete algorithms. Furthermore, we analytically demonstrate that the physical system can efficiently solve the SAT problem in continuous time, without the need to introduce chaos or an exponentially growing energy. The efficiency of the simulations is related to the collective dynamical properties of the original physical system that persist in the numerical integration to robustly guide the solution search even in the presence of numerical errors. We anticipate our results to broaden research directions in physics-inspired computing paradigms ranging from theory to application, from simulation to hardware implementation.

show abstract

Critical branching processes in digital memcomputing machines

Cited by 8 publications

References 24 publications

Mode-assisted unsupervised learning of restricted Boltzmann machines

Mode-assisted unsupervised learning of restricted Boltzmann machines

Mode-Assisted Unsupervised Learning of Restricted Boltzmann Machines

Efficient solution of Boolean satisfiability problems with digital memcomputing

Contact Info

Product

Resources

About