2019
DOI: 10.1126/sciadv.aay6946
|View full text |Cite
|
Sign up to set email alerts
|

Wave physics as an analog recurrent neural network

Abstract: Analog machine learning hardware platforms promise to be faster and more energy-efficient than their digital counterparts. Wave physics, as found in acoustics and optics, is a natural candidate for building analog processors for time-varying signals. Here we identify a mapping between the dynamics of wave physics, and the computation in recurrent neural networks. This mapping indicates that physical wave systems can be trained to learn complex features in temporal data, using standard training techniques for n… Show more

Help me understand this report
View preprint versions

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

3
162
0

Year Published

2020
2020
2024
2024

Publication Types

Select...
10

Relationship

0
10

Authors

Journals

citations
Cited by 273 publications
(180 citation statements)
references
References 40 publications
(37 reference statements)
3
162
0
Order By: Relevance
“…We also note that, recently, a mapping between the vanilla RNN's update equations and the wave equation has been proposed in [36] which demonstrates an equivalence between consecutive updates of the hidden state of an RNN and the dynamics of wave propagation. However, while intriguing, this equivalence is limited to a vanilla RNN and we believe that our guiding observations allow for a much broader field of interpretation and experimentation as will be demonstrated by affirming results.…”
Section: Research Methodology a Motivationmentioning
confidence: 84%
“…We also note that, recently, a mapping between the vanilla RNN's update equations and the wave equation has been proposed in [36] which demonstrates an equivalence between consecutive updates of the hidden state of an RNN and the dynamics of wave propagation. However, while intriguing, this equivalence is limited to a vanilla RNN and we believe that our guiding observations allow for a much broader field of interpretation and experimentation as will be demonstrated by affirming results.…”
Section: Research Methodology a Motivationmentioning
confidence: 84%
“…At its core, the presented framework can be interpreted as a training regularization method that avoids overfitting of a machine learning hardware to the specific 3D physical structure, distances and operational conditions, which are often assumed to be deterministic, precise and ideal during the training phase. In this respect, beyond its application to practically improve diffractive optical neural networks, the core principles introduced in our work can be extended to train other machine learning platforms [35,50,51] to mitigate various physical error sources that can cause deviations between the designed inference models and their corresponding physical implementations.…”
Section: Discussionmentioning
confidence: 99%
“…Therefore the complexity of NML in both theoretical and computational aspects will increase when the latent space dimension increases. Hughes et al (2019) and Sun et al (2020) showed that the wave-equation modeling is equivalent to the recurrent neural network (RNN) and the FWI gradient can be automatically calculated by the AD. Because CAE training also relies on the AD, therefore the AD is a perfect tool to numerically connect a CAE architecture to the wave-equation inversion.…”
Section: Hybrid Machine Learning Inversionmentioning
confidence: 99%