Programming nonlinearities inside multimode fibers for optical computing (Conference Presentation)

Oğuz, İlker; Hsieh, Jih-Liang; Dinç, Niyazi Ulaş; Teğin, Uğur; Yıldırım, Mustafa; Gigli, Carlo; Moser, Christophe

doi:10.1117/12.2632627

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Forward Gradient Forward gradient [4], DODGE [56], Belouze et al [5], Ren et al [51], Fournier et al [21], FDFA [2], Singhal et al [57] Zeroth-order Optimization BAFFLE [19], MeZO [35], DPZero [73], DeepZero [11] Evolution Strategy BBT [62], BBTv2 [61], PES [66], ES-Single [65] Perturbated Input Forward-Forward [24,42,41,40,55,36,38,13], PEPITA [14], MEMPEPITA [43] No Perturabation Wilamowski et al [67], Ma et al [33], d-RFs [28] Figure 2: A taxonomy of BP-free training methods.…”

Section: Perturbated Modelmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Backpropagation-free Training For LLMS

Mei,

Cai,

et al. 2024

Preprint

View full text Add to dashboard Cite

Large language models (LLMs) have achieved remarkable performance in various downstream tasks. However, training LLMs is computationally expensive and requires a large amount of memory. To address this issue, backpropagation-free (BP-free) training has been proposed as a promising approach to reduce the computational and memory costs of training LLMs. In this survey, we provide a comprehensive overview of BP-free training for LLMs. We first outline three mainstream BP-free training methods. Subsequently, we introduce their optimizations for LLMs. The goal of this survey is to provide a comprehensive understanding of BP-free training for LLMs and to inspire future research in this area.

show abstract

Section: Perturbated Modelmentioning

confidence: 99%

“…It is notably more suitable for low-power analog hardware than the BP algorithm. Adapted for resource-limited settings such as wave-based physical platforms [36,38] and Micro-Controller Units (MCUs) [13], the FF algorithm enables neural network training beyond the traditional von Neumann architecture.…”

Section: Perturbated Inputmentioning

confidence: 99%

A Survey of Backpropagation-free Training For LLMS

Mei,

Cai,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The increasing popularity of AI has motivated applications in various scientific areas such as physics [57], chemistry [18], health [58], and biology, as well as on unconventional devices such as physical devices [79] and optics [45]. Such computational devices, that rely on brain-like analog information processing, are still mostly based on backprop-based schemes that are unsuitable for physical implementation.…”

Section: Application Of Ai In Sciencementioning

confidence: 99%

A Review of Neuroscience-Inspired Machine Learning

Ororbia,

Mali,

Kohan

et al. 2024

Preprint

View full text Add to dashboard Cite

One major criticism of deep learning centers around the biological implausibility of the credit assignment schema used for learning -- backpropagation of errors. This implausibility translates into practical limitations, spanning scientific fields, including incompatibility with hardware and non-differentiable implementations, thus leading to expensive energy requirements. In contrast, biologically plausible credit assignment is compatible with practically any learning condition and is energy-efficient. As a result, it accommodates hardware and scientific modeling, e.g. learning with physical systems and non-differentiable behavior. Furthermore, it can lead to the development of real-time, adaptive neuromorphic processing systems. In addressing this problem, an interdisciplinary branch of artificial intelligence research that lies at the intersection of neuroscience, cognitive science, and machine learning has emerged. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks, discussing the solutions they provide for different scientific fields as well as their advantages on CPUs, GPUs, and novel implementations of neuromorphic hardware. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.

show abstract

“…This approach exploits the system's sensitivity to phase and amplitude variations at the input. As a result, task‐specific transient information graphs in nonlinear waveguides can be trained in two ways: either online, by adjusting a phase/amplitude mask of the input field [ 34 ] (known as input layer training), or offline, by weighting the system read‐out. The latter is known as output layer training from computational concepts such as reservoir computing [ 35 , 36 ] or extreme learning machines (ELM).…”

Section: Neuromorphic Wave Computing With Transient Nonlinear Opticsmentioning

confidence: 99%

Neuromorphic Computing via Fission‐based Broadband Frequency Generation

Fischer,

Chemnitz,

Zhu

et al. 2023

Advanced Science

View full text Add to dashboard Cite

The performance limitations of traditional computer architectures have led to the rise of brain‐inspired hardware, with optical solutions gaining popularity due to the energy efficiency, high speed, and scalability of linear operations. However, the use of optics to emulate the synaptic activity of neurons has remained a challenge since the integration of nonlinear nodes is power‐hungry and, thus, hard to scale. Neuromorphic wave computing offers a new paradigm for energy‐efficient information processing, building upon transient and passively nonlinear interactions between optical modes in a waveguide. Here, an implementation of this concept is presented using broadband frequency conversion by coherent higher‐order soliton fission in a single‐mode fiber. It is shown that phase encoding on femtosecond pulses at the input, alongside frequency selection and weighting at the system output, makes transient spectro‐temporal system states interpretable and allows for the energy‐efficient emulation of various digital neural networks. The experiments in a compact, fully fiber‐integrated setup substantiate an anticipated enhancement in computational performance with increasing system nonlinearity. The findings suggest that broadband frequency generation, accessible on‐chip and in‐fiber with off‐the‐shelf components, may challenge the traditional approach to node‐based brain‐inspired hardware design, ultimately leading to energy‐efficient, scalable, and dependable computing with minimal optical hardware requirements.

show abstract

Programming nonlinearities inside multimode fibers for optical computing (Conference Presentation)

Cited by 3 publications

References 0 publications

A Survey of Backpropagation-free Training For LLMS

A Survey of Backpropagation-free Training For LLMS

A Review of Neuroscience-Inspired Machine Learning

Neuromorphic Computing via Fission‐based Broadband Frequency Generation

Contact Info

Product

Resources

About