Optical imaging through the intact mouse skull is challenging because of skull-induced aberrations and scattering. We found that three-photon excitation provided improved optical sectioning compared with that obtained with two-photon excitation, even when we used the same excitation wavelength and imaging system. Here we demonstrate three-photon imaging of vasculature through the adult mouse skull at >500-μm depth, as well as GCaMP6s calcium imaging over weeks in cortical layers 2/3 and 4 in awake mice, with 8.5 frames per second and a field of view spanning hundreds of micrometers.

Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability1. Deep-learning accelerators2–9 aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far10–22 have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ–in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics23–26, materials27–29 and smart sensors30–32.

Three-photon microscopy has been increasingly adopted for probing neural activities beyond the typical two-photon imaging depth. In this review, we outline the unique properties that differentiate three-photon microscopy from two-photon microscopy for in vivo imaging in biological samples, especially in the mouse brain. We present a systematic summary of the optimization of three-photon imaging parameters for neural imaging, based on their effects on calcium imaging quality and perturbation to brain tissues. Furthermore, we review the existing techniques for volumetric imaging and discuss their prospects in mesoscale three-photon imaging in deep tissue.

1300 nm three-photon calcium imaging has emerged as a useful technique to allow calcium imaging in deep brain regions. Application to large-scale neural activity imaging entails a careful balance between recording fidelity and perturbation to the sample. We calculated and experimentally verified the excitation pulse energy to achieve the minimum photon count required for the detection of calcium transients in GCaMP6s-expressing neurons for 920 nm two-photon and 1320 nm three-photon excitation. By considering the combined effects of in-focus signal attenuation and out-of-focus background generation, we quantified the cross-over depth beyond which three-photon microscopy outpeforms two-photon microscopy in recording fidelity. Brain tissue heating by continuous three-photon imaging was simulated with Monte Carlo method and experimentally validated with immunohistochemistry. Increased immunoreactivity was observed with 150 mW excitation power at 1 and 1.2 mm imaging depths. Our analysis presents a translatable model for the optimization of three-photon calcium imaging based on experimentally tractable parameters.

Deep learning has become a widespread tool in both science and industry. However, continued progress is hampered by the rapid growth in energy costs of ever-larger deep neural networks. Optical neural networks provide a potential means to solve the energy-cost problem faced by deep learning. Here, we experimentally demonstrate an optical neural network based on optical dot products that achieves 99% accuracy on handwritten-digit classification using ~3.1 detected photons per weight multiplication and ~90% accuracy using ~0.66 photons (~2.5 × 10−19 J of optical energy) per weight multiplication. The fundamental principle enabling our sub-photon-per-multiplication demonstration—noise reduction from the accumulation of scalar multiplications in dot-product sums—is applicable to many different optical-neural-network architectures. Our work shows that optical neural networks can achieve accurate results using extremely low optical energies.