R on ne be rg er , K a t hr yn T un ya su vu na ko ol,

This is a PDF file of a peer-reviewed paper that has been accepted for publication. Although unedited, the content has been subjected to preliminary formatting. Nature is providing this early version of the typeset paper as a service to our authors and readers. The text and figures will undergo copyediting and a proof review before the paper is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers apply.

Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments (30, 40, 45,46,56) and two-player turn-based games (47,58,66). However, the realworld contains multiple agents, each learning and acting independently to cooperate and compete with other agents, and environments reflecting this degree of complexity remain an open challenge. In this work, we demonstrate for the first time that an agent can achieve human-level in a popular 3D multiplayer first-person video game, Quake III Arena Capture the Flag (28), using only pixels and game points as input. These results were achieved by a novel two-tier optimisation process in which a population of independent RL agents are trained concurrently from thousands of parallel matches with agents playing in teams together and against each other on randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning, and selects actions using a novel temporally hierarchical representation that enables the agent to reason at multiple timescales. During game-play, these agents display humanlike behaviours such as navigating, following, and defending based on a rich learned representation that is shown to encode high-level game knowledge. In an extensive tournament-style evaluation the trained agents exceeded the winrate of strong human players both as teammates and opponents, and proved far stronger than existing state-of-the-art agents. These results demonstrate a 1 arXiv:1807.01281v1 [cs.LG] 3 Jul 2018 significant jump in the capabilities of artificial agents, bringing us closer to the goal of human-level intelligence.We demonstrate how intelligent behaviour can emerge from training sophisticated new learning agents within complex multi-agent environments. End-to-end reinforcement learning methods (45, 46) have so far not succeeded in training agents in multi-agent games that combine team and competitive play due to the high complexity of the learning problem (7, 43) that arises from the concurrent adaptation of other learning agents in the environment. We approach this challenge by studying team-based multiplayer 3D first-person video games, a genre which is particularly immersive for humans (16) and has even been shown to improve a wide range of cognitive abilities (21). We focus specifically on a modified version (5) of Quake III Arena (28), the canonical multiplayer 3D first-person video game, whose game mechanics served as the basis for many subsequent games, and which has a thriving professional scene (1). The task we consider is the game mode Capture the Flag (CTF) on per game randomly generated maps of both indoor and outdoor theme ( Figure 1 (a,b)). Two opposing teams consisting of multiple individual players compete to capture each other's flags by strategically navigating, tagging, and evading opponents. The team with the greatest number of flag captures after five minutes wins. CTF is play...

We describe AlphaFold, the protein structure prediction system that was entered by the group A7D in CASP13. Submissions were made by three free‐modeling (FM) methods which combine the predictions of three neural networks. All three systems were guided by predictions of distances between pairs of residues produced by a neural network. Two systems assembled fragments produced by a generative neural network, one using scores from a network trained to regress GDT_TS. The third system shows that simple gradient descent on a properly constructed potential is able to perform on par with more expensive traditional search techniques and without requiring domain segmentation. In the CASP13 FM assessors' ranking by summed z‐scores, this system scored highest with 68.3 vs 48.2 for the next closest group (an average GDT_TS of 61.4). The system produced high‐accuracy structures (with GDT_TS scores of 70 or higher) for 11 out of 43 FM domains. Despite not explicitly using template information, the results in the template category were comparable to the best performing template‐based methods.

We have investigated the Al/Si ordering in the pseudoisolated pairs of tetrahedral sites of the structure of crystalline gehlenite Ca 2 Al 2 SiO 7 by means of 29 Si and 27 Al NMR and firstprinciples quantum mechanical calculations. 29 Si NMR spectra of isotopically enriched samples enables the precise determination of the population of the two silicon sites Si−(OAl) 3-n (OSi) n (n = 0, 1) and hence the amount of Al−O−Al linkages. This leads to a reliable and model-free quantification of the departure from the Loewenstein rule and to an experimental Al/Si ordering enthalpy of 50.4 ± 1.6 kJ/mol fully reproduced by the quantum mechanical calculations. The seven aluminum sites arising from the Al/Si substitutions Al−(OAl) 4-p (OSi) p (0 ≤ p ≤ 4) and Al−(OAl) 3-p (OSi) p (p = 0, 1) are identified by 27 Al MAS, MQMAS, and { 29 Si} 27 Al HMQC experiments, with their quantification being consistent with a fully disordered arrangement of the tetrahedral pairs in the a−b plane of the structure. Assignments of those strongly overlapping lines are further confirmed by density functional theory (DFT) calculations performed on a series of 2 × 25 supercells. An experimental and computational variation of −3 ppm of the 27 Al isotropic chemical shift is obtained for the substitution of one Al by one Si in the second coordination sphere of a central Al atom. 29 Si and 27 Al isotropic chemical shifts are seen to be sensitive primarily to short-range structural variations whereas a more complex behavior related to the nearby presence of Loewenstein-violating pairs is observed for the 27 Al quadrupolar coupling constant. Decomposition of the calculated EFG tensors into a sum of local, nonlocal, and ionic components demonstrates that it is almost entirely determined by the local electronic structure near the T 1 nucleus. The width of the distribution of NMR parameters is seen to strongly correlate to the degree of ordering present in the material. Scalar coupling constants 2 J(T−O−T) (with T = Al, Si) are found to be linearly related to the ∠TOT bond angle.

We introduce two open source tools to aid the processing and visualisation of ab-initio computed solid-state NMR parameters. The Magres file format for computed NMR parameters (as implemented in CASTEP v8.0 and QuantumEspresso v5.0.0) is implemented. MagresView is built upon the widely used Jmol crystal viewer, and provides an intuitive environment to display computed NMR parameters. It can provide simple pictorial representation of one- and two-dimensional NMR spectra as well as output a selected spin-system for exact simulations with dedicated spin-dynamics software. MagresPython provides a simple scripting environment to manipulate large numbers of computed NMR parameters to search for structural correlations.

We present a method for the first-principles calculation of nuclear magnetic resonance (NMR) J-coupling in extended systems using state-of-the-art ultrasoft pseudopotentials and including scalar-relativistic effects. The use of ultrasoft pseudopotentials is allowed by extending the projector augmented wave (PAW) method of Joyce et al. [J. Chem. Phys. 127, 204107 (2007)]. We benchmark it against existing local-orbital quantum chemical calculations and experiments for small molecules containing light elements, with good agreement. Scalar-relativistic effects are included at the zeroth-order regular approximation level of theory and benchmarked against existing local-orbital quantum chemical calculations and experiments for a number of small molecules containing the heavy row six elements W, Pt, Hg, Tl, and Pb, with good agreement. Finally, (1)J(P-Ag) and (2)J(P-Ag-P) couplings are calculated in some larger molecular crystals and compared against solid-state NMR experiments. Some remarks are also made as to improving the numerical stability of dipole perturbations using PAW.

Solid-state NMR spectra of new P-Se heterocycles based on peri-substituted naphthalene motifs show the presence of unusual J couplings between Se and P. These couplings are between atoms in adjacent molecules and occur "through space", rather than through conventional covalent bonds. Experimental measurements are supported by relativistic DFT calculations, which confirm the presence of couplings between nonbonded atoms, and provide information on the pathway of the interaction. This observation improves the understanding of J couplings and offers insight into the factors that affect crystal packing in solids, for future synthetic exploitation.