No abstract

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM).We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-ofthe-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned stateof-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies. * Equal Contribution. Author contributions and ordering details are listed in Appendix A.

The Greenland and Antarctic Ice Sheets cover ~\n10% of global land surface, but are rarely considered as active components of the global iron cycle. The ocean waters around both ice sheets harbour highly productive coastal ecosystems, many of which are iron limited. Measurements of iron concentrations in subglacial runoff from a large Greenland Ice Sheet catchment reveal the potential for globally significant export of labile iron fractions to the near-coastal euphotic zone. We estimate that the flux of bioavailable iron associated with glacial runoff is 0.40–2.54 Tg per year in Greenland and 0.06–0.17 Tg per year in Antarctica. Iron fluxes are dominated by a highly reactive and potentially bioavailable nanoparticulate suspended sediment fraction, similar to that identified in Antarctic icebergs. Estimates of labile iron fluxes in meltwater are comparable with aeolian dust fluxes to the oceans surrounding Greenland and Antarctica, and are similarly expected to increase in a warming climate with enhanced melting.

Background In laboratory animals, exposure to most general anaesthetics leads to neurotoxicity manifested by neuronal cell death and abnormal behaviour and cognition. Some large human cohort studies have shown an association between general anaesthesia at a young age and subsequent neurodevelopmental deficits, but these studies are prone to bias. Others have found no evidence for an association. We aimed to establish whether general anaesthesia in early infancy affects neurodevelopmental outcomes. Methods In this international, assessor-masked, equivalence, randomised, controlled trial conducted at 28 hospitals in Australia, Italy, the USA, the UK, Canada, the Netherlands, and New Zealand, we recruited infants of less than 60 weeks' postmenstrual age who were born at more than 26 weeks' gestation and were undergoing inguinal herniorrhaphy, without previous exposure to general anaesthesia or risk factors for neurological injury. Patients were randomly assigned (1:1) by use of a web-based randomisation service to receive either awake-regional anaesthetic or sevoflurane-based general anaesthetic. Anaesthetists were aware of group allocation, but individuals administering the neurodevelopmental assessments were not. Parents were informed of their infants group allocation upon request, but were told to mask this information from assessors. The primary outcome measure was full-scale intelligence quotient (FSIQ) on the Wechsler Preschool and Primary Scale of Intelligence, third edition (WPPSI-III), at 5 years of age. The primary analysis was done on a per-protocol basis, adjusted for gestational age at birth and country, with multiple imputation used to account for missing data. An intention-totreat analysis was also done. A difference in means of 5 points was predefined as the clinical equivalence margin. This completed trial is registered with ANZCTR, number ACTRN12606000441516, and ClinicalTrials.gov, number NCT00756600. Findings Between Feb 9, 2007, and Jan 31, 2013, 4023 infants were screened and 722 were randomly allocated: 363 (50%) to the awake-regional anaesthesia group and 359 (50%) to the general anaesthesia group. There were 74 protocol violations in the awake-regional anaesthesia group and two in the general anaesthesia group. Primary outcome data for the per-protocol analysis were obtained from 205 children in the awake-regional anaesthesia group and 242 in the general anaesthesia group. The median duration of general anaesthesia was 54 min (IQR 41-70). The mean FSIQ score was 99•08 (SD 18•35) in the awake-regional anaesthesia group and 98•97 (19•66) in the general anaesthesia group, with a difference in means (awake-regional anaesthesia minus general anaesthesia) of 0•23 (95% CI-2•59 to 3•06), providing strong evidence of equivalence. The results of the intention-to-treat analysis were similar to those of the per-protocol analysis. Interpretation Slightly less than 1 h of general anaesthesia in early infancy does not alter neurodevelopmental outcome at age 5 years compared with awake-regional anaesthesia ...

After birth, preterm infants face a stressful environment, which may negatively impact early brain development and subsequent neurobehavioral outcomes. This randomized controlled trial involving 45 women with infants Ͻ30-wk gestation, assessed the effectiveness of training parents in reducing stressful experiences. Intervention consisted of 10 sessions in the Neonatal Intensive Care Unit (NICU). Postintervention, at term-equivalent (40-wk postmenstrual age), magnetic resonance imaging (MRI) was performed to evaluate brain structure and development. Quantitative volumetric techniques were used to estimate overall and regional brain volumes for different tissue types including CSF, CGM, DNGM, UWM, and MWM. DTI was used to evaluate the integrity and maturation of white matter by ADC and FA. Maturation and connectivity of white matter, characterized by diffusion MR measures of ADC and FA, were significantly enhanced in the intervention group, who displayed greater restriction in ADC and increase in FA. There were no significant effects on either brain volumes or on short-term medical outcomes. Thus, sensitivity training for parents in the NICU is associated with improved cerebral white matter micro-structural development in preterm infants. (Pediatr Res 67: 330-335, 2010)

Social support could be a powerful weight-loss treatment moderator or mediator but is rarely assessed. We assessed the psychometric properties, initial levels, and predictive validity of a measure of perceived social support and sabotage from friends and family for healthy eating and physical activity (eight subscales). Overweight/obese women randomized to one of two 6-month, group-based behavioral weight-loss programs (N=267; mean BMI 32.1±3.5; 66.3% White) completed subscales at baseline, and weight loss was assessed at 6 months. Internal consistency, discriminant validity, and content validity were excellent for support subscales and adequate for sabotage subscales; qualitative responses revealed novel deliberate instances not reflected in current sabotage items. Most women (>75%) “never” or “rarely” experienced support from friends or family. Using non-parametric classification methods, we identified two subscales—support from friends for healthy eating and support from family for physical activity—that predicted three clinically meaningful subgroups who ranged in likelihood of losing ≥5% of initial weight at 6 months. Women who “never” experienced family support were least likely to lose weight (45.7% lost weight) whereas women who experienced both frequent friend and family support were more likely to lose weight (71.6% lost weight). Paradoxically, women who “never” experienced friend support were most likely to lose weight (80.0% lost weight), perhaps because the group-based programs provided support lacking from friendships. Psychometrics for support subscales were excellent; initial support was rare; and the differential roles of friend versus family support could inform future targeted weight-loss interventions to subgroups at risk.

BackgroundThe dorsolateral prefrontal cortex (DLPFC) is a common target for repetitive transcranial magnetic stimulation (rTMS) in major depression, but the conventional “5 cm rule” misses DLPFC in > 1/3 cases. Another heuristic, BeamF3, locates the F3 EEG site from scalp measurements. MRI-guided neuronavigation is more onerous, but can target a specific DLPFC stereotaxic coordinate directly. The concordance between these two approaches has not previously been assessed.ObjectiveTo quantify the discrepancy in scalp site between BeamF3 versus MRI-guided neuronavigation for left DLPFC.MethodsUsing 100 pre-treatment MRIs from subjects undergoing left DLPFC-rTMS, we localized the scalp site at minimum Euclidean distance from a target MNI coordinate (X − 38 Y + 44 Z + 26) derived from our previous work. We performed nasion-inion, tragus–tragus, and head-circumference measurements on the same subjects’ MRIs, and applied the BeamF3 heuristic. We then compared the distance between BeamF3 and MRI-guided scalp sites.ResultsBeamF3-to-MRI-guided discrepancies were <0.65 cm in 50% of subjects, <0.99 cm in 75% of subjects, and <1.36 cm in 95% of subjects. The angle from midline to the scalp site did not differ significantly using MRI-guided versus BeamF3 methods. However, the length of the radial arc from vertex to target site was slightly but significantly longer (mean 0.35 cm) with MRI-guidance versus BeamF3.ConclusionsThe BeamF3 heuristic may provide a reasonable approximation to MRI-guided neuronavigation for locating left DLPFC in a majority of subjects. A minor optimization of the heuristic may yield additional concordance.

No abstract