No abstract

Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.

We update the phenomenology of gauge singlet extensions of the Standard Model scalar sector and their implications for the electroweak phase transition. Considering the introduction of one real scalar singlet to the scalar potential, we analyze present constraints on the potential parameters from Higgs coupling measurements at the Large Hadron Collider (LHC) and electroweak precision observables for the kinematic regime in which no new scalar decay modes arise. We then show how future precision measurements of Higgs boson signal strengths and Higgs self-coupling could probe the scalar potential parameter space associated with a strong first-order electroweak phase transition. We illustrate using benchmark precision for several future collider options, including the High Luminosity LHC (HL-LHC), the International Linear Collider (ILC), TLEP, China Electron Positron Collider (CEPC), and a 100 TeV proton-proton collider, such as the Very High Energy LHC (VHE-LHC) or the Super proton-proton Collider (SPPC). For the regions of parameter space leading to a strong first order electroweak phase transition, we find that there exists considerable potential for observable deviations from purely Standard Model Higgs properties at these prospective future colliders.

We present a comprehensive study of the morphological properties of 42 γ-ray burst (GRB) host galaxies imaged with the Hubble Space Telescope in the optical band. The purpose of this study is to understand the relation of GRBs to their macro-environments, and to compare the GRB-selected galaxies to other high redshift samples. We perform both qualitative and quantitative analyses by categorizing the galaxies according to their visual properties, and by examining their surface brightness profiles. We find that all of the galaxies have approximately exponential profiles, indicative of galactic disks, and have a median scale length of about 1.7 kpc. Inspection of the visual morphologies reveals a high fraction of merging and interacting systems, with ∼ 30% showing clear signs of interaction, and an additional 30% exhibiting irregular and asymmetric structure which may be the result of recent mergers; these fractions are independent of redshift and galaxy luminosity. On the other hand, the three GRB host galaxies for which submillimeter and radio emission has been detected are isolated and compact, unlike the luminous submillimeter-selected galaxies. The fraction of mergers appears to be elevated compared to other high redshift samples, particularly for the low luminosities of GRB hosts (M B ∼ −16 to −21 mag). This suggests that merging and interacting galaxies undergoing a burst of star formation may be an efficient site for the production of GRB progenitors. Finally, we show that GRB hosts clearly follow the size-luminosity relation present in other galaxy samples, but thanks to absorption redshifts they help extend this relation to lower luminosities.

We study cosmological phase transitions in the Next-to-Minimal Supersymmetric Standard Model (NMSSM) in light of the Higgs discovery. We use an effective field theory approach to calculate the finite temperature effective potential, focusing on regions with significant tree-level contributions to the Higgs mass, a viable neutralino dark matter candidate, 1-2 TeV stops, and with the remaining particle spectrum compatible with current LHC searches and results. The phase transition structure in viable regions of parameter space exhibits a rich phenomenology, potentially giving rise to one-or two-step first-order phase transitions in the singlet and/or SU (2) directions. We compute several parameters pertaining to the bubble wall profile, including the bubble wall width and ∆β (the variation of the ratio in Higgs vacuum expectation values across the wall). These quantities can vary significantly across small regions of parameter space and can be promising for successful electroweak baryogenesis. We estimate the wall velocity microphysically, taking into account the various sources of friction acting on the expanding bubble wall. Ultra-relativistic solutions to the bubble wall equations of motion typically exist when the electroweak phase transition features substantial supercooling. For somewhat weaker transitions, the bubble wall instead tends to be sub-luminal and, in fact, likely sub-sonic, suggesting that successful electroweak baryogenesis may indeed occur in regions of the NMSSM compatible with the Higgs