We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking. Dynabench runs in a web browser and supports human-and-model-inthe-loop dataset creation: annotators seek to create examples that a target model will misclassify, but that another person will not. In this paper, we argue that Dynabench addresses a critical need in our community: contemporary models quickly achieve outstanding performance on benchmark tasks but nonetheless fail on simple challenge examples and falter in real-world scenarios. With Dynabench, dataset creation, model development, and model assessment can directly inform each other, leading to more robust and informative benchmarks. We report on four initial NLP tasks, illustrating these concepts and highlighting the promise of the platform, and address potential objections to dynamic benchmarking as a new standard for the field.

In spring 2014 (May–June), in the southern East China Sea (ECS) and east of Taiwan, nutrient and isotope samples were collected and analyzed. Also, dissolved oxygen (DO), chlorophyll‐a, and physical parameters were determined. The Kuroshio subsurface water intruded into the ECS and separated into two branches on the continental shelf: the nearshore Kuroshio branch current (NKBC) and the offshore Kuroshio branch current (OKBC). Nitrate concentration in the Kuroshio subsurface water slightly decreased east of Taiwan by assimilation and was supplied by upwelling currents northeast of Taiwan. The variations of DO, nitrate, and nitrogen isotopes in the bottom water showed that continuous nitrification occurred in the NKBC after intrusion into the ECS. This process might contribute to the hypoxia zone near the coast of Zhejiang Province, China. Our results also indicated that internal nitrogen cycles appeared in NKBC since the isotope fractionation ratio of oxygen and nitrogen (18ɛ:15ɛ) in NO3− was 1.40. The NO3− flux and budget were calculated in the NKBC via numerical simulations. It demonstrated that at least ∼0.52 kmol NO3−·s−1 was produced by nitrification from DH9 to DH5 transect, and ∼0.11 kmol NO3−·s−1 was consumed in the DH4 transect. Moreover, according to the Rayleigh model, primary production in most of southern ECS was supported by the intrusion of the Kuroshio subsurface water, causing 5‰ isotope fractionation. In some of nearshore stations which located in the northern investigated area, the assimilated nitrate was contributed from both the NKBC and coastal currents originated from the Changjiang diluted water.

Human emotions unfold over time, and more affective computing research has to prioritize capturing this crucial component of real-world affect. Modeling dynamic emotional stimuli requires solving the twin challenges of time-series modeling and of collecting high-quality time-series datasets. We begin by assessing the state-of-the-art in time-series emotion recognition, and we review contemporary time-series approaches in affective computing, including discriminative and generative models. We then introduce the first version of the Stanford Emotional Narratives Dataset (SENDv1): a set of rich, multimodal videos of self-paced, unscripted emotional narratives, annotated for emotional valence over time. The complex narratives and naturalistic expressions in this dataset provide a challenging test for contemporary time-series emotion recognition models. We demonstrate several baseline and state-of-the-art modeling approaches on the SEND, including a Long Short-Term Memory model and a multimodal Variational Recurrent Neural Network, which perform comparably to the human-benchmark. We end by discussing the implications for future research in time-series affective computing.recognition. Specifically, we define time-series modeling as taking in temporally continuous input data and producing temporally continuous output, with an explicit consideration of how information is propagated over time. For instance, in order to engage in such inference, a social robot in conversation with its user would have to take in a continuous stream of sensor data, process them, and reason about their user's emotions over time, perhaps after every second or after every sentence, as well as across many sentences in the conversation and across multiple conversations [11].Despite the progress that has been made in time-series emotion recognition in the past decade, the field is still far from affective robots that can understand human emotions in daily life. What is needed to achieve this ambitious goal? We suggest that the biggest barriers to overcome are due to (1) the inherent difficulty of building computational time-series models, and (2) the difficulty of collecting highquality datasets. To address this first gap, we conduct a review covering different machine-learning-based approaches to time-series modeling (Section 2). We begin by discussing the most common time-series techniques in affective computing: deep neural network models, part of a broader class of discriminative models. We also cover generative time-series approaches, which are comparatively less popular within affective computing, but offer interesting modeling capabilities and hold exciting potential for emotion understanding.We turn next to discuss the second gap: Researchers need high-quality time-series datasets on which to train models. These are expensive to construct, in terms of both the production of stimuli and the collection of timeseries annotations of emotion and affective labeling [12]. There are several existing time-series datasets that have been used by the ...