This paper presents the machine learning architecture of the Snips Voice Platform, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices. The embedded inference is fast and accurate while enforcing privacy by design, as no personal user data is ever collected. Focusing on Automatic Speech Recognition and Natural Language Understanding, we detail our approach to training high-performance Machine Learning models that are small enough to run in real-time on small devices. Additionally, we describe a data generation procedure that provides sufficient, high-quality training data without compromising user privacy.1 https://www.voicebot.ai/2018/03/07/new-voicebot-report-says-nearly-20-u-sadults-smart-speakers/ 2 In French: https://www.cnil.fr/fr/enceintes-intelligentes-des-assistants-vocauxconnectes-votre-vie-privee 3 https://www.eugdpr.org/

Attentional theories of associative learning and categorization propose that learning about the predictiveness of a stimulus influences the amount of attention that is paid to that stimulus. Three experiments tested this idea by looking at the extent to which stimuli that had previously been experienced as predictive or nonpredictive in a categorization task were able to capture attention in a dot probe task. Consistent with certain attentional theories of learning, responses to the dot probe were faster when it appeared in a location cued by a predictive stimulus compared to a location cued by a nonpredictive stimulus. This result was obtained only with short (250-ms or 350-ms) but not long (1,000-ms) delays between onset of the stimuli and the dot probe, suggesting that the observed spatial cuing effect reflects the operation of a relatively rapid, automatic process. These findings are consistent with the approach to the relationship between attention and learning taken by the class of models exemplified by Mackintosh's (1975) theory.

We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mimics a federation of wake word users. We empirically demonstrate that using an adaptive averaging strategy inspired from Adam in place of standard weighted model averaging highly reduces the number of communication rounds required to reach our target performance. The associated upstream communication costs per user are estimated at 8 MB, which is a reasonable in the context of smart home voice assistants. Additionally, the dataset used for these experiments is being open sourced with the aim of fostering further transparent research in the application of federated learning to speech data.

Recent research has shown that perceptual processing of stimuli previously associated with high-value rewards is automatically prioritized even when rewards are no longer available. It has been hypothesized that such reward-related modulation of stimulus salience is conceptually similar to an "attentional habit." Recording event-related potentials in humans during a reinforcement learning task, we show strong evidence in favor of this hypothesis. Resistance to outcome devaluation (the defining feature of a habit) was shown by the stimulus-locked P1 component, reflecting activity in the extrastriate visual cortex. Analysis at longer latencies revealed a positive component (corresponding to the P3b, from 550-700 ms) sensitive to outcome devaluation. Therefore, distinct spatiotemporal patterns of brain activity were observed corresponding to habitual and goal-directed processes. These results demonstrate that reinforcement learning engages both attentional habits and goal-directed processes in parallel. Consequences for brain and computational models of reinforcement learning are discussed. The human attentional network adapts to detect stimuli that predict important rewards. A recent hypothesis suggests that the visual cortex automatically prioritizes reward-related stimuli, driven by cached representations of reward value; that is, stimulus-response habits. Alternatively, the neural system may track the current value of the predicted outcome. Our results demonstrate for the first time that visual cortex activity is increased for reward-related stimuli even when the rewarding event is temporarily devalued. In contrast, longer-latency brain activity was specifically sensitive to transient changes in reward value. Therefore, we show that both habit-like attention and goal-directed processes occur in the same learning episode at different latencies. This result has important consequences for computational models of reinforcement learning.

The neural basis of feedback expectation, which is crucial in learning theory, has only been minimally studied. Stimulus-preceding negativity (SPN), an ERP component that appears prior to the presentation of feedback, has been proposed as being related to feedback expectation. The present study showed, for the first time, amplitude modulations of the SPN component during learning acquisition in a trial-by-trial associative learning task. The results indicate that SPN could be a plausible electrophysiological index of the cognitive processes engaged while expecting the appearance of relevant feedback during reinforcement learning.