Statistical Learning, Inductive Bias, and Bayesian Inference in Language Acquisition

Pearl, Lisa; Goldwater, Sharon

doi:10.1093/oxfordhb/9780199601264.013.28

Cited by 14 publications

(11 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This analysis suggests that the differences between models could be more related to the choice of technical options than to their theoretical underpinnings. Several contributors to Bayesian literature (e.g., Goldwater et al., ; Pearl & Goldwater, ; Qian, Jaeger, & Aslin, ), referring to Marr ()'s framework, have emphasized that Bayesian models provide a computational ‐level account for learners’ behavior, while most other models, either symbolic or connectionist, would be aimed at simulating the same behavior at the algorithmic level. Marr's framework provides an attractive solution to the heterogeneity of models.…”

Section: Models Of Chunkingmentioning

confidence: 99%

What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

Perruchet¹

2018

Topics in Cognitive Science

View full text Add to dashboard Cite

In a prior review, Perrruchet and Pacton (2006) noted that the literature on implicit learning and the more recent studies on statistical learning focused on the same phenomena, namely the domain‐general learning mechanisms acting in incidental, unsupervised learning situations. However, they also noted that implicit learning and statistical learning research favored different interpretations, focusing on the selection of chunks and the computation of transitional probabilities aimed at discovering chunk boundaries, respectively. This paper examines the state of the debate 12 years later. The link between contrasting theories and their historical roots has disappeared, but a number of studies were aimed at contrasting the predictions of these two approaches. Overall, these studies strongly question the still prevalent account based on the statistical computation of pairwise associations. Various chunk‐based models provide much better predictions in a number of experimental situations. However, these models rely on very different conceptual frameworks, as illustrated by a comparison between Bayesian models of word segmentation, PARSER, and a connectionist model (TRACX).

show abstract

Section: Models Of Chunkingmentioning

confidence: 99%

What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

Perruchet¹

2018

Topics in Cognitive Science

View full text Add to dashboard Cite

show abstract

“…MaxEnt grammars), thus opening the door to the importation of the learning theories discussed in §4.1 (see Johnson 2013b for a more general introduction to statistical learning of grammars). It has been pointed out that statistical learning theory, especially Bayesian modeling, can permit a more rigorous assessment of claims about UG (see the overview in Pearl & Goldwater 2016). 18 When neural network modeling is integrated with grammatical formalisms in the ways discussed in §4, we may be able to go further in assessing the extent to which grammatical representations can be learned from experience and what aspects of the grammar must be hardwired.…”

mentioning

confidence: 99%

Generative linguistics and neural networks at 60: Foundation, friction, and fusion

Pater¹

2019

Language

View full text Add to dashboard Cite

The birthdate of both generative linguistics and neural networks can be taken as 1957, the year of the publication of foundational work by both Noam Chomsky and Frank Rosenblatt. This article traces the development of these two approaches to cognitive science, from their largely autonomous early development in the first thirty years, through their collision in the 1980s around the past-tense debate (Rumelhart & McClelland 1986, Pinker & Prince 1988) and their integration in much subsequent work up to the present. Although this integration has produced a considerable body of results, the continued general gulf between these two lines of research is likely impeding progress in both: on learning in generative linguistics, and on the representation of language in neural modeling. The article concludes with a brief argument that generative linguistics is unlikely to fulfill its promise of accounting for language learning if it continues to maintain its distance from neural and statistical approaches to learning.*

show abstract

“…Human infants are not as tabula rasa as models like InferSent but rather encode useful inductive biases (Chomsky & Lightfoot, 2002; Lightfoot & Julia, 1984; Mitchell, 1980; Pearl & Goldwater, 2016; Seidenberg, 1997). Building such biases into our models (Battaglia et al, 2018; Dubey, Agrawal, Pathak, Griffiths, & Efros, 2018; Gandhi & Lake, 2019; Lake et al, 2017) is a promising direction toward scalably learning systematic representations.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Analyzing Machine‐Learned Representations: A Natural Language Case Study

et al. 2020

View full text Add to dashboard Cite

As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding.

show abstract

Statistical Learning, Inductive Bias, and Bayesian Inference in Language Acquisition

Cited by 14 publications

References 108 publications

What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

Generative linguistics and neural networks at 60: Foundation, friction, and fusion

Analyzing Machine‐Learned Representations: A Natural Language Case Study

Contact Info

Product

Resources

About