A modular, extensible approach to massive ecologically valid behavioral data

VanDam, Mark; Palma, Paul De

doi:10.3758/s13428-018-1167-8

Cited by 10 publications

(9 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One approach that has been making its way into the mainstream literature across basic and applied research on language and cognition relies on day-long recordings gathered with a LENA audiorecorder (e.g., Gilkerson et al 2017;Greenwood et al, 2011;Oller et al 2010;VanDam & De Palma 2018), and further analyzed using automated, closed-source algorithms. As we summarize below, this approach has many advantages, which may explain its expanding popularity.…”

mentioning

confidence: 99%

A thorough evaluation of the Language Environment Analysis (LENA) system

et al. 2020

View full text Add to dashboard Cite

In the previous decade, dozens of studies involving thousands of children across several research disciplines have made use of a combined daylong audio-recorder and automated algorithmic analysis called the LENA system, which aims to assess children's language environment. While the system's prevalence in the language acquisition domain is steadily growing, there are only scattered validation efforts on only some of its key characteristics. Here, we assess the LENA system's accuracy across all of its key measures: speaker classification, Child Vocalization Counts (CVC), Conversational Turn Counts (CTC), and Adult Word Counts (AWC). Our assessment is based on manual annotation of clips that have been randomly or periodically sampled out of daylong recordings, collected from (a) populations similar to the system's original training data (North American English-learning children aged 3-36 months), (b) children learning another dialect of English (UK), and (c) slightly older children growing up in a different linguistic and socio-cultural setting (Tsimane' learners in rural Bolivia). We find reasonably high accuracy in some measures (AWC, CVC), with more problematic levels of performance in others (CTC, precision of male adults and other children). Statistical analyses do not support the view that performance is worse for children who are dissimilar from the LENA original training set. Whether LENA results are accurate enough for a given research, educational, or clinical application depends largely on the specifics at hand. We therefore conclude with a set of recommendations to help researchers make this determination for their goals.

show abstract

mentioning

confidence: 99%

A thorough evaluation of the Language Environment Analysis (LENA) system

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Results from those studies may shed light not only on linguistic and cultural differences but also on shared characteristics of children at risk for or diagnosed with speech and language disorders [51,52,53,54]. Finally, an alternative to the LENA processing strategies may offer additional diagnostic options, transparency, and reduce the cost of implementation [55]. It is also clear that this technology is not a diagnostic tool, but is best used as a screening tool in conjunction with other validated clinical tolls and professional expertise.…”

Section: Discussionmentioning

confidence: 99%

“…The technology of automatic vocal analysis has been accelerating recently and its application has become more widespread in academic, commercial, and therapeutic contexts [55]. The methodological approach common to the work here is the future of communication disorders, including its application and potential in tele practice [57], developing communities, and in the modern provision of services to a diverse, dynamic population.…”

Section: Discussionmentioning

confidence: 99%

Use of the LENA Autism Screen with Children who are Deaf or Hard of Hearing

VanDam

Yoshinaga-Itano

2019

Medicina

Self Cite

View full text Add to dashboard Cite

Background and Objectives: This systematic review reports the evidence from the literature concerning the potential for using an automated vocal analysis, the Language ENvironment Analysis (LENA, LENA Research Foundation, Boulder, CO, USA) in the screening process for children at risk for autism spectrum disorder (ASD) and deaf or hard of hearing (D/HH). ASD and D/HH have increased comorbidity, but current behavioral diagnostic and screening tools have limitations. The LENA Language Autism Screen (LLAS) may offer an additional tool to disambiguate ASD from D/HH in young children. Materials and Methods: We examine empirical reports that use automatic vocal analysis methods to differentiate disordered from typically developing children. Results: Consensus across the sampled scientific literature shows support for use of automatic methods for screening and disambiguation of children with ASD and D/HH. There is some evidence of vocal differentiation between ASD, D/HH, and typically-developing children warranting use of the LLAS, but additional empirical evidence is needed to better understand the strengths and weaknesses of the tool. Conclusions: The findings reported here warrant further, more substantive, methodologically-sound research that is fully powered to show a reliable difference. Findings may be useful for both clinicians and researchers in better identification and understanding of communication disorders.

show abstract

“…Reaching conventional thresholds for reliably identifying annotation targets is therefore a Sisyphean task that often demands updated rationale. Finally, because successful annotation requires manyfold the duration of the annotated recording (MacWhinney, 2000), manually annotating many hours of everyday life requires very large investments of time, personnel, and other resources VanDam & De Palma, 2019). Theorists must achieve remarkable "operations manager" prowess in their laboratories.…”

Section: Quantifying Everyday Ecologies: Principles For Manual Annotation Of Many Hours Of Infants' Livesmentioning

confidence: 99%

Quantifying everyday ecologies: Principles for manual annotation of many hours of infants' lives

Mendoza¹,

Fausey²

2021

Preprint

View full text Add to dashboard Cite

Everyday experiences are the experiences available to shape developmental change. Remarkable advances in devices used to record infants’ and toddlers’ everyday experiences, as well as in repositories to aggregate and share such recordings across teams of theorists, have yielded a potential gold mine of insights to spur next-generation theories of experience-dependent change. Making full use of these advances, however, currently requires manual annotation. Manually annotating many hours of everyday life is a dedicated pursuit requiring significant time and resources, and in many domains is an endeavor currently lacking foundational facts to guide potentially consequential implementation decisions. These realities make manual annotation a frequent ‘off ramp’ en route to discoveries, as theorists instead opt for narrower scoped activities. Here, we provide theorists with a framework for manually annotating many hours of everyday life designed to reduce both theoretical and practical overwhelm. We share insights based on our team’s recent adventures in the previously uncharted territory of everyday music. We identify principles, and share implementation examples and tools, to help theorists achieve scalable solutions to challenges that are especially fierce when annotating extended timescales. These principles for quantifying everyday ecologies will help theorists collectively maximize return on investment in databases of everyday recordings and will enable a broad community of scholars – across institutions, skillsets, experiences, and working environments – to make discoveries about the experiences upon which development may depend.

show abstract

A modular, extensible approach to massive ecologically valid behavioral data

Cited by 10 publications

References 70 publications

A thorough evaluation of the Language Environment Analysis (LENA) system

A thorough evaluation of the Language Environment Analysis (LENA) system

Use of the LENA Autism Screen with Children who are Deaf or Hard of Hearing

Quantifying everyday ecologies: Principles for manual annotation of many hours of infants' lives

Contact Info

Product

Resources

About