An Extensive Knowledge Mapping Review of Measurement and Validity in Language Assessment and SLA Research

Abstract

This study set out to investigate intellectual domains as well as the use of measurement and validation methods in language assessment research and second language acquisition (SLA) published in English in peer-reviewed journals. Using Scopus, we created two datasets: (i) a dataset of core journals consisting of 1,561 articles published in four language assessment journals, and (ii) a dataset of general journals consisting of 3,175 articles on language assessment published in the top journals of SLA and applied linguistics. We applied document co-citation analysis to detect thematically distinct research clusters. Next, we coded citing papers in each cluster based on an analytical framework for measurement and validation. We found that the focus of the core journals was more exclusively on reading and listening comprehension assessment (primary), facets of speaking and writing performance such as raters and validation (secondary), as well as feedback, corpus linguistics, and washback (tertiary). By contrast, the primary focus of assessment research in the general journals was on vocabulary, oral proficiency, essay writing, grammar, and reading. The secondary focus was on affective schemata, awareness, memory, language proficiency, explicit vs. implicit language knowledge, language or semantic awareness, and semantic complexity. With the exception of language proficiency, this second area of focus was absent in the core journals. It was further found that the majority of citing publications in the two datasets did not carry out inference-based validation on their instruments before using them. More research is needed to determine what motivates authors to select and investigate a topic, how thoroughly they cite past research, and what internal (within a field) and external (between fields) factors lead to the sustainability of a Research Topic in language assessment.

A Scientometric Review of Alexithymia: Mapping Thematic and Disciplinary Shifts in Half a Century of Research

“…The title automatically extracted by the LLR method does not capture the main topic of the cluster, although it captures the fact that the majority of empirical studies mentioned here employ the TAS-20 in their design. In accordance with Aryadoust et al ( 114 ), highlighting the opportunity to employ cluster counter-labels based on expert evaluations of the content of documents within the cluster, we argue that an analysis of the major bursts suggests that a more appropriate title for this cluster would be “Emotion Information Processing.” Indeed, the most cited paper within the cluster is Lane et al ( 115 ): this paper puts forward the hypothesis that alexithymia is not simply a problem of emotion labeling, but it is “associated with impaired verbal and non-verbal recognition of emotion stimuli and that the hallmark of alexithymia, a difficulty in putting emotion into words, may be a marker of a more general impairment in the capacity for emotion information processing” ( 115 ). Consistently with this idea, the most pursued line of research consists in the exploration of deficits in the recognition of facial expressions among individuals with alexithymia ( 116 – 119 ).…”

A Scientometric Approach to Review the Role of the Medial Preoptic Area (MPOA) in Parental Behavior

“…For the present study, the sample of publications was downloaded from the database on Scopus, as done in previous scientometric publications [ 23 , 24 ]. A total of 677 scientific works, published between 01 January 1972 to 19 January 2021, was available on Scopus when using the following research string for the search: “( TITLE-ABS-KEY ( “medial preoptic” ) OR TITLE-ABS-KEY ( “MPOA” ) AND TITLE-ABS-KEY ( parent* ) OR TITLE-ABS-KEY ( matern* ) OR TITLE-ABS-KEY ( patern* ) OR TITLE-ABS-KEY ( mother* ) OR TITLE-ABS-KEY ( attach* ) OR TITLE-ABS-KEY ( nurtur* ) OR TITLE-ABS-KEY ( offspring ) OR TITLE-ABS-KEY ( pup* ) OR TITLE-ABS-KEY ( attack* ) OR TITLE-ABS-KEY ( infanticid* ) OR TITLE-ABS-KEY ( “young” )) AND (LIMIT-TO( LANGUAGE, “English”))”.…”

“…Betweenness centrality applies to single nodes to describe the degree in which a single node functions as a bridge to connect other nodes which would otherwise be separate. Centrality values range from 0 to 1, where high scores close to 1 indicates likely groundbreaking ideas [ 24 ]. For the analysis of single nodes, alongside the already mentioned structural metrics, temporal metrics were examined as well.…”

A Scientometric Approach to Review the Role of the Medial Preoptic Area (MPOA) in Parental Behavior

“…For the present study, the sample of publications was downloaded from the database on Scopus, as done in previous scientometric publications [23,24]. A total of 677 scientific works, published between 01 January 1972 to 19 January 2021, was obtained by using the following research string for the search: "( TITLE-ABS-KEY ( "medial preoptic" ) OR TITLE-ABS-KEY ( "MPOA" ) AND TITLE-ABS-KEY ( parent* ) OR TITLE-ABS-KEY ( matern* ) OR TITLE-ABS-KEY ( patern* ) OR TITLE-ABS-KEY ( mother* ) OR TITLE-ABS-KEY ( attach* ) OR TITLE-ABS-KEY ( nurtur* ) OR TITLE-ABS-KEY ( offspring ) OR TITLE-ABS-KEY ( pup* ) OR TITLE-ABS-KEY ( attack* ) OR TITLE-ABS-KEY ( infanticid* ) OR TITLE-ABS-KEY ( "young" )) AND (LIMIT-TO( LANGUAGE, "English"))".…”