A Survey of Methods for Improving Review Quality

Gehringer, Edward F.

doi:10.1007/978-3-319-13296-9_10

Cited by 15 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future research could also consider whether an intervention can enhance communication quality in GVTs. In this regard, two initiatives that seem promising for increasing communication quality, particularly that which occurs within the peer feedback context, might be calibration training (see Gehringer, 2014) and double loop mutual assessments (see Babik, Iyer, & Ford, 2012).…”

Section: Strengths Limitations and Additional Directions For Future R...mentioning

confidence: 99%

Global virtual team communication, coordination, and performance across three peer feedback strategies.

McLarnon

O’Neill

Taras

et al. 2019

Canadian Journal of Behavioural Science / Revue canadienne des

View full text Add to dashboard Cite

Global virtual teams (GVTs) are faced with numerous communication and coordination difficulties. This study examined whether peer feedback, implemented via a quasi-experiment, strengthened linkages between GVT communication, coordination, and performance. Drawing upon self-regulation as a critical mechanism underlying team effectiveness, we proposed a model in which the indirect relation between communication and performance, mediated by process coordination, would be stronger as peer feedback intensity increased. The effects of 3 feedback strategies were investigated: (a) feedback given and received only at project completion, (b) feedback given weekly, but not distributed until project completion, and (c) feedback given and received weekly. We examined this model using a large sample of GVTs (n ϭ 1,839 teams; n ϭ 13,224 individuals) and multilevel, multisource data. Results supported a stronger indirect effect between communication frequency and performance, via process coordination, when GVT members gave and received weekly feedback. These findings suggest GVTs should use structured peer feedback systems that allows giving and receiving regular feedback. Further implications for research and practice are considered. Public Significance StatementGlobal virtual teams (GVTs) are prevalent in today's organizations. Yet despite their commonality, they may suffer from numerous teamwork challenges related to separation in time, geography, and language proficiency. Results of this study suggest that GVTs should use a peer feedback system that allows giving and receiving regular intrateam feedback, as such a system can enhance the relations between communication frequency, coordination, and GVT performance.

show abstract

Section: Strengths Limitations and Additional Directions For Future R...mentioning

confidence: 99%

Global virtual team communication, coordination, and performance across three peer feedback strategies.

McLarnon

O’Neill

Taras

et al. 2019

Canadian Journal of Behavioural Science / Revue canadienne des

View full text Add to dashboard Cite

show abstract

“…Peer-grading as a specific aspect of peer-assessment is especially challenging when it comes to teaching at scale. The situation was ameliorated with the introduction of tools like CrowdGrader 3 (Dasgupta & Ghosh, 2013;De Alfaro & Shavlovsky, 2014) and various massively open online courses (MOOCs) platforms (Gehringer, 2014). De Alfaro and Shavlovsky (2014) found that the grades computed by CrowdGrader were precise and suitable for student homework evaluation.…”

Section: Peer Assessment As Grading Methodsmentioning

confidence: 99%

''To Prove There Is Another Way'': A Qualitative Study on Digital Tools for Education in the Storstrøm Prison of Denmark

Chatzipolakis,

Yasuoka,

Georgoulakis

2024

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Teaching research methods is important in any curriculum that prepares students for an academic career. While theoretical frameworks for qualitative theory building can be adequately conveyed through lecturing, the practices of qualitative data analysis (QDA) cannot. However, using experiential learning techniques for teaching QDA methods to large numbers of students presents a challenge to the instructor due to the effort required for the grading of homework. Any homework involving the coding of qualitative data will result in a myriad of different interpretations of the same data with varying quality. Grading such assignments requires significant effort. We approached this problem by using methods of inter-rater agreement and a model solution as a proxy for the quality of the submission. The automated agreement data serves as the foundation for a semi-automated grading process. Within this paper, we demonstrate that this proxy has a high correlation with the manual grading of submissions.

show abstract

“…We explored existing literature to conduct a systematic analysis of the multitude, variety, and complexity of such implementations, functionalities, and design choices in OPRA. Several attempts have been made to survey computerized peerassessment practices (Bouzidi & Jaillet, 2009;Chang et al, 2021;Davies, 2000;Doiron, 2003;Gikandi et al, 2011;Luxton-Reilly, 2009;Tenório et al, 2016;Topping, 2005), or some specific aspects of peer assessment, such as approaches to reliability and validity of peer evaluations (Gehringer, 2014;Misiejuk & Wasson, 2021;Patchan et al, 2017). However, meta-analysis of OPRA systems is complicated because their design space has high dimensionality; OPRA practices and designs vary across many disciplines in many different ways (Søndergaard & Mulder, 2012).…”

Section: Literature Reviewmentioning

confidence: 99%

A systematic review of educational online peer-review and assessment systems: charting the landscape

Babik,

Gehringer,

Kidd

et al. 2024

Education Tech Research Dev

View full text Add to dashboard Cite

Over the past two decades, there has been an explosion of innovation in software tools that encapsulate and expand the capabilities of the widely used student peer assessment. While the affordances and pedagogical impacts of traditional in-person, “paper-and-pencil” peer assessment have been studied extensively and are relatively well understood, computerized (online) peer assessment introduced not only shifts in scalability and efficiency, but also entirely new capabilities and forms of social learning interactions, instructor leverage, and distributed cognition, that still need to be researched and systematized. Despite the ample research on traditional peer assessment and evidence of its efficacy, common vocabulary and shared understanding of online peer-assessment system design, including the variety of methods, techniques, and implementations, is still missing. We present key findings of a comprehensive survey based on a systematic research framework for examining and generalizing affordances and constraints of online peer-assessment systems. This framework (a) provides a foundation of a design-science metatheory of online peer assessment, (b) helps structure the discussion of user needs and design options, and (c) informs educators and system design practitioners. We identified two major themes in existing and potential research—orientation towards scaffolded learning vs. exploratory learning and system maturity. We also outlined an agenda for future studies.

show abstract

A Survey of Methods for Improving Review Quality

Cited by 15 publications

References 10 publications

Global virtual team communication, coordination, and performance across three peer feedback strategies.

Global virtual team communication, coordination, and performance across three peer feedback strategies.

''To Prove There Is Another Way'': A Qualitative Study on Digital Tools for Education in the Storstrøm Prison of Denmark

A systematic review of educational online peer-review and assessment systems: charting the landscape

Contact Info

Product

Resources

About