Social validation of collective annotations: Definition and experiment

Abstract

People taking part in argumentative debates through collective annotations face a highly cognitive task when trying to estimate the group's global opinion. In order to reduce this effort, we propose in this paper to model such debates prior to evaluating their "social validation." Computing the degree of global confirmation (or refutation) enables the identification of consensual (or controversial) debates. Readers as well as prominent information systems may thus benefit from this information. The accuracy of the social validation measure was tested through an online study conducted with 121 participants. We compared their human perception of consensus in argumentative debates with the results of the three proposed social validation algorithms. Their efficiency in synthesizing opinions was demonstrated by the fact that they achieved an accuracy of up to 84%. Introduction and MotivationsAnnotating paper documents is a common activity practiced since the early middle ages (Fraenkel & Klein, 1999). Field studies show that readers still make extensive use of annotations nowadays (Marshall, 1998;Wolfe & Neuwirth, 2001). Although seemingly insignificant, they actually facilitate key purposes such as "active reading" by supporting critical thinking while reading (Adler & van Doren, 1972), learning by facilitating document appropriation, and proofreading, among many others. With the widespread adoption of digital documents both in the workplace and at home, people have felt frustrated at not being able to annotate them (Sellen & Harper, 2003, p. 96). Such a need led both research labs and companies to design a plethora of annotation systems-mostly targeting Web documents-since the 1990s (Wolfe, 2002). At first, they implemented the usual paper-based annotation functions, mainly for personal use. Then, taking advantage of modern computer storage and networking capabilities, annotation systems provided novel features for collective use. In particular, they enabled annotation sharing through dedicated servers, so that users could view and access them in context, i.e., within the annotated document. Moreover, subsequent readers may later reply to any annotation or any reply in turn, thus forming a debate anchored to the discussed passage. Such a debate is also called a "discussion thread" (e.g., Figure 1).As an asynchronous way of communicating, collective annotations (i.e., annotations along with their discussion threads) are useful at two levels:• From the readers' point of view, collective annotations enable them to discuss document passages in context. This is an advantage in comparison with Internet bulletin boards and forums where one needs to explain the context of his/her post to be understood. Moreover, feedback about a given annotated document is directly accessible to readers, whereas it would be scattered over multiple sources otherwise. Besides being useful for readers, authors also benefit from collective annotations as they can improve their documents by taking into account the associated remarks.• For syste...

Functionalities provided by Annotation Systems for Learners in Educational Context: An Overview

“…Electronic annotation tools may also limit a reader's expressive intent (e.g., forcing a highlight to be continuous when the reader wants to fragment it, or imposing neatness on a reader when she wants to scrawl). Sometimes the electronic annotations are stored in infelicitous ways so they are either gone when the learner returns to the eBook on a different computer or so they are recoverable when the reader believes them to be deleted and loans the book or document to a colleague [41].…”

“…Many researchers have been interested in the creation of annotation tools to facilitate learner's annotative activity in digital environment. We can find numerous commercial software and research prototypes created to annotate electronic educational resources such as: TafAnnote [41]; Notebook [26]; Crocodoc [46]; New-WebAnnot [2]; etc.…”

“…Storage in RDF format improves interoperability and sharing data between annotation systems. Therefore, Cabanac et al [41] indicate that "RDF based annotation can be used by different software if they structure the data in the same way". Besides, some annotation systems enrich the practice of electronic annotation by making of every annotation an entry point for a debate in context.…”

“…Furthermore, annotations are not only a way of explaining and enriching an information resource with personal learner observations, but also a means of transmitting and sharing ideas to improve collaborative work practices [41]. Thus, the possibility of enriching digital contents by adding annotations has attracted many researchers, who have looked at this opportunity from many different perspectives and with a number of purposes in mind [3,29,39].…”

A new annotation system based on a semantic analysis of a learner's annotative activity to invoke web services

“…Among these activities, we focus in our research on the annotative activity of the learner because annotation practice is very common and omnipresent. While reading, the learner usually uses comments, highlights, circles sections and posts it to annotate the consulted resources (Cabanac et al, 2010;Marshall, 2009).…”