The CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue

Abstract

In this paper, we provide an overview of the CODI-CRAC 2021 Shared Task. The shared task focuses on detecting anaphoric relations in different genres of conversations. Using five conversational datasets, four of which have been newly annotated with a wide range of anaphoric relations: identity, bridging references and discourse deixis, we defined multiple tasks focusing individually on these key relations. We discuss the evaluation scripts used to assess the system performance on these tasks, and provide a brief summary of the participating systems and the results obtained across 115 runs from six teams, with most submissions achieving significantly better results than our baseline methods.

Scoring Coreference Chains with Split-Antecedent Anaphors

“…However, many other types of anaphoric reference exist beyond basic identity anaphora, including, e.g., anaphoric reference depending on relations other than identity reference, and anaphoric reference to entities not introduced using nominals. These types of anaphoric reference were not covered in ONTONOTES because of complexity and cost reasons (see Pradhan et al (2012) as well as the discussion in Zeldes ( 2022)), but are covered in many of the most recent corpora, including ANCORA Recasens and Martí (2010), ARRAU Poesio and Artstein (2008); Poesio et al (2018); Uryupina et al (2020), GUM Zeldes (2017), Phrase Detectives Poesio et al (2019), the Prague Dependency Treebank Nedoluzhko (2013), the T ÜBA-DZ corpus Versley (2008) and the recently created CODI/CRAC corpus of anaphoric reference in dialogue Khosla et al (2021). The objective of the UA initiative is to define a common scheme to annotate all types of anaphoric reference and to develop a scorer that can be used to evaluate anaphora resolution with all these datasets.…”

“…Although no Reference scorer for bridging reference existed when we started work on the Universal Anaphora scorer, approaches for evaluating bridging reference resolution did exist, in particular the approach from Hou et al (2018) already used for the CODI/CRAC 2018 Shared Task Poesio et al (2018). This method was integrated in the Universal Anaphora scorer Khosla et al (2021).…”

“…The extension proposed in this paper can then be immediately used for the cases of split-antecedent discourse deixis such as (1). It's not clear however whether such an extension is needed for bridging references, as the evaluation metric proposed by Hou et al (2018) and used both in the 2018 CRAC Shared Task and in the 2021 CODI/CRAC shared task (and implemented in the UA scorer) (Khosla et al, 2021) appears adequate.…”

“…Split-antecedent plural anaphors can be found in all corpora and in all languages, so an increasing number of annotation schemes cover them, including ANCORA Recasens and Martí (2010), ARRAU Poesio et al (2018); Uryupina et al (2020), GUM Zeldes (2017), FRIENDS Zhou and Choi (2018), Phrase Detectives Poesio et al (2019), the Prague Dependency Treebank Nedoluzhko (2013), and the recently created CODI/CRAC 2021 Shared Task corpus of anaphora resolution in dialogue Khosla et al (2021). 2 Split-antecedent anaphors are not always common but e.g., in the FRIENDS corpus 9% of the mentions have more than one antecedent Zhou and Choi (2018).…”

Anaphora Resolution in Dialogue: Description of the DFKI-TalkingRobots System for the CODI-CRAC 2021 Shared-Task

“…In this paper we present our systems submitted for the CODI-CRAC 2022 Shared Task (CCST) on Anaphora, Bridging, and Discourse Deixis in Dialogue 2 (Yu et al, 2022). The task is a follow-up to the one held last year and described in Khosla et al (2021). As its name suggests, besides identity anaphora this shared task tries to cover other, less-studied, anaphoric phenomena, and offers new multi-genre data that combines several types of annotations in Universal Anaphora 3 format.…”