Adapting Event Extractors to Medical Data: Bridging the Covariate Shift

Abstract

We tackle the task of adapting event extractors to new domains without labeled data, by aligning the marginal distributions of source and target domains. As a testbed, we create two new event extraction datasets using English texts from two medical domains: (i) clinical notes, and (ii) doctor-patient conversations. We test the efficacy of three marginal alignment techniques: (i) adversarial domain adaptation (ADA), (ii) domain adaptive fine-tuning (DAFT), and (iii) a new instance weighting technique based on language model likelihood scores (LIW). LIW and DAFT improve over a no-transfer BERT baseline on both domains, but ADA only improves on notes. Deeper analysis of performance under different types of shifts (e.g., lexical shift, semantic shift) explains some of the variations among models. Our best-performing models reach F1 scores of 70.0 and 72.9 on notes and conversations respectively, using no labeled target data.

Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks

“…Studies that present negative transfer results (e.g., Plank et al, 2014 ) are rare, but should be encouraged to develop a sound understanding of adaptation techniques. Analyses should also study ties between datasets used and methods applied, highlighting dimensions of variation between source-target domains and how adaptation methods bridge them (Kashyap et al, 2020 ; Naik et al, 2021 ). Such analyses can uncover important lessons about generalizability of adaptation methods and the kinds of source-target settings they can be expected to improve performance on.…”

“… i2b2 2012 (Sun et al, 2013 ): Discharge summaries annotated with events. MTSamples (Naik et al, 2021 ): Medical records annotated with events (test-only). …”

Year 2021: COVID-19, Information Extraction and BERTization among the Hottest Topics in Medical Natural Language Processing

“…• detection of events [29,30], including self-harm events [31]; • extraction of diagnoses [13,[32][33][34][35] and their codes [36][37][38]; • recognition of named entities [5,14,[39][40][41], and more specifically of personal information [21,42,43] and family history [20]; • localization of advices [44] and arguments [45] in scientific literature; • extraction of relations [46][47][48], including temporal [49] and causality [50,51] relations.…”