Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Goot, Rob van der; Üstün, Ahmet; Ramponi, Alan; Sharaf, Ibrahim; Plank, Barbara

doi:10.18653/v1/2021.eacl-demos.22

Cited by 36 publications

(28 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We split the top-10 among the data splits (i.e., train, development, and test set), and also between source splits (i.e., BIG, HOUSE, TECH). We use the default hyperparameters in MACHAMP (van der Goot et al, 2021) as shown in Table 4. For more details we refer to their paper.…”

Section: Type Of Skills Annotatedmentioning

confidence: 99%

SkillSpan: Hard and Soft Skill Extraction from English Job Postings

Zhang¹,

Jensen²,

Sonniks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Skill Extraction (SE) is an important and widely-studied task useful to gain insights into labor market dynamics. However, there is a lacuna of datasets and annotation guidelines; available datasets are few and contain crowd-sourced labels on the span-level or labels from a predefined skill inventory. To address this gap, we introduce SKILLSPAN, a novel SE dataset consisting of 14.5K sentences and over 12.5K annotated spans. We release its respective guidelines created over three different sources annotated for hard and soft skills by domain experts. We introduce a BERT baseline (Devlin et al., 2019). To improve upon this baseline, we experiment with language models that are optimized for long spans (Joshi et al., 2020;Beltagy et al., 2020), continuous pre-training on the job posting domain (Han and Eisenstein, 2019; Gururangan et al., 2020), and multi-task learning (Caruana, 1997). Our results show that the domainadapted models significantly outperform their non-adapted counterparts, and single-task outperforms multi-task learning. * Equal contribution.You will thrive working in a Dev/Sec Ops culture . SKILL KNOWLEDGEThe ability to manage large sections of guests . SKILL Knowledge of property law rules of Germany .

show abstract

Section: Type Of Skills Annotatedmentioning

confidence: 99%

SkillSpan: Hard and Soft Skill Extraction from English Job Postings

Zhang¹,

Jensen²,

Sonniks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both the Bi-LSTM (Plank et al, 2016) and the MaChAmp (van der Goot et al, 2021) toolkit are capable of Multi Task Learning (MTL) (Caruana, 1997). We therefore, set up a number of experiments testing the impact of three different auxiliary tasks.…”

Section: Auxiliary Tasksmentioning

confidence: 99%

De-identification of Privacy-related Entities in Job Postings

Jensen¹,

Zhang²,

Plank³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

De-identification is the task of detecting privacy-related entities in text, such as person names, emails and contact data. It has been well-studied within the medical domain. The need for deidentification technology is increasing, as privacy-preserving data handling is in high demand in many domains. In this paper, we focus on job postings. We present JOB-STACK, a new corpus for de-identification of personal data in job vacancies on Stackoverflow. We introduce baselines, comparing Long-Short Term Memory (LSTM) and Transformer models. To improve upon these baselines, we experiment with contextualized embeddings and distantly related auxiliary data via multi-task learning. Our results show that auxiliary data improves de-identification performance. Surprisingly, vanilla BERT turned out to be more effective than a BERT model trained on other portions of Stackoverflow.

show abstract

“…Models are fine-tuned for 100,000 steps with batch size of 16. For downstream tasks, we use MaChAmp (van der Goot et al, 2021) and train our models for 10 epochs. The best checkpoints were selected based on performance on the dev sets.…”

Section: Frameworkmentioning

confidence: 99%

On Language Models for Creoles

Lent,

Bugliarello,

de Lhoneux

et al. 2021

Preprint

View full text Add to dashboard Cite

Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with multiple local languages, and what grammatical and lexical features are transferred to the creole is a complex process (Sessarego, 2020). While creoles are generally stable, the prominence of some features may be much stronger with certain demographics or in some linguistic situations (Winford, 1999;Patrick, 1999). This paper makes several contributions: We collect existing corpora and release models for Haitian Creole, Nigerian Pidgin English, and Singaporean Colloquial English. We evaluate these models on intrinsic and extrinsic tasks. Motivated by the above literature, we compare standard language models with distributionally robust ones and find that, somewhat surprisingly, the standard language models are superior to the distributionally robust ones. We investigate whether this is an effect of overparameterization or relative distributional stability, and find that the difference persists in the absence of over-parameterization, and that drift is limited, confirming the relative stability of creole languages.

show abstract

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Cited by 36 publications

References 80 publications

SkillSpan: Hard and Soft Skill Extraction from English Job Postings

SkillSpan: Hard and Soft Skill Extraction from English Job Postings

De-identification of Privacy-related Entities in Job Postings

On Language Models for Creoles

Contact Info

Product

Resources

About