The Fourth Dialog State Tracking Challenge

Kim, Seokhwan; D’Haro, Luis Fernando; Banchs, Rafael E.; Williams, Jason D.; Henderson, Matthew; Yoshino, Katsumi

doi:10.1007/978-981-10-2585-3_36

Cited by 90 publications

(67 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Taking the idea of the Let's Go system forward, the second and third DSTCs (Henderson et al, 2014b,c) have produced bootstrapped human-machine datasets for a restaurant search domain in the Cambridge area, UK. Since then, DSTCs have become one of the central research topics in the dialogue community (Kim et al, 2016(Kim et al, , 2017.…”

Section: Machine-to-machinementioning

confidence: 99%

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Budzianowski¹,

Wen²,

Tseng³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

741

920

View full text Add to dashboard Cite

Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available. To address this fundamental obstacle, we introduce the Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics. At a size of 10k dialogues, it is at least one order of magnitude larger than all previous annotated task-oriented corpora. The contribution of this work apart from the open-sourced dataset labelled with dialogue belief states and dialogue actions is two-fold: firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The proposed data-collection pipeline is entirely based on crowd-sourcing without the need of hiring professional annotators; secondly, a set of benchmark results of belief tracking, dialogue act and response generation is reported, which shows the usability of the data and sets a baseline for future studies.

show abstract

Section: Machine-to-machinementioning

confidence: 99%

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Budzianowski¹,

Wen²,

Tseng³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

741

920

View full text Add to dashboard Cite

show abstract

“…The second refers to the concatenation of the normalized representations before providing them to the output layer. This approach achieved 65.8%, 84.6%, and 71.4% accuracy on the Dialog State Tracking Challenge 4 (DSTC4) [20], ICSI Meeting Recorder Dialog Act Corpus (MRDA) [21], and SwDA corpora, respectively. However, the influence of context information varied across corpora.…”

Section: Related Workmentioning

confidence: 99%

A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization

Ribeiro

Matos

2019

Information

View full text Add to dashboard Cite

Automatic dialog act recognition is an important step for dialog systems since it reveals the intention behind the words uttered by its conversational partners. Although most approaches on the task use word-level tokenization, there is information at the sub-word level that is related to the function of the words and, consequently, their intention. Thus, in this study, we explored the use of character-level tokenization to capture that information. We explored the use of multiple character windows of different sizes to capture morphological aspects, such as affixes and lemmas, as well as inter-word information. Furthermore, we assessed the importance of punctuation and capitalization for the task. To broaden the conclusions of our study, we performed experiments on dialogs in three languages—English, Spanish, and German—which have different morphological characteristics. Furthermore, the dialogs cover multiple domains and are annotated with both domain-dependent and domain-independent dialog act labels. The achieved results not only show that the character-level approach leads to similar or better performance than the state-of-the-art word-level approaches on the task, but also that both approaches are able to capture complementary information. Thus, the best results are achieved by combining tokenization at both levels.

show abstract

“…SwDA (Jurafsky et al, 1997) Humanos Aberto Inglês 220k 41 -44 N MRDA (Shriberg et al, 2004) Humanos Reuniões Inglês 106k 5 / 11 + 39 N AMI (Carletta et al, 2005) Humanos Reuniões Inglês 102k 15 N VERBMOBIL (Kay et al, 1992) Humanos Horários Múltiplas 59k 42 / 33 M CHS (Levin et al, 1998) Humanos Aberto Espanhol 45k 10 / 37 N DSTC4 (Kim et al, 2017) Humanos Viagens Inglês 31k 89 S MapTask (Anderson et al, 1991) Humanos Mapas Inglês 27k 12 N DIHANA (Benedí et al, 2006) WoZ Comboios Espanhol 23k 11 + 10 + 13 M LEGO (Schmitt et al, 2012) Máquina Autocarros Inglês 14k 22 + 28 S NESPOLE (Costantini et al, 2002) Humanos Viagens Múltiplas 8k 67 + 91 M DIME (Villaseñor et al, 2001) WoZ Cozinhas Espanhol 5k 15 + 15 M Tabela 1: Corpora anotados com actos de diálogo, ordenados por número aproximado de segmentos. A coluna referenteà interacção diz se os diálogos são entre humanos ou existe um sistema de diálogo envolvido.…”

Section: Corpora Para Reconhecimento De Actos De Diálogounclassified

Reconhecimento de Actos de Diálogo Hierárquicos e Multi-Etiqueta em Dados em Espanhol

Ribeiro

Ribeiro²,

Matos³

2019

Linguamática

View full text Add to dashboard Cite

Os actos de diálogo revelam a intenção por trás das palavras pronunciadas. Por isso, o seu reconhecimento automático é importante para um sistema de diálogo que tenta entender o seu interlocutor. O estudo apresentado neste artigo aborda essa tarefa no corpus DIHANA, cujo esquema de anotação de actos de diálogo em três níveis coloca problemas que não foram explorados em estudos recentes. Além do problema hierárquico, os dois níveis inferiores colocam problemas de classificação multi-etiqueta. Além disso, cada nível da hierarquia refere-se a um aspecto diferente relativo à intenção do orador, tanto em termos da estrutura do diálogo, como da tarefa. Por outro lado, uma vez que os diálogos são em espanhol, este corpus permite-nos avaliar se as melhores abordagens para dados em inglês generalizam para uma língua diferente. Mais especificamente, comparamos o desempenho de diferentes abordagens de representação de segmentos, com foco tanto em sequências como em padrões de palavras, e avaliamos a importância do histórico do diálogo e das relações entre os múltiplos níveis da hierarquia. No que diz respeito ao problema de classificação de etiqueta única colocado pelo nível superior, mostramos que as conclusões obtidas a partir de dados em inglês se mantêm em dados em espanhol. Para além disso, mostramos que as abordagens podem ser adaptadas para cenários multi-etiqueta. Por fim, combinando hierarquicamente os melhores classificadores para cada nível, obtemos os melhores resultados reportados para este corpus.

show abstract

The Fourth Dialog State Tracking Challenge

Cited by 90 publications

References 6 publications

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization

Reconhecimento de Actos de Diálogo Hierárquicos e Multi-Etiqueta em Dados em Espanhol

Contact Info

Product

Resources

About