Cooking with Semantics

Malmaud, Jonathan; Wagner, Earl J.; Chang, Nancy; Murphy, Kevin

doi:10.3115/v1/w14-2407

Cited by 54 publications

(37 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Localizing Video Segments with Natural Language. Prior work has considered aligning natural language with video, e.g., instructional videos with transcribed text (Kiddon et al, 2015;Huang et al, 2017;Malmaud et al, 2014Malmaud et al, , 2015. Our work is most related to recent work in video moment retrieval with natural language (Gao et al, 2017;Hendricks et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

Localizing Moments in Video with Temporal Language

Hendricks¹,

Wang²,

Shechtman³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

113

107

View full text Add to dashboard Cite

Localizing moments in a longer video via natural language queries is a new, challenging task at the intersection of language and video understanding. Though moment localization with natural language is similar to other language and vision tasks like natural language object retrieval in images, moment localization offers an interesting opportunity to model temporal dependencies and reasoning in text.We propose a new model that explicitly reasons about different temporal segments in a video, and shows that temporal context is important for localizing phrases which include temporal language. To benchmark whether our model, and other recent video localization models, can effectively reason about temporal language, we collect the novel TEMPOral reasoning in video and language (TEMPO) dataset. Our dataset consists of two parts: a dataset with real videos and template sentences (TEMPO -Template Language) which allows for controlled studies on temporal language, and a human language dataset which consists of temporal sentences annotated by humans (TEMPO -Human Language).

show abstract

Section: Related Workmentioning

confidence: 99%

Localizing Moments in Video with Temporal Language

Hendricks¹,

Wang²,

Shechtman³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

113

107

View full text Add to dashboard Cite

show abstract

“…For example, an approach has been developed to translate ifthis-than-that constructs from natural language into executable programs [9]. Other approaches are more domain specific, such as focusing on cooking recipes [4]. Similarly to Controlled Languages, most of those approaches rely on certain structural or lexical properties of the instructions to extract their meaning.…”

Section: Related Workmentioning

confidence: 99%

“…Humans are capable of specifying complex workflows based on human actions. Online instructions, such as those available on the wikiHow 4 website, often include different methods, steps, loops, and conditions. The main disadvantage of human instructions is that they are not machine understandable.…”

Section: Introductionmentioning

confidence: 99%

Linking Data, Services and Human Know-How

Pareti

Klein

Barker

2016

The Semantic Web. Latest Advances and New Domains

View full text Add to dashboard Cite

Abstract. An increasing number of everyday tasks involve a mixture of human actions and machine computation. This paper presents the first framework that allows non-programmer users to create and execute workflows where each task can be completed by a human or a machine. In this framework, humans and machines interact through a shared knowledge base which is both human and machine understandable. This knowledge base is based on the prohow Linked Data vocabulary that can represent human instructions and link them to machine functionalities. Our hypothesis is that non-programmer users can describe how to achieve certain tasks at a level of abstraction which is both human and machine understandable. This paper presents the prohow vocabulary and describes its usage within the proposed framework. We substantiate our claim with a concrete implementation of our framework and by experimental evidence.

show abstract

“…Other approaches for getting more context for the cooking recipe analysis are dynamic data structures (i.e. latent vector used by (Malmaud et al, 2014)) and graph-shaped resources (ontologies, semantic networks) as in our case.…”

Section: Cooking Instruction Analysismentioning

confidence: 99%

“…In the paradigm of semantic role labeling, the approach of (Malmaud et al, 2014) uses a Markov decision process where ingredients and utensils are propagated over the temporal order of instructions and where the context information is stored in a latent vector which disambiguates and augments the instruction statement under analysis. The context information corresponds to the state of the kitchen and integrates the changes of this state according to the evolving recipe instructions.…”

Section: Cooking Instruction Analysismentioning

confidence: 99%

Dish Classification using Knowledge based Dietary Conflict Detection

Clairet¹

2017

RANLP - Student Research Workshop

View full text Add to dashboard Cite

The present paper considers the problem of dietary conflict detection from dish titles. The proposed method explores the semantics associated with the dish title in order to discover a certain or possible incompatibility of a particular dish with a particular diet. Dish titles are parts of the elusive and metaphoric gastronomy language, their processing can be viewed as a combination of short text and domain-specific texts analysis. We build our algorithm on the basis of a common knowledge lexical semantic network and show how such network can be used for domain specific short text processing.

show abstract

Cooking with Semantics

Cited by 54 publications

References 16 publications

Localizing Moments in Video with Temporal Language

Localizing Moments in Video with Temporal Language

Linking Data, Services and Human Know-How

Dish Classification using Knowledge based Dietary Conflict Detection

Contact Info

Product

Resources

About