Aligning Visual and Lexical Semantics

Giunchiglia, Fausto; Bagchi, Mayukh; Diao, Xiaolei

doi:10.1007/978-3-031-28035-1_20

Cited by 5 publications

(3 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work, we plan to automate the method presented in this paper and apply it to new languages, such as the rest of the Arabic dialects and Indonesian language, as well as to new domains that are known to be diverse, such as body parts, food, color, or visual objects (Giunchiglia and Bagchi, 2021;Giunchiglia et al, 2023).…”

Section: Discussionmentioning

confidence: 99%

Lexical diversity in kinship across languages and dialects

Khalilia,

Bella,

Freihat

et al. 2023

Front. Psychol.

View full text Add to dashboard Cite

Languages are known to describe the world in diverse ways. Across lexicons, diversity is pervasive, appearing through phenomena such as lexical gaps and untranslatability. However, in computational resources, such as multilingual lexical databases, diversity is hardly ever represented. In this paper, we introduce a method to enrich computational lexicons with content relating to linguistic diversity. The method is verified through two large-scale case studies on kinship terminology, a domain known to be diverse across languages and cultures: one case study deals with seven Arabic dialects, while the other one with three Indonesian languages. Our results, made available as browseable and downloadable computational resources, extend prior linguistics research on kinship terminology, and provide insight into the extent of diversity even within linguistically and culturally close communities.

show abstract

Section: Discussionmentioning

confidence: 99%

Lexical diversity in kinship across languages and dialects

Khalilia,

Bella,

Freihat

et al. 2023

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…The second case labels which are synonyms (i.e., of manyto-one mapping between labels and images) wherein a single set of images, will wrongly be split into two sets. One such example is that of Koto which can also be synonymously labelled as, for instance, a Kin or a Jusangen [13].…”

Section: Sources Of Annotation Mistakesmentioning

confidence: 99%

A Semantics-Driven Methodology for High-Quality Image Annotation

Giunchiglia,

Bagchi,

Diao

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Recent work in Machine Learning and Computer Vision has highlighted the presence of various types of systematic flaws inside ground truth object recognition benchmark datasets. Our basic tenet is that these flaws are rooted in the many-to-many mappings which exist between the visual information encoded in images and the intended semantics of the labels annotating them. The net consequence is that the current annotation process is largely under-specified, thus leaving too much freedom to the subjective judgment of annotators. In this paper, we propose vTelos, an integrated Natural Language Processing, Knowledge Representation, and Computer Vision methodology whose main goal is to make explicit the (otherwise implicit) intended annotation semantics, thus minimizing the number and role of subjective choices. A key element of vTelos is the exploitation of the WordNet lexico-semantic hierarchy as the main means for providing the meaning of natural language labels and, as a consequence, for driving the annotation of images based on the objects and the visual properties they depict. The methodology is validated on images populating a subset of the ImageNet hierarchy.

show abstract

“…As an example consider Fig. 1, taken from a small classification of musical instruments [13]. In this figure (left), we can see how the meaning of each label is provided in terms of a genus and a differentia, and where the genus one level down is the label of the concept the level up.…”

Section: Introductionmentioning

confidence: 99%

Egocentric Hierarchical Visual Semantics

Erculiani

Bontempelli

Passerini

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

We are interested in aligning how people think about objects and what machines perceive, meaning by this the fact that object recognition, as performed by a machine, should follow a process which resembles that followed by humans when thinking of an object associated with a certain concept. The ultimate goal is to build systems which can meaningfully interact with their users, describing what they perceive in the users’ own terms. As from the field of Lexical Semantics, humans organize the meaning of words in hierarchies where the meaning of, e.g., a noun, is defined in terms of the meaning of a more general noun, its genus, and of one or more differentiating properties, its differentia. The main tenet of this paper is that object recognition should implement a hierarchical process which follows the hierarchical semantic structure used to define the meaning of words. We achieve this goal by implementing an algorithm which, for any object, recursively recognizes its visual genus and its visual differentia. In other words, the recognition of an object is decomposed in a sequence of steps where the locally relevant visual features are recognized. This paper presents the algorithm and a first evaluation.

show abstract

Aligning Visual and Lexical Semantics

Cited by 5 publications

References 52 publications

Lexical diversity in kinship across languages and dialects

Lexical diversity in kinship across languages and dialects

A Semantics-Driven Methodology for High-Quality Image Annotation

Egocentric Hierarchical Visual Semantics

Contact Info

Product

Resources

About