Machine Vision and Social Media Images: Why Hashtags Matter

Abstract

Studying images in social media poses specific methodological challenges, which in turn have directed scholarly attention toward the computational interpretation of visual data. When analyzing large numbers of images, both traditional content analysis as well as cultural analytics have proven valuable. However, these techniques do not take into account the contextualization of images within a socio-technical environment. As the meaning of social media images is co-created by online publics, bound through networked practices, these visuals should be analyzed on the level of their networked contextualization. Although machine vision is increasingly adept at recognizing faces and features, its performance in grasping the meaning of social media images remains limited. Combining automated analyses of images with platform data opens up the possibility to study images in the context of their resonance within and across online discursive spaces. This article explores the capacities of hashtags and retweet counts to complement the automated assessment of social media images, doing justice to both the visual elements of an image and the contextual elements encoded through the hashtag practices of networked publics.

Affordances-Based Brand Relations: An Inquire on Memetic Brands on Instagram

“…To analyze the content of a such big collection of pictures, we took advantage of a software for automated visual analysis: Google Vision API (GVA), a machine learning-based image recognition toolkit provided as a service by Google (Geboers & Van de Wiele, 2020;Mulfari et al, 2016). Although very useful, GVA releases an output that is not intuitive to interpret and, especially, not immediately usable to meet academic research objectives (Mintz & Silva, 2019).…”

“…Arguably, affectivity is the property that structures affective publics and keeps them together, even though it amounts to be a very ephemeral entity. In fact, affect is not an emotion: it is a generic flow that coalesces into a specific emotion according to the specific goals of a public (Geboers & Van de Wiele, 2020). This double movement between affect and emotions is particularly visible in visual vernaculars developing within affective publics on social media (Niederer, 2016).…”

Visual media analysis for Instagram and other online platforms

“…When studying social media images, the labelling of image content has been critiqued for its inability to appreciate the 'social value of the picture', which includes the intention of the uploader, such as 'social capital, self-image and memory' (Bechmann, 2017). Researchers emphasise how the labelling should be accompanied by data enhancement, namely an additional data layer, such as hashtags (Geboers and Van De Wiele, 2020).…”

“…Platform vernacular research, it should be pointed out, ought not ignore the absence of a typical user of a social media platform (Gerlitz and Rieder, 2013), let alone one with over one billion registered users, as Instagram. There are myriad uses of platform, be they documenting everyday life, styling like an influencer, campaigning or posting pictures of kittens (Caliandro and Graham, 2020); diversity of use lies within countries and cultures (Leaver et al., 2020).…”

A Graph Theory approach to assess nature’s contribution to people at a global scale

“…These studies have generally proved to be comparable to traditional surveys 21 . The use of social media data initially relied on photo content assessment (but see Geboers et al 22 ). The context and content of the photographs is classified into cultural ecosystem services’ categories based on the presence or absence of specific elements in the photos, such as views of flora and fauna, historical buildings, or touristic infrastructure and facilities 23 .…”