Representations of machine vision technologies in artworks, games and narratives: A dataset

Rettberg, Jill Walker; Kronman, Linda; Solberg, Ragnhild; Gunderson, Marianne; Bjørklund, Stein Magne; Stokkedal, Linn Heidi; Seta, Gabriele de; Jacob, Kurdin; Markham, Annette N.

doi:10.1016/j.dib.2022.108319

Cited by 10 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study could be repeated and used for other XAI algorithms, such GradCAM, to improve auguring problems. Furthermore, algorithms that mimic natural occurrences can be applied to heterogeneous datasets for false imaging modalities, such as the most current developments in computational capacity, deepfake technologies, and digital phenotyping tools [ 54 ].…”

Section: Discussionmentioning

confidence: 99%

Auguring Fake Face Images Using Dual Input Convolution Neural Network

et al. 2022

View full text Add to dashboard Cite

Deepfake technology uses auto-encoders and generative adversarial networks to replace or artificially construct fine-tuned faces, emotions, and sounds. Although there have been significant advancements in the identification of particular fake images, a reliable counterfeit face detector is still lacking, making it difficult to identify fake photos in situations with further compression, blurring, scaling, etc. Deep learning models resolve the research gap to correctly recognize phony images, whose objectionable content might encourage fraudulent activity and cause major problems. To reduce the gap and enlarge the fields of view of the network, we propose a dual input convolutional neural network (DICNN) model with ten-fold cross validation with an average training accuracy of 99.36 ±0.62, a test accuracy of 99.08 ± 0.64, and a validation accuracy of 99.30 ± 0.94. Additionally, we used ’SHapley Additive exPlanations (SHAP) ’ as explainable AI (XAI) Shapely values to explain the results and interoperability visually by imposing the model into SHAP. The proposed model holds significant importance for being accepted by forensics and security experts because of its distinctive features and considerably higher accuracy than state-of-the-art methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Auguring Fake Face Images Using Dual Input Convolution Neural Network

et al. 2022

View full text Add to dashboard Cite

show abstract

“…We developed the concept of the machine vision situation as we worked on the Database of Machine Vision in Art, Games, and Narratives . Completed in 2021, the database documents 77 digital games, 190 digital artworks, and 233 movies, novels, and other narratives that use or represent machine vision technologies ( Rettberg et al , 2022a ; Rettberg et al , 2022b ). Within a total corpus of 500 works, we have identified and analysed 874 specific situations where machine vision is central ( Rettberg et al , 2022a ; Rettberg et al , 2022b ).…”

Section: Machine Vision Situationsmentioning

confidence: 99%

“…Completed in 2021, the database documents 77 digital games, 190 digital artworks, and 233 movies, novels, and other narratives that use or represent machine vision technologies ( Rettberg et al , 2022a ; Rettberg et al , 2022b ). Within a total corpus of 500 works, we have identified and analysed 874 specific situations where machine vision is central ( Rettberg et al , 2022a ; Rettberg et al , 2022b ). Based on the definition of machine vision mentioned above, and an initial survey of creative works, we developed a list of 26 different technologies that formed the basis for identifying machine vision situations in the works.…”

Section: Machine Vision Situationsmentioning

confidence: 99%

Machine vision situations: Tracing distributed agency

Gunderson,

Solberg,

Kronman

et al. 2024

Open Res Europe

View full text Add to dashboard Cite

This article proposes a new method for tracing and examining agency in heterogeneous assemblages, focusing on the role of machine vision technologies in creative works. We introduce the concept of the “machine vision situation” and define it as the moment in which machine vision technologies come into play and make a difference to the course of events. By taking situations as the unit of analysis, we identify moments at which machine vision technologies take part in actions without reducing them to either tools or protagonists, instead allowing for more complex agential entanglements between human and non-human actors. Grounded on an interdisciplinary theoretical framework, this article demonstrates how an analytical unit such as the machine vision situation is a valuable method for tracing how agency is distributed. We illustrate this through three examples by applying the method to creative works – narratives, digital games, and artworks – revealing key aspects of distributed agency and calling attention to the excess, complications, and messy entanglements that might otherwise be overlooked in analyses of agential assemblages. The machine vision situation is shown to be a flexible unit of analysis that can be productively incorporated in both quantitative and qualitative studies and applied to other contexts in which human and non-human agencies interact.

show abstract

“…The dataset I will test the method on is from the Database of Machine Vision in Art, Games and Narratives , which contains structured analyses of how machine vision technologies are represented and used in 500 digital artworks, video games, novels and movies. The dataset is fully documented in Rettberg et al (2022b). The subset of data used for this test consists of 747 verbs describing an interaction between fictional characters and machine vision technologies.…”

Section: Introductionmentioning

confidence: 99%

“… Data Availability Statement The software used in the analysis is available on Github at https://github.com/jilltxt/algorithmicfailure and consists of a file with R scripts and comments explaining the code (Rettberg, 2022). The dataset is available on DataverseNO (Rettberg et al, 2022a), and there is also a data paper providing documentation (Rettberg et al 2022b). The Database of Machine Vision in Art, Games and Narratives can also be viewed in a web browser at https://machine-vision.no.…”

mentioning

confidence: 99%

Algorithmic failure as a humanities methodology: Machine learning's mispredictions identify rich cases for qualitative analysis

Rettberg

2022

Big Data & Society

Self Cite

View full text Add to dashboard Cite

This commentary tests a methodology proposed by Munk et al. (2022) for using failed predictions in machine learning as a method to identify ambiguous and rich cases for qualitative analysis. Using a dataset describing actions performed by fictional characters interacting with machine vision technologies in 500 artworks, movies, novels and videogames, I trained a simple machine learning algorithm (using the kNN algorithm in R) to predict whether or not an action was active or passive using only information about the fictional characters. Predictable actions were generally unemotional and unambiguous activities where machine vision technologies were treated as simple tools. Unpredictable actions, that is, actions that the algorithm could not correctly predict, were more ambivalent and emotionally loaded, with more complex power relationships between characters and technologies. The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis. This test goes beyond simply replicating Munk et al.'s results by demonstrating that the method can be applied to a broader humanities domain, and that it does not require complex neural networks but can also work with a simpler machine learning algorithm. Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative. To support this, the R code required to produce the results is included so the test can be replicated. The code can also be reused or adapted to test the method on other datasets.

show abstract

Representations of machine vision technologies in artworks, games and narratives: A dataset

Cited by 10 publications

References 8 publications

Auguring Fake Face Images Using Dual Input Convolution Neural Network

Auguring Fake Face Images Using Dual Input Convolution Neural Network

Machine vision situations: Tracing distributed agency

Algorithmic failure as a humanities methodology: Machine learning's mispredictions identify rich cases for qualitative analysis

Contact Info

Product

Resources

About