Revolutionising food advertising monitoring: a machine learning-based method for automated classification of food videos

Abstract

Objective: Food advertising is an important determinant of unhealthy eating. However, analyzing a large number of advertisements (ads) to distinguish between food and non-food content is a challenging task. This study aims to develop a machine learning-based method to automatically identify and classify food and non-food ad videos. Design: Methodological study to develop an algorithm model that prioritizes both accuracy and efficiency in monitoring and classifying advertising videos. Setting: From a collection of Brazilian television (TV) ads data, we created a database and split it into three sub-databases (i.e., training, validation, and test) by extracting frames from ads. Subsequently, the training database was classified using the EfficientNet neural network. The best models and data-balancing strategies were investigated using the validation database. Finally, the test database was used to apply the best model and strategy, and results were verified with field experts. Participants: The study used 2,124 recorded Brazilian TV programming hours from 2018-2020. It included 703 food ads and over 20,000 non-food ads, following the protocol developed by the INFORMAS network for monitoring food marketing on TV. Results: The results showed that the EfficientNet neural network associated with the balanced batches strategy achieved an overall accuracy of 90.5% on the test database, which represents a reduction of 99.9% of the time spent on identifying and classifying ads. Conclusions: The method studied represents a promising approach for differentiating food and non-food related video within monitoring food marketing, which has significant practical implications for researchers, public health policy-makers, and regulatory bodies.