Caption-Supervised Face Recognition: Training a State-of-the-Art Face Model Without Manual Annotation

Huang, Qingqiu; Yang, Lei; Huang, Huaiyi; Wu, Tong; Lin, Dahua

doi:10.1007/978-3-030-58520-4_9

Cited by 26 publications

(11 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Like LSMDC, MovieNet [21] and Condensed Movies [2] are big projects that contain several tasks, data, and annotations related to movie understanding. MovieNet includes works related to person reidentification [23,54,19,20], Movie Scene Temporal Segmentation [35], and trailer and synopses analysis [55,22]. All these works have shown that movies have rich information about human actions, including their specific challenges.…”

Section: Related Workmentioning

confidence: 99%

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Pardo¹,

Heilbron²,

Alcázar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding movies and their structural patterns is a crucial task to decode the craft of video editing. While previous works have developed tools for general analysis such as detecting characters or recognizing cinematography properties at the shot level, less effort has been devoted to understanding the most basic video edit, the Cut. This paper introduces the cut type recognition task, which requires modeling of multi-modal information. To ignite research in the new task, we construct a large-scale dataset called MovieCuts, which contains more than 170K video clips labeled among ten cut types. We benchmark a series of audio-visual approaches, including some that deal with the problem's multi-modal and multi-label nature. Our best model achieves 45.7% mAP, which suggests that the task is challenging and that attaining highly accurate cut type recognition is an open research problem.

show abstract

Section: Related Workmentioning

confidence: 99%

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Pardo¹,

Heilbron²,

Alcázar³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Similarly, Huang et al [33] created a dataset that comprises complete movies along with their trailers, pictures, synopses, transcripts, subtitles, and general metadata. Based on these datasets, the research community has developed solutions for new movie-related tasks, such as: shot-type classification [56], movie-scene segmentation [57], character re-identification and recognition [11,32,35,70], trailer and synopsis analysis [34,71], and visual-question answering in movies [21,36]. Unlike previous works in long-term video analysis, our work centers around the video creation process.…”

Section: Related Workmentioning

confidence: 99%

Learning to Cut by Watching Movies

Pardo¹,

Heilbron²,

Alcázar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video content creation keeps growing at an incredible pace; yet, creating engaging stories remains challenging and requires non-trivial video editing expertise. Many video editing components are astonishingly hard to automate primarily due to the lack of raw video materials. This paper focuses on a new task for computational video editing, namely the task of raking cut plausibility. Our key idea is to leverage content that has already been edited to learn fine-grained audiovisual patterns that trigger cuts. To do this, we first collected a data source of more than 10K videos, from which we extract more than 255K cuts. We devise a model that learns to discriminate between real and artificial cuts via contrastive learning. We set up a new task and a set of baselines to benchmark video cut generation. We observe that our proposed model outperforms the baselines by large margins. To demonstrate our model in real-world applications, we conduct human studies in a collection of unedited videos. The results show that our model does a better job at cutting than random and alternative baselines.

show abstract

“…The total number of meta information in MovieNet is 375K. Please note that each kind of data itself, even without the movie, can support some research topics [37]. So we try to get each kind of data as much as we can.…”

Section: Data In Movienetmentioning

confidence: 99%

MovieNet: A Holistic Dataset for Movie Understanding

Huang

Xiong

Rao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years have seen remarkable advances in visual understanding. However, how to understand a story-based long video with artistic styles, e.g. movie, remains challenging. In this paper, we introduce MovieNet -a holistic dataset for movie understanding. MovieNet contains 1, 100 movies with a large amount of multi-modal data, e.g. trailers, photos, plot descriptions, etc.. Besides, different aspects of manual annotations are provided in MovieNet, including 1.1M characters with bounding boxes and identities, 42K scene boundaries, 2.5K aligned description sentences, 65K tags of place and action, and 92K tags of cinematic style. To the best of our knowledge, MovieNet is the largest dataset with richest annotations for comprehensive movie understanding. Based on MovieNet, we set up several benchmarks for movie understanding from different angles. Extensive experiments are executed on these benchmarks to show the immeasurable value of MovieNet and the gap of current approaches towards comprehensive movie understanding. We believe that such a holistic dataset would promote the researches on storybased long video understanding and beyond. MovieNet will be published in compliance with regulations at https://movienet.github.io.

show abstract

Caption-Supervised Face Recognition: Training a State-of-the-Art Face Model Without Manual Annotation

Cited by 26 publications

References 40 publications

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Learning to Cut by Watching Movies

MovieNet: A Holistic Dataset for Movie Understanding

Contact Info

Product

Resources

About