Automated Music Video Generation Using Multi-level Feature-based Segmentation

Joon, Yoon Sung; Lee, In-Kwon; Byun, Siwoo

doi:10.1007/978-0-387-89024-1_17

Cited by 7 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wang et al also focused on the structure of a music clip and proposed to generate a music sports video by selecting the best video clip for each semantic music segment such as Intro, Verse, and Chorus considering their semantic content and tempo using certain user-defined rules [14]. Yoon et al proposed to conserve the intent of the video-maker by segmenting video and music clips into segments of near-uniform flow and selecting the best video segment for each music segment based on their novelty, velocity and brightness and by using timewarping [15]. While these work all tried to synthesize video clips to a music clip, Mulhem et al have proposed a method for synthesizing several music clips to a sequence of video clips by selecting the best music clip for a video clip based on heuristically determined rules about the aesthetic compatibility between video and music clips, more concretely, how the dynamic, motion, and pitch of video and music clips coincide with each other [16].…”

Section: Related Workmentioning

confidence: 99%

Example-based video remixing support system

Nitta

Babaguchi

2011

Proceedings of the 19th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Video remixes are generally created by sequentially arranging selected video clips and mixing them with other media streams such as audio clips and transition effects. Especially, mixing music clips often effectively improves the expressive quality of video clips. This paper proposes an example-based system for supporting average users in the 3 steps in video remixing: I)video shot sequence creation, II)music clip selection, and III)audio volume adjustment. The proposed system creates a template for a video remix and gives suggestions to users on an interface such as which video clips should be selected to create a video shot sequence and which music clips should be mixed to the created video shot sequence based on professionally created video remix examples. Then, the audio volume of each video shot is automatically adjusted based on its audio content so that the sounds in the video shots and the music clips would not interfere with each other. Experiments have verified that our system was able to create a video shot sequence whose quality was improved equally as the professionally created one by mixing the selected music clips, doubling the subjective scores from 1.7 to 3.7 on a scale of 1-5. Automatic audio volume adjustment improved the subjective scores by approximately 0.5 points on average. Further, the suggestions provided on the interface was evaluated useful by 5 subjects when creating a video remix by selecting 32 video clips and 4 music clips from 265 video clips and 180 music clips.

show abstract

Section: Related Workmentioning

confidence: 99%

Example-based video remixing support system

Nitta

Babaguchi

2011

Proceedings of the 19th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…They usually extract suitable features from the video to be matched with features from the audio. In [2] and [3] an offline alignment is performed, which is not applicable to our problem due to the non-linear computational costs and the need to have complete sequences for the alignment.…”

Section: Related Workmentioning

confidence: 99%

“…The user can therefore, instead of just listening to his favourite music, watch the corresponding music video at the same time. Indeed, many services nowadays allow for music video streaming, for example Youtube 1 3 ; or music streaming, such as Spotify 4 , Pandora 5 or GrooveShark 6 . Due to bandwidth limitations the audio in the streamed videos is usually of lower quality than when streaming audio alone, and neither is comparable to locally stored high quality audio files or music on audio CDs.…”

Section: Introductionmentioning

confidence: 99%

Real-time synchronisation of multimedia streams in a mobile device

Macrae

Neumann

Anguera

et al. 2011

2011 IEEE International Conference on Multimedia and Expo

View full text Add to dashboard Cite

With the constant improvements in the technical capabilities and bandwidth available to mobile phones, mobile audio and video streaming services are booming. This allows music enthusiasts to watch their favourite music videos instead of only listening to the audio tracks, thus augmenting their listening experience to be multimodal. Usually the highly compressed audio tracks in these videos result in poorer quality music, compared to music the user might already have locally on their phone or playing through another sound source. In this paper we present MuViSync Mobile, a mobile phone application that synchronises real-time high quality music, either stored locally or input through the microphone, with the corresponding streaming music video. We extend previous work on music to music video synchronisation by proposing an alternative algorithm for higher efficiency with similar alignment accuracy, which we tested with music video examples including simulated noise. This algorithm correctly aligns 90% of the audio frames to within 100 ms of the known alignment. We also describe its implementation on an iPhone.

show abstract

“…However, there is no bias for a linear progression through both recordings, which is desirable in this case. The work by Yoon, Lee and Byun [4] is an example of aligning music with arbitrary videos by comparing multi-level features.…”

Section: Audio To Video Matchingmentioning

confidence: 99%

“…In addition, users can now pay monthly subscription fees to stream music and videos over their internet connection using online services such as Spotify 3 , Rhapsody 4 or Last.fm 5 .…”

Section: Introductionmentioning

confidence: 99%

MuViSync: Realtime music video alignment

Macrae

Anguera

Oliver

2010

2010 IEEE International Conference on Multimedia and Expo

View full text Add to dashboard Cite

In recent years, the popularity of compressed music files and online music downloads has increased dramatically. Today's users own large digital collections of high quality music on their computers and portable devices to be played in their homes or on the go. In addition, music videos are being offered online both for free and through monthly subscriptions, opening up the opportunity to turn the music listening activity into a multimedia experience. The work presented in this paper addresses the challenge of music audio and music video synchronisation. In particular, we have developed a prototype, named MuViSync, to automatically synchronise music videos to the songs that users are listening to in real-time. At the core of the MuViSync prototype are novel audio synchronization algorithms to tackle the differences in tempo, pitch, sampling rates, structure, and introductions and endings that are common in the various digital recordings of the same song in modern music. The music and the music video are initially aligned and then kept in sync within the limits of human perception. In experiments with 320 matching pairs of audio files and music videos, the proposed algorithms successfully synchronise music and video to within 100 milliseconds of each other in over 90% of the cases.

show abstract

Automated Music Video Generation Using Multi-level Feature-based Segmentation

Cited by 7 publications

References 7 publications

Example-based video remixing support system

Example-based video remixing support system

Real-time synchronisation of multimedia streams in a mobile device

MuViSync: Realtime music video alignment

Contact Info

Product

Resources

About