Automatic background music generation based on actors' mood and motions

Abstract

Since adding background music and sound effects even to short animations is not simple, an automatic music generation system would help improve the total quality of computer generated animations. This paper describes a prototype system which automatically generates background music and sound effects for existing animations. The inputs to the system are music parameters (mood types and musical motifs) and motion parameters for individual scenes of an animation. Music is generated for each scene. The key for a scene is determined by considering the mood type and its degree, and the key of the previous scene. The melody for a scene is generated from the given motifs and the chord progression for the scene which is determined according to appropriate rules. The harmony accompaniment for a scene is selected based on the mood type. The rhythm accompaniment for a scene is selected based on the mood type and tempo. The sound effects for motions are determined according to the characteristics and intensity of the motions. Both the background music and sound effects are generated so that the transitions between scenes are smooth.* A phrase is an intermediate unit of melody. In the Classical period, it typically has four measures. Since it usually ends with some form of cadence, it is also the unit of a harmonic progression.

Auto-generation of professional background music for home-made videos

“…The mechanisms of automatic music composition can be classified into stochastic techniques and deterministic techniques. A pioneering deterministic method on automatic background music generation is proposed by Nakamura et aL [7]. They described a prototype system which generates background music according to the known mood and motion parameters.…”

Automatic Synchronization of Background Music and Motion in Computer Animation

“…Nakamura et. al [NKH*94] generated background music and sound effects based on the mood and motion of animated characters. Several studies on the automatic generation of sound effects from physical models have achieved realistic association between sound and motion [DYN03,DP96,DKP01,OCE01].…”

“…Windows media player visualization can be put into this category, since it uses an audio signal to generate an abstract animation. In other studies [NKH*94,DKP01,HFGL95] the opposite approach has been taken, and music has been synthesized to match the completed animation. An alternative idea, modifying existing music to match the scene, has received little attention.…”

An integrated approach to motion and sound

“…Our aim was to generate soundtracks that are both interesting, and intrinsically based on the motion parameter values. Such a technique has been previously explored 19 , however, in this previous work, the very limited correlation between the motion and sound was specified manually.…”