On Evaluating Systems for Generating Expressive Music Performance: the Rencon Experience

Abstract

Research into music generation and into emulating human musical competence has attracted much attention in the field of computer science. In general, the results of academic research should be verified by assessing `objective effectiveness', which is often represented by a `recognition ratio'. Although `objective effectiveness' is also a requirement for research in music generation, it is meaningless unless subjective requisites are also satisfied. However, it is not easy for researchers to execute subjective evaluations within their individual endeavours. To address this difficulty within the research area of computer systems for generating expressive music performances, the Performance Rendering Contest (Rencon) was created. This is an international competition in which entrants present computer systems and the performances generated are graded, and has been held in conjunction with related international conferences. This paper presents an overview of Rencon history, highlighting the evaluative motivation of each contest. In addition, we discuss the possibilities of a new scientific research field in which future Rencons may play a role

Physical modelling and supervised training of a virtual string quartet

“…Future work will provide a GUI to allow users to easily modify such stylistic decisions, allowing them to personalize the audio performances. There is an active research community focused on expressive computer music performances for piano music [16], but in the immediate future our goal is to give humans better/easier control over such decisions in a manner similar to Vocaloid.…”

“…Autonomous performance of music is tested in the Musical Performance Rendering Contest for Computer Systems [16], in which researchers write programs to perform piano music (generally Mozart or Chopin) which are then performed on a disklavier (a computer-controlled acoustic piano). In contrast, our work focuses on string quartets instead of piano.…”

The Role of Individual Difference in Judging Expressiveness of Computer-Assisted Music Performances by Experts

“…This suggestion may reflect the subjective reference to human performances and an unconscious but strong desire for a type of Turing test. Indeed, in various Rencon contests, an evaluation scale of "how human" was proposed [Hiraga et al 2004;Katayose et al 2012;Bresin and Friberg 2013]. Moreover several (20%) participants suggested assessment of how original and surprising are the generated performances.…”

“…The five systems and the order of presentation with two consecutive interpretations generated for each, were R2, R3, and R5 are autonomous systems, which automatically generate expressive performances directly from a score, while R1 and R4 are interactive performance systems [Katayose et al 2012;Kirke and Miranda 2013], where the user can control the low-level parameters of the model during the performance. The performances were played on site by the systems on a computer-controlled Yamaha Disklavier grand piano.…”

“…Over the years, the Rencon contest evolved toward a more structured format. In the final stage, the performances generated during a dedicated workshop are openly evaluated by a large and qualified audience of musicians and scientists under the same conditions in the manner of a competition [Katayose et al 2012].…”