Theme Transformer: Symbolic Music Generation With Theme-Conditioned Transformer

Shih, Yi-Jen; Wu, Shih-Lun; Zalkow, Frank; Müller, Meinard; Yang, Yi-Hsuan

doi:10.1109/tmm.2022.3161851

Cited by 42 publications

(24 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…MELONS [37] entails a multi-step generation method with Transformer-based models and graph representation for music structure generation and structure-conditional melody generation. Theme Transformer [30] achieves theme-based conditioning by producing it multiple times in the generation result so that the output music follows the thematic material. These models were constructed based on the structure of music from multiple levels (e.g., the motif level and phrase level).…”

Section: Related Workmentioning

confidence: 99%

“…There is no significant difference between the music generated by the proposed model and the original humancomposed music (p=0.351), while there is a significant difference when comparing results generated by other models and original human-composed music (p<0.05), indicating a better music quality of the proposed model. 3.47 1.15 t=-6.01, p<0.001 CP-NC [13] 3.64 0.96 t=-4.74, p<0.001 PopMNet [33] 3.77 0.95 t=-4.41, p<0.001 Theme [30] 3.46 0.91 t=-7.37, p<0.001 MELONS [37] 3.89…”

Section: Subjective Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

The Beauty of Repetition in Machine Composition Scenarios

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Repetition, a basic form of artistic creation, appears in most musical works and delivers enthralling aesthetic experiences. However, repetition remains underexplored in terms of automatic music composition. As an initial effort in repetition modelling, this paper focuses on generating motif-level repetitions via domain knowledge-based and example-based learning techniques. A novel repetition transformer (R-Transformer) that combines a Transformer encoder and a repetition-aware learner is trained on a new repetition dataset with 584,329 samples from different categories of motif repetition. The Transformer encoder learns the representation among music notes from the repetition dataset; the novel repetition-aware learner exploits repetitions' unique characteristics based on music theory. Experiments show that, with any given motif, R-Transformer can generate a large number of variable and beautiful repetitions. With ingenious fusion of these high-quality pieces, the musicality and appeal of machine-composed music have been greatly improved. CCS CONCEPTS• Applied computing → Sound and music computing.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Subjective Evaluationmentioning

confidence: 99%

The Beauty of Repetition in Machine Composition Scenarios

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Originated from the machine translation tasks, Transformer [80] has received great attention from broader research communities beyond natural language processing [24,30,31,74,85]. ViT [16] is marked as the first successful attempt on implementing transformer model into computer vision tasks, which simply splits the whole image into patches and feeds them into Transformer encoders with multihead attention mechanisms.…”

Section: Vision Transformermentioning

confidence: 99%

Sparse Fusion Mixture-of-Experts are Domain Generalizable Learners

Li¹,

Shen²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Domain generalization (DG) aims at learning generalizable models under distribution shifts to avoid redundantly overfitting massive training data. Previous works with complex loss design and gradient constraint have not yet led to empirical success on large-scale benchmarks. In this work, we reveal the mixture-of-experts (MoE) model's generalizability on DG by leveraging to distributively handle multiple aspects of the predictive features across domains. To this end, we propose Sparse Fusion Mixture-of-Experts (SF-MoE), which incorporates sparsity and fusion mechanisms into the MoE framework to keep the model both sparse and predictive. SF-MoE has two dedicated modules: 1) sparse block and 2) fusion block, which disentangle and aggregate the diverse learned signals of an object, respectively. Extensive experiments demonstrate that SF-MoE is a domain-generalizable learner on large-scale benchmarks. It outperforms state-of-the-art counterparts by more than 2% across 5 large-scale DG datasets (e.g., DomainNet), with the same or even lower computational costs. We further reveal the internal mechanism of SF-MoE from distributed representation perspective (e.g., visual attributes). We hope this framework could facilitate future research to push generalizable object recognition to the real world. Our code and models will be released at SF-MoE.Preprint. Under review.

show abstract

“…Desde una visión tecnológica del dominio de la IA, puede decirse que los sistemas inteligentes de generación musical simbólica han evolucionado de manera rica y diversa a la par con los principales métodos de análisis informático en los diversos períodos de este dominio. Inicialmente, dichos sistemas han estado fundamentados en métodos de IA basada en grafos y probabilidades (Hiller y Baker, 1964;Ames, 1989) que aún se mantienen vivos (Catak et al, 2021;Glines, 2022) y que se expanden y fusionan hoy en día con técnicas contemporáneas de redes neuronales (Briot et al, 2020) como LSTM (Eck y Schmidhuber, 2002;Privato et al, 2022) o transformers (Shih et al, 2022).…”

Section: Introductionunclassified

NEA (New Electronic Assistant), un sistema para la co-creatividad computacional musical

Marín

2022

kepes

View full text Add to dashboard Cite

Este artículo presenta un sistema para la creatividad computacional colaborativa artística aplicada a la creación musical, llamado New Electronic Assistant (NEA). NEA es un sistema que puede aprender de estilos musicales en formato simbólico, generar piezas siguiendo los estilos aprendidos y transformar sus resultados a través de la interacción en tiempo real. A lo largo del texto, NEA es introducido y enmarcado dentro de los conceptos de la creatividadcolaborativa computacional. Para analizar el desempeño de NEA en entornos reales de co-creación, se realiza un proceso de generación sistemática de piezas musicales, de las que se extraen fragmentos que son usados en un experimento de validación con humanos. Los resultados del experimento sugieren altos niveles de eficiencia en el proceso de co-creación, relacionados con el tiempo de producción de nuevas piezas, la sorpresa expresada porlos participantes del experimento y el valor percibido de las piezas generadas por NEA. El experimento concluye con una sección donde se comentan detalles de las piezas percibidas con mayor valor por los participantes del experimento. Al final del texto se hace una reflexión sobre el rol de los sistemas generativos, que requieren interacción humana, en procesos reales de creación colaborativa.

show abstract

Theme Transformer: Symbolic Music Generation With Theme-Conditioned Transformer

Cited by 42 publications

References 39 publications

The Beauty of Repetition in Machine Composition Scenarios

The Beauty of Repetition in Machine Composition Scenarios

Sparse Fusion Mixture-of-Experts are Domain Generalizable Learners

NEA (New Electronic Assistant), un sistema para la co-creatividad computacional musical

Contact Info

Product

Resources

About