Combining LSTM and Feed Forward Neural Networks for Conditional Rhythm Composition

Makris, Dimos; Καλιακάτσος-Παπακώστας, Μάξιμος; Karydis, Ioannis; Kermanidis, Katia Lida

doi:10.1007/978-3-319-65172-9_48

Cited by 24 publications

(26 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that this system also includes -as an additional voice/track -a condensed representation of the bass line part and some information representing the meter, see more details in Section 6.10.3.1. The authors [123] argue that this extra explicit information ensures that the network architecture is aware of the beat structure at any given point.…”

Section: Drums and Percussionmentioning

confidence: 99%

“…The system proposed by Makris et al [123] is specific in that it is dedicated to the generation of sequences of rhythm. Another specificity is the possibility to condition the generation relative to some particular information, such as a given beat or bass line.…”

Section: #1 Example: Rhythm Symbolic Music Generation Systemmentioning

confidence: 99%

“…Last, the representation includes some additional information representing the metrical structure (the beat structure), also through binary words. See further details in [123].…”

Section: #1 Example: Rhythm Symbolic Music Generation Systemmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning Techniques for Music Generation

Briot¹,

Hadjeres²,

Pachet³

2020

Computational Synthesis and Creative Systems

177

142

View full text Add to dashboard Cite

8 As Richard Feynman coined it: "What I cannot create, I do not understand." 9 Initially codified in 1950 by Alan Turing and named by him the "imitation game" [191], the "Turing test" is a test of the ability for a machine to exhibit intelligent behavior equivalent to (and more precisely, indistinguishable from) the behavior of a human. In his imaginary experimental setting, Turing proposed the test to be a natural language conversation between a human (the evaluator) and a hidden actor (another human or a machine). If the evaluator cannot reliably tell the machine from the human, the machine is said to have passed the test. 10 This is to avoid the bias (synthetic flavor) of a computer rendered generated music. 11 Improvisation is a form of real time composition. 12 With some precaution, as this division is not that strict. 13 The general objective of integrating sub-symbolic and symbolic levels into a complete AI system is among the "Holy Grails" of AI. Chapter 2 Method In our analysis, we consider five main dimensions to characterize different ways of applying deep learning techniques to generate musical content. This typology is aimed at helping the analysis of the various perspectives (and elements) leading to the design of different deep learning-based music generation systems 1. 2.1 Dimensions The five dimensions that we consider are as follows. 2.1.1 Objective The objective 2 consists in: • The musical nature of the content to be generated. Examples are a melody, a polyphony or an accompaniment; and • The destination and use of the content generated. Examples are a musical score to be performed by some human musician(s) or an audio file to be played. 2.1.2 Representation The representation is the nature and format of the information (data) used to train and to generate musical content. Examples are signal, transformed signal (e.g., a spectrum, via a Fourier transform), piano roll, MIDI or text. 2.1.3 Architecture The architecture is the nature of the assemblage of processing units (the artificial neurons) and their connexions.

show abstract

Section: Drums and Percussionmentioning

confidence: 99%

Section: #1 Example: Rhythm Symbolic Music Generation Systemmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Techniques for Music Generation

Briot¹,

Hadjeres²,

Pachet³

2020

Computational Synthesis and Creative Systems

177

142

View full text Add to dashboard Cite

show abstract

“…The model is applied to the data collected from kidney failure patients and predict among three possible endpoints that would occur after kidney transplantation. In a similar study (Makris et al 2017), but in musical research, a deep-learning architecture is developed by combining LSTM and FNN to generate drum sequences. In this architecture, drum sequences, i.e., the dynamic data, collected from three bands, are fed into an LSTM layer while the FNN takes the bass information as static data.…”

Section: Using Recurrent Neural Network To Model Sequential Datamentioning

confidence: 99%

Predicting human design decisions with deep recurrent neural network combining static and dynamic data

et al. 2020

View full text Add to dashboard Cite

Computational modeling of the human sequential design process and successful prediction of future design decisions are fundamental to design knowledge extraction, transfer, and the development of artificial design agents. However, it is often difficult to obtain designer-related attributes (static data) in design practices, and the research based on combining static and dynamic data (design action sequences) in engineering design is still underexplored. This paper presents an approach that combines both static and dynamic data for human design decision prediction using two different methods. The first method directly combines the sequential design actions with static data in a recurrent neural network (RNN) model, while the second method integrates a feed-forward neural network that handles static data separately, yet in parallel with RNN. This study contributes to the field from three aspects: (a) we developed a method of utilizing designers’ cluster information as a surrogate static feature to combine with a design action sequence in order to tackle the challenge of obtaining designer-related attributes; (b) we devised a method that integrates the function–behavior–structure design process model with the one-hot vectorization in RNN to transform design action data to design process stages where the insights into design thinking can be drawn; (c) to the best of our knowledge, it is the first time that two methods of combining static and dynamic data in RNN are compared, which provides new knowledge about the utility of different combination methods in studying sequential design decisions. The approach is demonstrated in two case studies on solar energy system design. The results indicate that with appropriate kernel models, the RNN with both static and dynamic data outperforms traditional models that only rely on design action sequences, thereby better supporting design research where static features, such as human characteristics, often play an important role.

show abstract

“…metric structure. Similar techniques have been used for generating drum rhythms [15], given information about the metric structure and the activity of other instruments in a recording. For a more detailed description of music work on deep learning in music, the reader is referred to [2].…”

Section: Introductionmentioning

confidence: 99%

Interactive Control of Explicit Musical Features in Generative LSTM-based Systems

Καλιακάτσος-Παπακώστας

Γκιόκας

Katsouros

2018

Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion

Self Cite

View full text Add to dashboard Cite

Long Short-Term Memory (LSTM) neural networks have been effectively applied on learning and generating musical sequences, powered by sophisticated musical representations and integrations into other deep learning models. Deep neural networks, alongside LSTM-based systems, learn implicitly: given a sufficiently large amount of data, they transform information into high-level features that, however, do not relate with the high-level features perceived by humans. For instance, such models are able to compose music in the style of the Bach chorales, but they are not able to compose a less rhythmically dense version of them, or a Bach choral that begins with low and ends with high pitches-even more so in an interactive way in real-time. This paper presents an approach to creating such systems. A very basic LSTM-based architecture is developed that can compose music that corresponds to user-provided values of rhythm density and pitch height/register. A small initial dataset is augmented to incorporate more intense variations of these two features and the system learns and generates music that not only reflects the style, but also (and most importantly) reflects the features that are explicitly given as input at each specific time. This system-and future versions that will incorporate more advanced architectures and representation-is suitable for generating music the features of which are defined in real-time and/or interactively. CCS CONCEPTS • Computing methodologies → Neural networks; • Humancentered computing → HCI theory, concepts and models;

show abstract

Combining LSTM and Feed Forward Neural Networks for Conditional Rhythm Composition

Cited by 24 publications

References 8 publications

Deep Learning Techniques for Music Generation

Deep Learning Techniques for Music Generation

Predicting human design decisions with deep recurrent neural network combining static and dynamic data

Interactive Control of Explicit Musical Features in Generative LSTM-based Systems

Contact Info

Product

Resources

About