Sequential activity has been observed in multiple neuronal circuits across species, neural structures, and behaviors. It has been hypothesized that sequences could arise from unsupervised learning processes. However, it is still unclear whether biologically plausible synaptic plasticity rules can organize neuronal activity to form sequences whose statistics match experimental observations. Here we investigate temporally asymmetric Hebbian rules in sparsely connected recurrent rate networks, and develop a theory of the transient sequential activity observed after learning. These rules transform a sequence of random input patterns into synaptic weight updates. After learning, recalled sequential activity is reflected in the transient correlation of network activity with each of the stored input patterns. Using mean-field theory, we derive a low-dimensional description of the network dynamics and compute the storage capacity of these networks. Multiple temporal characteristics of the recalled sequential activity are consistent with experimental observations. We find that the degree of sparseness of the recalled sequences can be controlled by non-linearities in the learning rule. Furthermore, sequences maintain robust decoding, but display highly labile dynamics, when synaptic connectivity is continuously modified due to noise or storage of other patterns, similar to recent observations in hippocampus and parietal cortex. Finally, we demonstrate that our results also hold in recurrent networks of spiking neurons with separate excitatory and inhibitory populations.S equential activity has been reported across a wide range of neural systems and behavioral contexts, where it plays a critical role in temporal information encoding. Sequences can encode choice-selective information (1), the timing of motor actions (2), planned or recalled trajectories through the environment (3), and elapsed time (4-6). This diversity in function is also reflected at the level of neuronal activity. Sequences occur at varying timescales, from those lasting tens of milliseconds during hippocampal sharp-wave ripples, to those spanning several seconds in the striatum (7,8). Sequential activity also varies in temporal sparsity. In Nucleus HVC of zebra finch, highly-precise sequential activity is present during song production, where many neurons fire only a single short burst during a syllable (9). In primate motor cortex, single neurons are typically active throughout a whole reach movement, but with heterogeneous and rich dynamics (10).Numerous models have explored how networks with specific synaptic connectivity can generate sequential activity (11)(12)(13)(14)(15)(16)(17)(18)(19)(20). These models span a wide range of single neuron models, from binary to spiking, and a wide range of synaptic connectivities. A large class of models employs temporally asymmetric Hebbian (TAH) learning rules to generate a synaptic connectivity necessary for sequence retrieval. In these models, a sequence of random input patterns are presented to the network, a...