Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is normally controlled by effective host immunity including innate, humoral and cellular responses. However, the trajectories and correlates of acquired immunity, and the capacity of memory responses months after infection to neutralise variants of concern - which has important public health implications - is not fully understood. To address this, we studied a cohort of 78 UK healthcare workers who presented in April to June 2020 with symptomatic PCR-confirmed infection or who tested positive during an asymptomatic screening programme and tracked virus-specific B and T cell responses longitudinally at 5-6 time points each over 6 months, prior to vaccination. We observed a highly variable range of responses, some of which - T cell interferon-gamma (IFN-γ) ELISpot, N-specific antibody waned over time across the cohort, while others (spike-specific antibody, B cell memory ELISpot) were stable. In such cohorts, antiviral antibody has been linked to protection against re-infection. We used integrative analysis and a machine-learning approach (SIMON - Sequential Iterative Modeling Over Night) to explore this heterogeneity and to identify predictors of sustained immune responses. Hierarchical clustering defined a group of high and low antibody responders, which showed stability over time regardless of clinical presentation. These antibody responses correlated with IFN-γ ELISpot measures of T cell immunity and represent a subgroup of patients with a robust trajectory for longer term immunity. Importantly, this immune-phenotype associates with higher levels of neutralising antibodies not only against the infecting (Victoria) strain but also against variants B.1.1.7 (alpha) and B.1.351 (beta). Overall memory responses to SARS-CoV-2 show distinct trajectories following early priming, that may define subsequent protection against infection and severe disease from novel variants.