The interplay between abiotic (resource supply, temperature) and biotic (herbivore grazing) factors determines growth and loss processes in phytoplankton through resource competition and trophic interactions, which are mediated by morphological traits like size. Here, we study the relative importance of different grazers, water physics and chemistry on the daily net accumulation rates (AR) of six phytoplankton size classes, obtained by grouping individual organisms from natural communities based on their realised dimensions. Using a Random Forest modelling approach and four years of daily underwater imaging data, we find that the AR of small phytoplankton is mostly predicted by water temperature and herbivore grazing whereas, with increasing size, resources become more important in predicting ARs. Our results show that microzooplankton is an important predictor of AR in all phytoplankton regardless of their size, but mesozooplankton is more important for small phytoplankton forms. Our results support previous expectations and reveal surprising new patterns.Significance StatementUnderstanding the relative importance of biotic and abiotic controls of biodiversity change, in terms of taxa and traits, is critical in times of global and local anthropogenic impact on ecosystem processes. The relative importance of different mechanisms driving phytoplankton community dynamics is still debated, and might vary depending on the environmental conditions. We harnessed four years of data from an underwater imaging microscope, and ancillary monitoring sensors, to study the biotic and abiotic controls of phytoplankton size classes using machine-learning. We find that, contrary to prior expectations, large (blooming) phytoplankton are favoured by high temperature and low levels of microzooplankton. These results help us understand phytoplankton dynamics and improve the modelling of plankton food-webs, with implications for predicting plankton community structure in changing aquatic ecosystems.