Understanding how specific secretory cargoes are targeted to distinct domains of the plasma membrane in epithelial cells requires analysing the trafficking of post-Golgi vesicles to their sites of secretion. Here, we introduce an interactive vesicle tracking framework, the multiscale particle tracker and viewer, that exploits developments in computer vision and deep learning to determine vesicle trajectories in noisy cellular environments without the need for extensive training data. We analysed the movements of secreted cargoes in the Drosophila follicular epithelium ex vivo with high temporal and spatial resolution and found that MSP-tracker outperformed other tracking software. Using the RUSH (retention using selective hooks) system to synchronously release cargoes from the endoplasmic reticulum, we followed the movements the microvillar protein, Cadherin 99C, to the plasma membrane. A dominant slow dynein mutant reduces the speed of apical Cad99C movements, demonstrating that dynein transports Cad99C vesicles to the apical cortex. Furthermore, both the dynein mutant and microtubule depolymerisation cause lateral Cad99C secretion. Thus, microtubule organisation plays a central role in targeting secretion, suggesting that Drosophila does not have distinct apical versus basolateral vesicle fusion machinery. Imaging the extracellular matrix protein (ECM) proteins, Nidogen and Collagen IV revealed that Nidogen vesicles undergo planar-polarised transport to the leading edge of follicle cells as they migrate over the ECM, whereas most Collagen is secreted at trailing edges. The follicle cells therefore secrete different ECM components at opposite sides of the cell, revealing that the secretory pathway is more spatially organised than previously thought.