A large change in yield strength and ductility takes place in metals with a recrystallized average grain size in the near-micrometer grain size regime. This has been utilized to explore the influence of grain size heterogeneity on plastic deformation for recrystallized grains in aluminum prepared by spark plasma sintering, where tailoring of the grain size distribution and spatial arrangement can be achieved by selecting powder sizes and mixing conditions prior to sintering. The spatial variation of plastic deformation has been explored by digital image correlation (DIC), either at the sample scale or at the grain scale. For the latter colloidal SiO2 markers also allow repeated collection of electron backscatter diffraction data over the same area during in-situ deformation, proving correlated information on local displacement gradients and crystal lattice rotations. For samples prepared from well-mixed powders only small deviations from a rule-of-mixtures in the yield strength are observed. A mesoscale pattern of strain distribution is observed from DIC investigations where this is related to, but not fully consistent with, the spatial arrangement of fine and coarse grains. In the case of macroscopically layered arrangements of coarse and fine grains, deformation is dominated by development of local strain concentrations, which take place preferentially in the fine grain regions.