Using scanning tunneling microscopy, we mapped the distribution of the local density of states in a single crystal superconductor heterostructure with an array of submicron normal metal islands. We observe the coexistence of strongly interacting multiquanta vortex lattice with interstitial Abrikosov vortices. The newly formed composite magnetic flux structure undergoes a series of phase transitions between different topological configuration states. The vortex configuration states are strongly dependent on the number of flux quanta and the nanoscale confinement architecture of the mesoscopic superconductor. Here, we present images of vortex phase transitions due to confinement effects when the number of magnetic flux quanta in the system changes. The vortex dynamics in these systems could serve as a model for behavior of confined many-body systems when the number of particles changes.

Using scanning tunneling microscopy and Ginzburg-Landau simulations, we explore vortex configurations in magnetically coupled NbSe2-Permalloy superconductor-ferromagnet bilayer. The Permalloy film with stripe domain structure induces periodic local magnetic induction in the superconductor, creating a series of pinning-antipinning channels for externally added magnetic flux quanta. Such laterally confined Abrikosov vortices form quasi-1D arrays (chains). The transitions between multichain states occur through propagation of kinks at the intermediate fields. At high fields we show that the system becomes non-linear due to a change in both the number of vortices and the confining potential. The longitudinal instabilities of the resulting vortex structures lead to vortices 'levitating' in the anti-pinning channels.