The magnetic and structural properties of single crystal Fe1−xCoxV2O4 samples have been investigated by performing specific heat, susceptibility, neutron diffraction, and X-ray diffraction measurements. As the orbital-active Fe 2+ ions with larger ionic size are gradually substituted by the orbital-inactive Co 2+ ions with smaller ionic size, the system approaches the itinerant electron limit with decreasing V-V distance. Then, various factors such as the Jahn-Teller distortion and the spin-orbital coupling of the Fe 2+ ions on the A sites and the orbital ordering and electronic itinerancy of the V 3+ ions on the B sites compete with each other to produce a complex magnetic and structural phase diagram. This phase diagram is compared to those of Fe1−xMnxV2O4 and Mn1−xCoxV2O4 to emphasize several distinct features.