Scanning tunneling microscopy is employed to investigate the recombinative desorption of H2 from hydrogenated Si(100) surfaces consisting of dihydride (SiH2) and monohydride (SiH) surface species organized in (1 x 1), (3 x 1), and (2 x 1) configurations. The results show that desorption from dihydrides involves a pair of neighboring dihydrides linked along the tetrahedral bond direction. Dihydrides in (3 x 1) domains are separated in the same direction by monohydrides, and desorption from a pair is geometrically impossible. The same desorption mechanism nevertheless applies via first a position switching of dihydrides with neighboring monohydrides.