Poly-silicon passivating contacts, consisting of a stack of tunnel-oxide and doped poly-silicon layers, can simultaneously provide excellent surface passivation and low contact resistivity for silicon solar cells. Nevertheless, the microscopic interfacial characteristics of such contacts are not yet fully understood. In this work, by investigating the surface passivation evolution of poly-silicon passivating contacts under increasing annealing temperatures, we unveil these characteristics. Before annealing, we find that the Si and O atoms within the tunneloxide layer are mostly unsaturated, whereas the O atoms introduce acceptor-like defects. These defects cause Fermi-level pinning and high carrier recombination. During annealing, we identify two distinct chemical passivation regimes driven by surface hydrogenation and oxidation. We attribute the excellent chemical passivation activated by high-temperature annealing (850°C) mainly to the tunnel oxide reconstruction, which effectively reduces the acceptor-like state density. During the oxide reconstruction, we also find that sub-nanometer pits (rather than pinholes) are formed in the oxide. A combination of experimental and theoretical investigations demonstrates these sub-nanometer pits provide excellent surface passivation and efficient tunneling for majority carriers.