The development of highly efficient non‐precious metal electrocatalysts for the oxygen evolution reaction (OER) in low‐grade or saline water is currently of great importance for the large‐scale production of hydrogen. In this study, by using an electrochemical activation pretreatment, metal oxy(hydroxide) nanosheet structures derived from self‐supported nickel–iron phosphide and nitride nanoarrays grown on Ni foam are successfully fabricated for OER catalysis in saline water. It is demonstrated that the different NiOOH and NiOOH@FeOOH (NiOOH grown on FeOOH) structures are generated from nickel–iron nitride and phosphide, respectively, after electrochemical activation. In particular, the NiOOH@FeOOH heteroarchitecture shows outstanding electrocatalytic performance with an ultralow overpotential of 292 mV to drive the current density of 500 mA cm−2. An unconventional dual‐sites mechanism (UDSM) is proposed to address the OER process on NiOOH@FeOOH and show that the FeOOH underlayer plays a critical role regarding the enhanced OER activity of NiOOH. The new possible UDSM involving two reaction sites presents a different understanding of the OER process on multi‐OH layer complexes, which is expected to guide the design of heteroarchitecture electrocatalysts.