Recent observations of macroscopic quantum condensation using electron-hole (e-h) bilayers have activated the research of its application to electronics. However, to the best of our knowledge, no attempts have been made to observe the condensation in silicon, the major material in electronics, due to the lack of technology to form closely-packed and uniform bilayers. Here, we propose a method to meet such requirements. Our method uses the transient response of carriers to a rapid gate-voltage change, permitting the self-organized bilayer formation at the metal-oxide-semiconductor interface with an e-h distance as small as the exciton Bohr radius. Recombination lifetime measurements show that the fast process is followed by a slow process, strongly suggesting that the e-h system changes its configuration depending on carrier density. This method could thus enable controlling the phase of the e-h system, paving the way for condensation and, ultimately, for low-power cryogenic silicon metal-oxide-semiconductor devices.