The applications of magnetic composites in electromagnetic shielding, resonance imaging, and biomedical drug research are growing rapidly. Therefore, there is a huge demand to characterize these materials in order to obtain their dielectric and magnetic properties in the microwave frequency range. In this paper, a novel rectangular cavity method is proposed to obtain the complex permeability and permittivity of the test sample placed in the E-plane of the rectangular cavity. The proposed method eliminates the need of physical relocation of a specimen in the H-plane, which is usually required in the case of presently available methods to characterize the magnetic samples. The closed-form formula for this unified approach is developed from the first principle, and is later modified to consider some practical consideration. The proposed method is first numerically tested and verified for various standard samples, where the data are taken from the literature. Thereafter, the X-band rectangular cavity is designed and fabricated to measure a number of available samples. The significance of the proposed approach can be appreciated from the fact that even for finite-size samples, the permeability and permittivity can be extracted with the typical errors of 5% and 2%, respectively.