Dielectric relaxation is universal in characterizing polar liquids and solids, insulators, and semiconductors, and the theoretical models are well developed. However, in high magnetic fields, previously unknown aspects of dielectric relaxation can be revealed and exploited. Here, we report low temperature dielectric relaxation measurements in lightly doped silicon in high dc magnetic fields B both parallel and perpendicular to the applied ac electric field E. For B//E, we observe a temperature and magnetic field dependent dielectric dispersion ε(ω) characteristic of conventional Debye relaxation where the free carrier concentration is dependent on thermal dopant ionization, magnetic freeze-out, and/or magnetic localization effects. However, for B⊥E, anomalous dispersion emerges in ε(ω) with increasing magnetic field. It is shown that the Debye formalism can be simply extended by adding the Lorentz force to describe the general response of a dielectric in crossed magnetic and electric fields. Moreover, we predict and observe a new transverse dielectric response E H ⊥ B ⊥ E not previously described in magnetodielectric measurements. The new formalism allows the determination of the mobility and the ability to discriminate between magnetic localization/freeze out and Lorentz force effects in the magneto-dielectric response.