We report structural, magnetic, and dielectric properties of oxygen deficient hexagonal BaFeO3−δ. A large dielectric permittivity comparable to that of other semiconducting oxides is observed in BaFeO3−δ. Magnetization measurements indicate magnetic inhomogeneity and the system shows a paramagnetic to antiferromagnetic transition at ∼160 K. Remarkably, the temperature, at which paramagnetic to antiferromagnetic transition occurs, around this temperature, a huge drop in the dissipation factor takes place and resistivity shoots up; this indicates the possible correlation among magnetic and dielectric properties. First principle simulations reveal that some of these behaviors may be explained in terms of many body electron correlation effect in the presence of oxygen vacancy present in BaFeO3−δ indicating its importance in both fundamental science as well as in applications.