We calculate the probe spectrum for multilevel atoms in a damped, weakly driven cavity supporting two degenerate orthogonally polarized modes. One mode is weakly driven by a linearly polarized external field. The atoms, initially prepared in a m F = 0 hyperfine ground state, couple to the driven mode by making a m F = 0 transition and to the orthogonally polarized mode by making m F = ±1 transitions to other hyperfine sublevels. We compare probe spectra, the intensity of the driven mode, the undriven mode, and the atomic excitation probability as a function of the detuning of the driving laser from the cavity resonance for three-and four-level atomic models. In both cases the spectrum of the driven mode is a vacuum Rabi doublet familiar from the driven, damped Jaynes-Cummings model. The undriven mode spectrum is a triplet with the maximum on resonance for the three-level model while the four-level model has a four-peaked spectrum. We discuss the role of strong coupling to the undriven mode in accounting for the qualitative difference between the models.