We study electronic excitations of a degenerate gas of atoms trapped in pairs in an optical lattice. Local dipole-dipole interactions produce a long lived antisymmetric and a short lived symmetric superposition of individual atomic excitations as the lowest internal on-site excitations. Due to the much larger dipole moment the symmetric states couple efficiently to neighbouring lattice sites and can be well represented by Frenkel excitons, while the antisymmetric dark states stay localized. Within a cavity only symmetric states couple to cavity photons inducing long range interactions to form polaritons. We calculate their dispersion curves as well as cavity transmission and reflection spectra to observe them. For a lattice with aspherical sites bright and dark states get mixed and their relative excitation energies depend on photon polarizations. The system should allow to study new types of solid state phenomena in atom filled optical lattices.PACS numbers: 71.36.+c, 71.35.Lk A dilute gas of bosonic atoms near T = 0 in an opticallattice has proved an ideal test-system to study important quantum phenomena of solid state physics with well controllable parameters [1,2]. A striking example is a reversible quantum phase transition from the superfluid into the Mott-insulator phase [3] simply by changing the optical potential depth. In the Mott insulator case each optical-lattice site is filled with a fixed number of atoms down to one or two atoms per site. For a deep enough lattice the atoms cannot move and form an artificial crystal. Naturally it is now interesting to study further complex solid state phenomena in this system, e.g. by exploiting the internal atomic level structure, which bears a strong analogy to excitonic dynamics of molecular crystals (Frenkel excitons) as predicted in Ref. [4]. By the help of an optical cavity these excitons get strongly coupled over large distances via photons and form polaritons. In the present paper we investigate a special interesting case of such excitons [5] and cavity polaritons [6] which can appear only for lattice filled with two atoms per site, which is a straight forward to prepare in optical lattices by the help of a Mott insulator state with filling factor 2.Let us start from a degenerate gas of effective two-level atoms trapped in a 2D optical lattice, located within a cavity with a single cavity mode close to resonance with an internal atomic transition. The lattice laser is tuned far off resonance to the atomic excitations and results in light shifts of the ground and excited states with periodicity of half the laser wave length. Here we assume the two optical-lattices for ground and excited states located at the same positions, which can be realized for Alkali or Alkaline atoms. At certain magic laser frequencies the excited state even experiences an equal shift as the ground state [2,7]. We believe that more general lattice configurations might imply new physics, but this goes beyond our aim here. At temperatures close to T = 0, the atomic center of mass mo...