Recent experiments by Hayashi et al. [Phys. Rev. Lett. 91, 226804 (2003)] demonstrate coherent oscillations of a charge quantum bit (qubit) in laterally defined quantum dots. We study the intrinsic electron-phonon decoherence and gate performance for the next step: a system of two coupled charge qubits. The effective decoherence model contains properties of local as well as collective decoherence. Decoherence channels can be classified by their multipole moments, which leads to different lowenergy spectra. It is shown that due to the super-Ohmic spectrum, the gate quality is limited by the single-qubit Hadamard gates. It can be significantly improved, by using double-dots with weak tunnel coupling.