In this paper, we propose a scheme for implementing a Toffoli gate in a system with an atom that has six levels in a lambda configuration, interacting with a high-Q cavity containing four modes. Here, we reduce a six-level system into an effective three-level behavior by applying the adiabatic elimination method. Next, we calculate the probabilities of the states of the interest as well as the fidelity. We also study the effects of photonic and atomic decay rates on the evolution of the system which is reasonably less sensitive to decoherence.