Most quantum computer realizations require the ability to apply local fields and tune the couplings between qubits, in order to realize single bit and two bit gates which are necessary for universal quantum computation. We present a scheme to remove the necessity of switching the couplings between qubits for two bit gates, which are more costly in many cases. Our strategy is to compute in and out of carefully designed interaction free subspaces analogous to decoherence free subspaces, which allows us to effectively turn off and turn on the interactions between the encoded qubits. We give two examples to show how universal quantum computation is realized in our scheme with local manipulations to physical qubits only, for both diagonal and off diagonal interactions. Quantum computation is generally formulated in terms of a collection of qubits subject to a sequence of single and two bit operations [1]. This implies that the effective local fields applied to individual qubits, and the couplings between the qubits, are variable functions subject to external control. In many cases, two bit operations, whose implementation depends on certain interactions between qubits, are more difficult than single bit gates. They can require more sophisticated manipulations, therefore may take a longer time and cause stronger decoherence. This usually results from the requirement to vary (in the simplest case just switch on and off) the couplings between qubits, which is not always possible, or easy to realize. One such example is quantum computing with Josephson junction devices, both charge and flux type [2,3,4,5,6]. In this case, the coupling between qubits is most naturally realized with a hard wired capacitor or inductor, whose value is fixed by the fabrication and cannot be tuned during the computation. The superconducting quantum computing community has been working hard to devise variable coupling schemes [5,7,8,9], but it is generally agreed that none of these proposed switches is completely satisfactory [9]. Most of them [5,7] require external controls, thus are likely to be major decoherence sources. Others were designed to avoid such external controls, but may suffer other problems, for instance the number of qubits that can be incorporated into the system can be limited [8,9], which is at odd with the supposed scalability of a solid state quantum computer.An always on and un-tunable coupling causes certain problems for quantum computation, depending on the particular form of the interaction. If the interaction Hamiltonian is diagonal in the computational basis, each qubit state will gain additional phases depending on the states of the qubits to which it is coupled, even in the idle mode. It is then necessary to keep track of these phases, or suppress them by repeated refocusing pulses like those used in NMR, which requires high precisions and complicates the operation [8,9]. The situation is more serious in the case of off diagonal interactions, because these interactions will cause the states of the qubits to propaga...