The initial aim of the present paper is to provide a complete description for the eigenvalue problem of the non-commutative harmonic oscillator (NcHO), which is given by a two-by-two system of paritypreserving ordinary differential operator [19], in terms of Heun's ordinary differential equations, the second order Fuchsian differential equations with four regular singularities in a complex domain. This description has been achieved for odd eigenfunctions in Ochiai [16] nicely but missing for even eigenfunctions up to now. As a by-product of this study, using the monodromy representation of Heun's equation, we prove that the multiplicity of the eigenvalue of the NcHO is at most two. Moreover, we give a condition for the existence of a finite-type eigenfunction (essentially, given by a finite sum of Hermite functions) of the eigenvalue problem and an explicit example of such eigenvalues, from which one finds that doubly degenerate eigenstates of the NcHO actually exist even in the same parity. In the final section, as the second main purpose of this paper, we discuss a connection between the quantum Rabi model [2, 13, 28] and the operator naturally arising from the NcHO through the oscillator representation of the Lie algebra sl2, by the general confluence procedure for Heun's equation and Langlands' quotient realization of a different representation of sl2.