In this note we construct Q-operators for the spin s open Heisenberg XXX chain with diagonal boundaries in the framework of the quantum inverse scattering method. Following the algebraic Bethe ansatz we diagonalise the introduced Q-operators using the fundamental commutation relations. By acting on Bethe off-shell states and explicitly evaluating the trace in the auxiliary space we compute the eigenvalues of the Q-operators in terms of Bethe roots and show that the unwanted terms vanish if the Bethe equations are satisfied.