In this paper we discuss the adjoint stabilised finite element method introduced in E. Burman, Stabilized finite element methods for nonsymmetric, noncoercive and ill-posed problems. Part I: elliptic equations, SIAM Journal on Scientific Computing [10] and how it may be used for the computation of solutions to problems for which the standard stability theory given by the Lax-Milgram Lemma or the Babuska-Brezzi Theorem fails. We pay particular attention to ill-posed problems that have some conditional stability property and prove (conditional) error estimates in an abstract framework. As a model problem we consider the elliptic Cauchy problem and provide a complete numerical analysis for this case. Some numerical examples are given to illustrate the theory.