Feature Dimensionality Reduction (DR) is an important topic in medical image retrieval. However, most medical image retrieval systems use naive principal component analysis for DR, which is not optimal. Recently, the class conditional nearest neighbour relation over pairs of points was introduced. In this paper, we extend it to class conditional K-nearest neighbour (ccknn). Based on ccknn, we define the within-class and between-class graph for Large-Margin Discriminat Analysis (LMDA). Moreover, an iterative expectation-maximisation framework is applied to LMDA to boost the performance. The experimental results demonstrate that the proposed approach yields significant improvements over the state-of-the-art DR algorithms.