The local descriptors have gained wide range of attention due to their enhanced discriminative abilities. It has been proved that the consideration of multi-scale local neighborhood improves the performance of the descriptor, though at the cost of increased dimension. This paper proposes a novel method to construct a local descriptor using multi-scale neighborhood by finding the local directional order among the intensity values at different scales in a particular direction. Local directional order is the multi-radius relationship factor in a particular direction. The proposed local directional order pattern (LDOP) for a particular pixel is computed by finding the relationship between the center pixel and local directional order indexes. It is required to transform the center value into the range of neighboring orders. Finally, the histogram of LDOP is computed over whole image to construct the descriptor. In contrast to the state-of-the-art descriptors, the dimension of the proposed descriptor does not depend upon the number of neighbors involved to compute the order; it only depends upon the number of directions. The introduced descriptor is evaluated over the image retrieval framework and compared with the state-of-the-art descriptors over challenging face databases such as PaSC, LFW, PubFig, FERET, AR, AT&T, and ExtendedYale. The experimental results confirm the superiority and robustness of the LDOP descriptor.Face description is required to make the face recognition approaches robust to intra-class variations and discriminative to inter-class similarities. The face image-descriptor based methods [10], [11], [12], [13], and deep learning based methods [14], [15], are the two major approaches widely adapted in the research community, for face retrieval. The advantages of former methods are data independence, ease of use, no complex computation facility needed and robustness to realsituation variations like rotation, scale, expression and illumination differences.The image-descriptor based methods can be further classified into two categories, i.e., hand-crafted descriptor and learning based descriptor. The designing of hand-crafted image descriptors are the mostly followed research area for the face representation. The local binary pattern (LBP) is proved as a very efficient and simple feature descriptor to capture the micro information of the image [16], [17]. The LBP based approaches have shown promising performance for different computer vision applications such as texture classification and retrieval [18], [19], [20], [21]. Ahonen et al. investigated the suitability of LBP for the face recognition task [10]. They computed the LBP descriptor over several blocks and concatenated to form a single descriptor. This approach outperformed the classical methods such as PCA, Bayesian Classifier and Elastic Bunch Graph Matching. Recently, Huang et al. surveyed the LBP based approaches for face recognition [22] and Yang and Chen [23] have presented a comparative study over LBP based face recognition techniques. Seve...