Image annotation with high-level words using generalised attributes

Abstract

The emergence of social media sharing communities has led to the need for accurate context-based image retrieval methods, which can be accomplished by an automatic annotation system. The ability to annotate high-level context-based words is necessary for such a system; however, it is not well researched due to the inherent difficulty caused by the semantic gap. This thesis identifies a set of high-level words that are frequently used by users to describe images, with a baseline system constructed using linear classifiers. The concept of 'generalised attributes' is then proposed and used to improve prediction by bridging the gap between image features and high-level words. The generalised attribute 'anchor feature' proposed, together with the 'total distance' feature selection method, leads to optimal performance. The resulting system yields not only an improvement in statistical accuracy over the baseline, but also a huge improvement in the quality and relevance of images retrieved in image retrieval and tags predicted in tag recommendation.