The bag of words approach describes an image as a histogram of visual words. Therefore, the structural relation between words is lost. Since graphs are well adapted to represent these structural relations, we propose, in this paper, an image classification framework which draws benefit from the efficiency of the graph in modeling structural information and the good classification performances given by the bag of words method. For each image in the dataset, a graph is created by modeling the spatial relations between dense local patches. Thus, we obtain a graph dataset. From the graph dataset, we select the most frequent subgraphs to construct the bag of subgraphs (BoSG) and we associate to each image a subgraph histogram that describes its visual content. For experiments, we have used the two challenging datasets: 15 Scenes and Pascal VOC 2007. Experimental results show that the proposed method outperforms the bag of words and the spatial pyramid models in terms of recognition rate.