When machine vision is used to monitor the health of tunnel lining structure, there are errors in measuring the crack length information due to the prevalence of fracture in the detected cracks. This paper proposes a graph theory based crack backbone connection algorithm. First, the morphological principle is used to describe the point characteristics of the crack skeleton line. Then the graph theory principle is employed to establish the tree structure of the crack skeleton line and the mapping relationship between them, converting the crack backbone extraction problem into a path optimization problem. Next, the complete connection of segment-by-segment cracks is completed according to the shortest path distance and direction continuity criterion. Finally, the burr removal effect and complete connection effect of the algorithm in this paper are compared with other baseline methods by experiments, respectively. The experimental results show that this algorithm is more accurate for crack connections. In addition, the effectiveness, necessity, robustness, and generalization of this algorithm are evaluated and demonstrated using multiple real scenarios, including e.g. houses and road pavements.