SLAM approaches rely on loop closure strategies to avoid and/or correct the inconsistencies in the resulting map. These inconsistencies are mainly caused by the effect of sensor noise in odometry sources. For the case of visual SLAM, loop detection typically rely on the repetitive detection and matching of texture-based keypoints. Weakly textured environments, however, can lead to scenes lacking these kind of points and, hence, poor-performing loop detectors. An alternative for these environments is the use of geometrical cues such as line segments, which are frequently present within human-made, structured environments. Under this context, in this work, we introduce a novel appearance-based loop closure detection method that integrates lines and points to enhance performance in these scenarios. For this purpose, we build an incremental Bag-of-Binary-Words scheme for each visual cue to retrieve previously seen images from the two complementary perspectives. Furthermore, we rely on a late fusion strategy to combine the image candidates resulting for both visual vocabularies. An effective mechanism to group similar images close in time is applied next to reduce the effort of the image candidate search. Finally, we propose a novel scheme to validate geometrically the loop candidates, integrating lines into the procedure. The proposed approach compares favourably with other state-of-the-art methods for several datasets.