Eddy Current Pulsed Thermography is a crucial non-destructive testing technology which has a rapidly increasing range of applications for crack detection on metals. Although the unsupervised learning method has been widely adopted in thermal sequences processing, the research on supervised learning in crack detection remains unexplored. In this paper, we propose an end-to-end pattern, deep region learning structure to achieve precise crack detection and localization. The proposed structure integrates both time and spatial pattern mining for crack information with a deep region convolution neural network. Experiments on both artificial and natural cracks have shown attractive performance and verified the efficacy of the proposed structure.