Background: Intrapapillary capillary loops (IPCLs) represent an endoscopically visible feature of early squamous cell neoplasia (ESCN) which correlate with invasion depth-an important factor in the success of curative endoscopic therapy. IPCLs visualised on magnification endoscopy with Narrow Band Imaging (ME-NBI) can be used to train convolutional neural networks (CNNs) to detect the presence and classify staging of ESCN lesions. Methods: A total of 7046 sequential high-definition ME-NBI images from 17 patients (10 ESCN, 7 normal) were used to train a CNN. IPCL patterns were classified by three expert endoscopists according to the Japanese Endoscopic Society classification. Normal IPCLs were defined as type A, abnormal as B1-3. Matched histology was obtained for all imaged areas. Results: This CNN differentiates abnormal from normal IPCL patterns with 93.7% accuracy (86.2% to 98.3%) and sensitivity and specificity for classifying abnormal IPCL patterns of 89.3% (78.1% to 100%) and 98% (92% to 99.7%), respectively. Our CNN operates in real time with diagnostic prediction times between 26.17 ms and 37.48 ms. Conclusion: Our novel and proof-of-concept application of computer-aided endoscopic diagnosis shows that a CNN can accurately classify IPCL patterns as normal or abnormal. This system could be used as an in vivo, real-time clinical decision support tool for endoscopists assessing and directing local therapy of ESCN.