Objective: Mycobacterium tuberculosis (Mtb) is an airborne, contagious bacterial pathogen that causes widespread infections in humans. Using Mycobacterium marinum (Mm), a surrogate model organism for Mtb research, the present study develops a deep learning-based scheme that can classify the Mminfected and uninfected macrophages in tissue culture solely based on morphological changes. Methods: A novel weak-and semisupervised learning method is developed to detect and extract the cells, firstly. Then, transfer learning and fine-tuning from the CNN is built to classify the infected and uninfected cells. Results: The performance is evaluated by accuracy (ACC), sensitivity (SENS) and specificity (SPEC) with 10-fold cross-validation. It demonstrates that the scheme can classify the infected cells accurately and efficiently at the early infection stage. At 2 hour post infection (hpi), we achieve the ACC of 0.923 ± 0.005, SENS of 0.938 ± 0.020, and SPEC of 0.905 ± 0.019, indicating that the scheme has detected significant morphological differences between the infected and uninfected macrophages, although these differences are hardly visible to naked eyes. Interestingly, the ACC at 12 and 24 hpi are 0.749 ± 0.010 and 0.824 ± 0.009, respectively, suggesting that the infection-induced morphological changes are dynamic throughout the infection. Finally, deconvolution with guided propagation maps the key morphological features contributing to the classification. Significance: This proof-ofconcept study provides a novel venue to investigate bacterial pathogenesis in a macroscopic level and has a great promise in diagnosis of bacterial infections.