Visual fatigue evaluation plays an important role in applications such as virtual reality since the visual fatigue symptoms always affect the user experience seriously. Existing visual evaluation methods require hand-crafted features for classification, and conduct feature extraction and classification in a separated manner. In this paper, we conduct a designed experiment to collect electroencephalogram (EEG) signals of various visual fatigue levels, and present a multi-scale convolutional neural network (CNN) architecture named MorletInceptionNet to detect visual fatigue using EEG as input, which exploits the spatial-temporal structure of multichannel EEG signals. Our MorletInceptionNet adopts a joint space-time-frequency features extraction scheme in which Morlet wavelet-like kernels are used for time-frequency raw feature extraction and inception architecture are further used to extract multi-scale temporal features. Then, the multi-scale temporal features are concatenated and fed to the fully connected layer for visual fatigue evaluation using classification. In experiment evaluation, we compare our method with five state-of-the-art methods, and the results demonstrate that our model achieve overally the best performance better performance for two widely used evaluation metrics, i.e., classification accuracy and kappa value. Furthermore, we use input-perturbation network-prediction correlation maps to conduct in-depth analysis into the reason why the proposed method outperforms other methods. The results suggest that our model is sensitive to the perturbation of β (14-30 Hz) and γ (30-40 Hz) bands. Furthermore, their spatial patterns are of high correlation with that of the corresponding power spectral densities which are used as evaluation features traditionally. This finding provides evidence of the hypothesis that the proposed model can learn the joint time-frequency-space features to distinguish fatigue levels automatically.2 of 18 ideally indicators. Since mental fatigue is supposed as one aspect of 3D visual fatigue, these features can hardly represent psychological changes [7].Recently, researches have opened up the possibility for visual fatigue assessment with noninvasive approaches based on electroencephalography (EEG) [1]. EEG signals record integration of spontaneous electrical activities of a large number of brain cells from the scalp, which is closely related with mental and physical states [8]. All event-related potentials in EEG are limited in duration and in frequency. The majority of events activate distinct brain regions [9]. This means researchers can extract features from space, time, and frequency dimensions of the EEG data to build efficient classification models for visual fatigue evaluation. Traditionally, the most popular features for visual fatigue evaluation are time-frequency features (e.g., power spectral), which can be obtained by time-frequency transformation (e.g., wavelet transformation). Power spectral of various frequency bands are proven to be related with visual fatigue l...