Exponential growths of social media and microblogging sites not only provide platforms for empowering freedom of expressions and individual voices, but also enables people to express antisocial be havior like online harassment, cyberbullying, and hate speech. Nu merous works have been proposed to utilize these data for social and antisocial behavior analysis, by predicting the contexts mostly for highlyresourced languages like English. However, some lan guages are underresourced, e.g., South Asian languages like Ben gali, that lack of computational resources for natural language pro cessing (NLP). In this paper 1 , we propose an explainable approach for hate speech detection from underresourced Bengali language, which we called DeepHateExplainer. In our approach, Bengali texts are first comprehensively preprocessed, before classifying them into political, personal, geopolitical, and religious hates, by employing neural ensemble of different transformerbased neural architectures (i.e., monolingual Bangla BERTbase, multilingual BERTcased and un cased, and XLMRoBERTa), followed by identifying important terms with sensitivity analysis and layerwise relevance propagation (LRP) to provide humaninterpretable explanations. Evaluations against several machine learning (linear and treebased models) and deep neural networks (i.e., CNN, BiLSTM, and ConvLSTM with word embeddings) baselines yield F1 scores of 84%, 90%, 88%, and 88%, for political, personal, geopolitical, and religious hates, respectively, during 3fold crossvalidation tests.