We present a binary classifier based on neural networks to detect gapped quantum phases. By considering the errors on top of a suitable reference state describing the gapped phase, we show that a neural network trained on the errors can capture the correlation between the errors and can be used to detect the phase boundaries of the gapped quantum phase. We demonstrate the application of the method for matrix product state calculations for different quantum phases exhibiting local symmetry-breaking order, symmetry-protected topological order, and intrinsic topological order.