Zero velocity Update (ZUPT) is an effective method to restrain the error divergence of the Inertial Navigation System (INS). Right detection of zero velocity points and appropriate filtering algorithm are the key factors for the success of ZUPT. In this paper, a ZUPT method for vehicle mounted INS based on neural network and Kalman Filter is proposed. The efficiency and accuracy of the zero velocity detection is improved by neural network. The precision of the proposed method can reach 99.19%, and the recall rate is improved by 24% compared with the method based on SVM. And this method has similar accuracy and better real-time performance with the method based on LSTM. Based on the zero velocity detection by neural network, the navigation error is estimated and compensated by Kalman Filter. The effectiveness of the proposed method is proved by vehicular experiment which shows that the velocity error is reduced to 24.2% and the position error is reduced to 9.5%.