The random batch Ewald (RBE) is an efficient and accurate method for molecular dynamics (MD) simulations of physical systems at the nano-/micro-scale. The method shows great potential to solve the computational bottleneck of long-range interactions, motivating a necessity to accelerating short-range components of the non-bonded interactions for a further speedup of MD simulations. In this work, we present an improved RBE method for the non-bonding interactions by introducing the random batch idea to constructing neighbor lists for the treatment of both the short-range part of the Ewald splitting and the Lennard-Jones potential. The efficiency of the novel neighbor list algorithm owes to the stochastic minibatch strategy which can significantly reduce the total number of neighbors. We implement the improved RBE method in the LAMMPS package. The accuracy and stability of the algorithm are demonstrated by both theoretical analysis and benchmark simulations. Numerical tests on computer performance by conducting large-scaled MD simulations for systems including up to 0.1 billion water molecules, run on massive cluster with up to 50 thousand CPU cores, demonstrating the attractive features such as the high parallel scalability and memory-saving of the method in comparison to the existing methods.