Recent cosmic-ray measurements have found an anomaly in the cosmic-ray energy spectrum at GeV-TeV energies. Although the origin of the anomaly is not clearly understood, suggested explanations include the effect of cosmic-ray source spectrum, propagation effects, and the effect of nearby sources. In this paper, we propose that the spectral anomaly might be an effect of reacceleration of cosmic rays by weak shocks in the Galaxy. After acceleration by strong supernova remnant shock waves, cosmic rays undergo diffusive propagation through the Galaxy. During the propagation, cosmic rays may again encounter expanding supernova remnant shock waves, and get re-accelerated. As the probability of encountering old supernova remnants is expected to be larger than the younger remnants because of their bigger sizes, reacceleration is expected to be produced mainly by weaker shocks. Since weaker shocks generate a softer particle spectrum, the resulting re-accelerated component will have a spectrum steeper than the initial cosmic-ray source spectrum produced by strong shocks. For a reasonable set of model parameters, it is shown that the re-accelerated component can dominate the GeV energy region while the non-reaccelerated component dominates at higher energies, thereby explaining the observed GeV-TeV spectral anomaly.