In this article, discrete variants of several results from vector calculus are studied for classical finite difference summation by parts operators in two and three space dimensions. It is shown that existence theorems for scalar/vector potentials of irrotational/solenoidal vector fields cannot hold discretely because of grid oscillations, which are characterised explicitly. This results in a non-vanishing remainder associated to grid oscillations in the discrete Helmholtz Hodge decomposition. Nevertheless, iterative numerical methods based on an interpretation of the Helmholtz Hodge decomposition via orthogonal projections are proposed and applied successfully.In numerical experiments, the discrete remainder vanishes and the potentials converge with the same order of accuracy as usual in other first order partial differential equations. Motivated by the successful application of the Helmholtz Hodge decomposition in theoretical plasma physics, applications to the discrete analysis of magnetohydrodynamic (MHD) wave modes are presented and discussed.