Classic exponent-Fourier moments (EFMs) have been popularly used for image reconstruction and invariant classification. However, EFMs lack natively the translation and scaling-invariant; in addition, they exhibit two types of drawbacks, namely numerical instability and reconstruction error, which in turn influence their reconstruction capability and image classification accuracy. This study considers the challenge of defining modified EFMs (MEFMs), which are based on modified exponent polynomials. In our methods, the basis function of traditional EFMs is appropriately modified, and these modified basis functions are used to replace the original ones. The basis function of the proposed moments is composed of piecewise modified exponent polynomials modulated by a variable parameter exponential envelope. Various types of optimal-order moments can be established by slightly adjusting the bandwidth of the modified basis functions. Finally, we extend the rotation-invariant feature of previous works and propose a new method of scaling and rotation-invariant image recognition using the proposed moments in a log-polar coordinate domain. The translation invariance can then be achieved by an image projection operation, which is substituted for the traditional approach based on the calculation of image geometric moments. The experimental results demonstrate that the MEFMs perform better than traditional EFMs and other classic orthogonal moments including the latest image moments in terms of the image reconstruction capability and the invariant recognition accuracy of smoothing filters, in both noise-free and noisy conditions.