Permutation testing has been widely implemented in voxel-based morphometry (VBM) tools. However, this type of non-parametric inference has yet to be thoroughly compared with traditional parametric inference in VBM studies of brain structure. Here we compare both types of inference and investigate what influence the number of permutations in permutation testing has on results in an exemplar study of how grey matter proportion changes with age in a group of working age adults. High resolution T1-weighted volume scans were acquired from 80 healthy adults aged 25–64 years. Using a validated VBM procedure and voxel-based permutation testing for Pearson product-moment coefficient, the effect sizes of changes in grey matter proportion with age were assessed using traditional parametric and permutation testing inference with 100, 500, 1000, 5000, 10000 and 20000 permutations. The statistical significance was set at P < 0.05 and false discovery rate (FDR) used to correct for multiple comparisons. Clusters of voxels with statistically significant (PFDR < 0.05) declines in grey matter proportion with age identified with permutation testing inference (N ≈ 6000) were approximately twice the size of those identified with parametric inference (N = 3221 voxels). Permutation testing with 10000 (N = 6251 voxels) and 20000 (N = 6233 voxels) permutations produced clusters that were generally consistent with each other. However, with ≥ 1000 permutations there were approximately 20% more statistically significant voxels (N = 7117 voxels) than with 10000 permutations. Permutation testing inference may provide a more sensitive method than traditional parametric inference for identifying age-related differences in grey matter proportion. Based on the results reported here, at least 10000 permutations should be used in future univariate VBM studies investigating age related changes in grey matter to avoid potential false findings. Additional studies using permutation testing in large imaging databanks are required to address the impact of model complexity, multivariate analysis, number of observations, sampling bias and data quality on the accuracy with which subtle differences in brain structure associated with normal ageing can be identified.