Cement-based self-sensing composites with carbon nanotubes (CNT) have attracted attention due to their multifunctional properties and great potential for their application in the smart monitoring of concrete structures. In this study, the self-sensing properties of one paste and three mortars containing 0.50 and 0.75 wt% of CNT, and 1.5 and 1.0 sand/cement ratio were investigated, aiming to evaluate their impact on the piezoresistive response of the composites. The inclusion of sand in the cement paste with CNT led to a reduced gauge factor and a higher electrical noise response. The inert aggregates modified the compressive loading mechanical response of the composites and possibly acted as barriers to electronic mobility, by increasing the CNT conductive paths’ tortuosity or even interrupting them. The mortar containing 0.50% of CNT showed a higher electrical resistivity and, at the same time, greater sensitivity and a more linear self-sensing response than the one with 0.75% CNT, which can be explained by the CNT content being closer to its percolation threshold in the first. In this way, a lower CNT concentration generated a conductive network with a higher capacity to be rearranged under loading, generating significant changes in resistivity, but a higher CNT concentration presented a more stable and conductive network. The results suggested that both the conductive and non-conductive phases affect the detection performance of the composites and, therefore, must be dosed appropriately. Additionally, the test setup modifications positively affected the self-sensing response signal, which is particularly useful to reduce the deleterious effects of the sand additions in the matrix. This overall approach can make the use of self-sensing mortars in structural monitoring a viable option.