Due to their outstanding mechanical, thermal and durability properties, polymer matrix nanocomposites (PMCs) are nowadays a prominent area of research. The opportunity of applying PMCs in structural reinforcement and rehabilitation of damaged infrastructures, as well as working as a new structural material, justifies the increasing number of recent studies. In this review article, the effect of additing different reinforcements at nano-scale, such as carbon nanotubes, nanoclay, graphene or nanosilica to polymer matrices is discussed and the improvement in mechanical properties of PMCs is evaluated. Some concluding remarks and new perspectives on the use of PMCs in structures are given.