The viscoelastic property of vitrimers, which belong to a group of covalent bond adaptable networks, is one of the most critical properties during the applications of these materials. Herein, the significant effect of the network architecture of the cross-linked polymers, particularly the arrangements of cross-links and chains, on the viscoelastic property, which is determined directly from the structure of the precursor polymers, is reported. The systematic investigation reveals that vitrimers from telechelic polymers with reactive groups at the chain ends show a sharp elastic modulus decrease. Furthermore, these polymers demonstrate highly fluid-like behavior at elevated temperature and fast stress relaxation, compared to vitrimers from random copolymers that have similar chain length and number of cross-linking groups.