The growth of nanocrystals has widely been researched recently through an in situ high‐resolution transmission electron microscopy technique, which reveals the process of morphological and structural evolutions. For nanocrystals, the underlying growth modes are mostly determined by growth environment and crystal morphology. Here, the direct growth process of the PbSe nanocrystals via controlling the temperature is clearly observed. The results show that the PbSe nanocrystals start growth following oriented attachment growth mode, and then change to growth with grain boundary migration at moderate temperature as the heat activated nanocrystals gather together with decreased degree of freedom for crystal rotation. During the grain boundary migration, the smaller nanocrystals are inclined to be assimilated by larger ones through interfacial atom reconfigurations, which are observed to take place through strain mediated atom migration. The growth mode changes in different growth states with a hybrid growth mode of oriented attachment and grain boundary migration during the whole growth process.