In this work, we investigate the generation of second harmonic light by gold nanorods and demonstrate that the collected nonlinear intensity depends upon a phase interplay between different modes available in the nanostructure. By recording the backward and forward emitted second harmonic signals from nanorods with various lengths, we find that the maximum nonlinear signal emitted in the forward and backward directions is not obtained for the same 2 nanorod length. We confirm the experimental results with the help of full-wave computations done with a surface integral equation method. These observations are explained by the multipolar nature of the second harmonic emission, which emphasizes the role played by the relative phase between the second harmonic modes. Our findings are of a particular importance for the design of plasmonic nanostructures with controllable nonlinear emission and nonlinear plasmonic sensors as well as for the coherent control of harmonic generations in plasmonic nanostructures. Motivated by different specific features, the study of nonlinear optical processes in plasmonic nanostructures has become a vivid field of research [1, 2]. First, the intrinsic nonlinear response of plasmonic materials enables the investigation of subtle nonlinear mechanisms associated with the surface [3, 4], the shape [2], the roughness [5, 6], and the symmetry [2] of plasmonic nanostructures. Second, local field enhancement associated with the plasmon resonances can boost nonlinear processes including second harmonic generation (SHG) [7-9], third harmonic generation [10][11][12], and nonlinear photoluminescence [13][14][15][16], such that these nonlinear signals provide indirect entry to the local field enhancement [17][18][19]. Third, the plasmonic modes associated with a nanostructure are the underlying framework upon which nonlinear processes can be built [20,21]. It is only quite recently that the role played in nonlinear plasmonics by the interaction between the different modes available at the fundamental and harmonic frequencies has been recognized, leading to different multiresonant nanostructure designs that benefit from the interaction of several plasmonic modes at different frequencies [21][22][23][24][25]. This is especially the case for SHG where a dipolar excitation at the fundamental frequency produces a nonlinear signal which is essentially quadrupolar [26].