Due to the sticky and soft characteristics of aluminum alloy, it is easy to form adhesion on the tool rake face during cutting process. In this work, the relationship between adhesion and cutting force has been explored, and the orthogonal cutting tests were carried out at cutting speeds of 20 m/min-300 m/min and feed rates of 0.05 mm-0.3 mm. The slip line model was used to separate the edge force of the cutting edge to obtain the pure friction force of the tool rake face, thereby obtaining the average friction coefficient. The adhesion morphology was observed under the scanning electron microscope (SEM) and compared with the average friction coefficient. It is found that different adhesion morphologies correspond to different friction coefficient ranges. Simultaneously, the relationship between the temperature and normal stress on the tool rake face and the adhesion morphology and friction coefficient was investigated, and the results showed that the temperature is the major factor of the adhesion evolution and friction coefficient change. Combining the tool rake face temperature and the adhesion morphology, it was noted that the reduction of the friction coefficient is not only owing to the self-lubrication caused by thermal softening, but also due to the decrease of the total shear force caused by the incomplete contact. Finally, the relationship between the friction coefficient of the tool rake face and the cutting force and specific cutting energy was discussed, with the conclusion that the friction coefficient has an impact on the cutting force and specific cutting energy by changing the shear angle.