Several studies have investigated factors associated with physical aggression during adolescence. Yet, little is known about the longitudinal relationship between drug use, particularly alcohol use, and physical aggression among minority youth. The present study examined the effects of alcohol and substance use at age 11 on trajectories of physical aggression over time (ages 12–14) among urban adolescents from Chicago, IL. Data from the Project Northland Chicago (n = 3038, 49.4% female) was used. The current study sample included 1,160 Black, 1,015 Hispanic and 863 White/other adolescents for a total of 3,038 adolescents. Four trajectories of physical aggression were identified: Non-aggressive (16%), Desistors (9%), Escalators (20%) and Chronic Aggressive (55%). After adjusting for physical aggression behaviors, delinquent friends, lack of supervised time, demographic variables, smoking and marijuana use, past year alcohol users at age 11 were 2.1 times more likely to be “Escalators” and 1.9 times more likely to be in the “Chronic Aggressive” group. Gender and ethnic differences were also observed in the trajectories of physical aggression. Black youth were 2.5 times more likely to be in the “Chronic Aggressive” group. Findings highlight the importance of targeting alcohol prevention to reduce physical aggression among urban young adolescents.