Pigs are important biomedical model animals for the study of human neurological diseases. Similar to human aggressive behavior in children and adolescents, weaned pigs also show more aggressive behavior after mixing, which has negative effects on animal welfare and growth performance. The identification of functional single-nucleotide polymorphisms (SNPs) related to the aggressive behavior of pigs would provide valuable molecular markers of the aggressive behavioral trait for genetic improvement program. The Rho GTPase–activating protein 24 (ARHGAP24) gene plays an important role in regulating the process of axon guidance, which may impact the aggressive behavior of pigs. By resequencing the entire coding region, partially adjacent introns and the 5′ and 3′ flanking regions, six and four SNPs were identified in the 5′ flanking region and 5′ untranslated region (UTR) of the porcine ARHGAP24 gene, respectively. Association analyses revealed that nine SNPs were significantly associated with aggressive behavioral traits (p = < 1.00 × 10–4–4.51 × 10–2), and their haplotypes were significantly associated with aggressive behavior (p = < 1.00 × 10–4–2.99 × 10–2). The core promoter region of the ARHGAP24 gene has been identified between −670 and −1,113 bp. Furthermore, the luciferase activity of allele A of rs335052970 was significantly less than that of allele G, suggesting that the transcriptional activity of the ARHGAP24 gene was inhibited by allele A of rs335052970. It was identified that the transcription factor p53 bound to the transcription factor binding sites (TFBSs) containing allele A of rs335052970. In porcine primary neural cells, p53 binds to the target promoter region of the ARHGAP24 gene, reduces its promoter transcriptional activity, and then reduces its messenger RNA (mRNA) and protein expression. The results demonstrated that the ARHGAP24 gene had significant genetic effects on aggressive behavioral traits of pigs. Therefore, rs335052970 in the ARHGAP24 gene can be used as a molecular marker to select the less aggressive pigs.