Alzheimer's disease (AD) is characterized by progressive cognitive decline. Besides cognitive deficit, AD is also characterized by behavioral and psychological symptoms in dementia (BPSD). However, therapeutic management of BPSD remains challenging. HuanglianJiedu decoction (HLJDD), a traditional Chinese prescription, consisting of four herbs, is applied to treat AD, especially AD with BPSD. Though HLJDD, has the traditional combination with the principal herb Coptidis rhizoma (Huang-lian), it might, however, not be suitable for treating BPSD. Elucidating the mechanism underlying each herb is critical to the disease-matched combination of HLJDD. In this study, network pharmacology was used to determine the targets and biological processes regulated by HLJDD in the treatment of BPSD. Moreover, molecular docking was utilized to evaluate the binding activity between the herbs' main active ingredients and neurotransmitter receptors. The results showed that Scutellariae radix (Huang-qin) and Phellodendri chinrnsis cortex (Huang-bai) exhibited better anti-BPSD effects when compared to Coptidis rhizoma and Gardeniae fructus (Zhi-zi). Scutellariae radix exhibited superior anti-neuroinflammation functions, with better blood vessel regulation effects. Phellodendri chinrnsis cortex showed a higher binding affinity to the dopamine D2 receptor (DRD2) and 5-hydroxytryptamine receptor 2A (HTR2A). Coptidis rhizoma and Gardeniae fructus were better in neuronal signaling. In conclusion, for treating BPSD, Scutellariae radix and Phellodendri chinrnsis cortex are the principal herbs while Coptidis rhizoma and Gardeniae fructus are the ancillary herbs.