In a silvi-medicinal system, the plant secondary metabolites (PSMs) released from medicinal herbs could affect tree litter decomposition and nutrient release. However, the specific effects of PSMs on arboreous litter decomposition are still not well understood. In this study, the extracts of nine types of medicinal herbs were used to treat Pinus armandii Franch. and Larix gmelinii (Rupr.) Kuzen. litter during a simulated half-year decomposition. The effects of the extracts on the decomposition and the N and P release of the conifer litter were investigated. The results indicated that most of the medicinal herb extracts significantly inhibited the late decomposition of P. armandii litter, whereas only two of them accelerated the entire decomposition process. Only a few significantly affected the decomposition of the L. gmelinii litter. Four of the nine types of extract significantly inhibited the N and P release of the P. armandii litter, while 3/9 and 6/9 inhibited the N and P release of the L. gmelinii litter, respectively. The accelerating effects of the extracts on the cellulase activity and the inhibitory effects on the polyphenol oxidase activity might be responsible for the early acceleration and late inhibition of litter decomposition, while the effects of the extracts on the activities of protease and phosphatase might not be the main reason for the inhibitory or accelerating effects on the N and P release. In general, the inhibitory effects of medicinal herbs on the nutrient cycling of ecosystems should be taken into consideration when building silvi-medicinal systems, especially in P. armandii forests.