The Indochina Peninsula (ICP) has a critical effect in shaping the Asian summer monsoon (ASM). However, the seasonal responses of the ASM to the ICP are not fully understood. This study employs a 1° atmospheric general circulation model to examine the different contributions of the ICP’s orography and land–sea contrast to the ASM during the early and late summer. Results indicate that the orographic effect increases South Asian rainfall and reduces the rainfall over the South China Sea (SCS) and North China in early summer, but its influence on monsoonal circulation and rainfall is limited to East Asia in late summer. The impact of the ICP’s land–sea contrast is basically opposite in the two summer stages. With the presence of the ICP, SCS rainfall is enhanced but South Asian rainfall is weakened in early summer. In late summer, however, rainfall from the ICP to the northwestern Pacific is strikingly reduced, accompanied by intensified rainfall over South Asia. Relatively, the orographic effect seems to be more important in modulating the South Asian monsoon in early summer, while the land–sea contrast is dominant in strengthening the SCS monsoon and suppressing the Northwest Pacific monsoon via the interaction between the induced local circulation and multi-level ASM subsystems. In late summer, the orographic effect on the ASM is much weaker compared to the land–sea contrast, which plays a critical role by shifting the subtropical high southwestwards and through the “thermal adaption” feedback mechanism. Therefore, the orographic impact of the ICP on the ASM differs from that of the land–sea contrast in the two summer stages.