Diselenide-containing paclitaxel nanoparticles (SePTX NPs) indicated selectivity of cytotoxicity between cancerous and normal cells in our previous work. Herein, the mechanism is revealed by molecular biology in detail. Cancer cells and normal cells were treated with the SePTX NPs and cell proliferation was measured using 3-(4, 5-dimethyl-2-thiazolyl)-2, 5-diphenyl-2-H-tetrazolium bromide (MTT) assay and cell morphology. Measurement of reactive oxygen species (ROS) levels and biochemical parameters were employed to monitor oxidative stress of the cells. JC-1 assay was used to detect the mitochondrial dysfunction of the cells. Terminal deoxynucleotidyl transferase-mediated dUTP nick end labeling (TUNEL) analysis was used to detect apoptosis of the cells. Immunofluorescence analysis and western blotting were employed to monitor changes in signaling pathway-related proteins. Compared with PTX, SePTX NPs has a good selectivity to cancer cells and can obviously induce the proliferation damage of cancer cells, but has no significant toxicity to normal cells, indicating that SePTX NPs has a specific killing effect on cancer cells. The results of mechanism research show that SePTX NPs can successfully inhibit the depolymerization of microtubules and induce cell cycle arrest, which is related to the upregulation of p53 and CyclinB1. Simultaneously, SePTX NPs can successfully induce oxidative stress, cause mitochondrial dysfunction, resulting in mitochondrial pathway-mediated apoptosis, which is related to the upregulation of autophagy-related protein LC3-II. On the other hand, lewis lung cancer C57BL/6 mice were used to evaluate the anti-tumor effects of SePTX NPs in vivo. Our data show that SePTX NPs exhibited high inhibiting efficiency against the growth of tumors and were able to reduce the side effects. Collectively, these data indicate that the high antitumor effect and selective cytotoxicities of SePTX NPs is promising in future cancer therapy.