This study aims to clarify the effect of selenium (Se) on the machinability of AISI 4140 steel in the as-hot rolled (HR) and quenched and tempered (QT) conditions. Machining tests were conducted to examine the progressive tool flank wear and the chip formation. The characteristics of the workpiece materials such as hardness, microstructure, and non-metallic inclusions were investigated. The worn tools were analyzed to characterize the possible deposit formed on the tool surface. In addition, micro-computed tomography (μ-CT) and high-resolution transmission electron microscopy (TEM) were utilized to study the Mn(S,Se) inclusions in severely deformed chips. The investigation showed that Se micro-alloying did not improve the machinability of the HR 4140 steel. However, in the QT condition, the superior machinability was observed for the Se-treated 4140 steel when compared with the untreated steel. Therefore, the effect of Se on improving steel machinability was considered to be strongly dependent on the structure and properties of metal matrix. The factors influencing the improvement in machinability of Se-treated QT steel are discussed.