Current perpendicular-to-plane (CPP) giant magnetoresistance (GMR) effects are of interest in a possible application of magnetic sensor elements, such as read-head of hard disk drives. To improve the junction performance, the interface tailoring effects were investigated for the Heulser alloy, Co2Fe0.4Mn0.6Si (CFMS), based CPP-GMR junctions with an L12-Ag3Mg ordered alloy spacer. Ultra-thin Fe or Mg inserts were utilized for the CFMS/Ag3Mg interfaces, and CPP-GMR at low bias current density, J and the J dependence were evaluated for the junctions. Although, at low bias J, MR ratio decreased with increasing the inserts thickness, the device output at high bias J exhibited quite weak dependence on the insert thickness. The output voltages of the order of 4 mV were obtained for the junctions regardless of the insert at an optimal bias J for each. The critical current density Jnormalc was evaluated by the shape of MR curves depending on J. Jnormalc increased with the insert thicknesses up to 0.45 nm. The enhancement of Jnormalc suggests that spin-transfer-torque effect may reduce in the junctions with inserts, which enables a reduction of noise and can be an advantage for device applications.