All-cellulose composite (ACC) films from oil palm empty fruit bunches (OPEFBs) were successfully fabricated through the surface selective dissolution of cellulose fibers in 8 wt% LiCl/DMAc via the solution casting method. The effect of dissolution time on the properties of the ACC films was assessed in the range of 5–45 min. The results showed that under the best conditions, there were sufficiently dissolved fiber surfaces that improved the interfacial adhesion while maintaining a sizable fraction of the fiber cores, acting as reinforcements for the material. The ACC films have the highest tensile strength and modulus of elasticity of up to 35.78 MPa and 2.63 GPa after 15 min of dissolution. Meanwhile, an X-ray diffraction analysis proved that cellulose I and II coexisted, which suggests that the crystallite size and degree of crystallinity of the ACC films had significantly declined. This is due to a change in the cellulose structure, which results in fewer voids and enhanced stress distribution in the matrix. Scanning electron microscopy revealed that the interfacial adhesion improved between the reinforcing fibers and matrices as the failure behavior of the film composite changed from fiber pullout to fiber breakage and matrix cracking. On the other hand, the thermal stability of the ACC film showed a declining trend as the dissolution time increased. Therefore, the best dissolution time to formulate the ACC film was 15 min, and the obtained ACC film is a promising material to replace synthetic polymers as a green composite.