Remote monitoring of physical frailty is important to personalize care for slowing down the frailty process and/or for the healthy recovery of older adults following acute or chronic stressors. Taking the Fried frailty criteria as a reference to determine physical frailty and frailty phenotypes (slowness, weakness, exhaustion, inactivity), this study aimed to explore the benefit of machine learning to determine the least number of digital biomarkers of physical frailty measurable from a pendant sensor during activities of daily living. Two hundred and fifty-nine older adults were classified into robust or pre-frail/frail groups based on the physical frailty assessments by the Fried frailty criteria. All participants wore a pendant sensor at the sternum level for 48 h. Of seventeen sensor-derived features extracted from a pendant sensor, fourteen significant features were used for machine learning based on logistic regression modeling and a recursive feature elimination technique incorporating bootstrapping. The combination of percentage time standing, percentage time walking, walking cadence, and longest walking bout were identified as optimal digital biomarkers of physical frailty and frailty phenotypes. These findings suggest that a combination of sensor-measured exhaustion, inactivity, and speed have potential to screen and monitor people for physical frailty and frailty phenotypes.