The modified Mallet scale (MMS) is commonly used to grade shoulder function in brachial plexus birth injury (BPBI) but has limited sensitivity and cannot grade scapulothoracic and glenohumeral mobility. This study aims to evaluate if the addition of a wearable inertial movement unit (IMU) system could improve clinical assessment based on MMS. The system validity was analyzed with simultaneous measurements with the IMU system and an optical camera system in three asymptomatic individuals. Test–retest and interrater reliability were analyzed in nine asymptomatic individuals and six BPBI patients. IMUs were placed on the upper arm, forearm, scapula, and thorax. Peak angles, range of motion, and average joint angular speed in the shoulder, scapulothoracic, glenohumeral, and elbow joints were analyzed during mobility assessments and MMS tasks. In the validity tests, clusters of reflective markers were placed on the sensors. The validity was high with an error standard deviation below 3.6°. Intraclass correlation coefficients showed that 90.3% of the 69 outcome scores showed good-to-excellent test–retest reliability, and 41% of the scores gave significant differences between BPBI patients and controls with good-to-excellent test–retest reliability. The interrater reliability was moderate to excellent, implying that standardization is important if the patient is followed-up longitudinally.