We search for ultra-luminous Quasi-Stellar Objects (QSOs) at high redshift using photometry from the SkyMapper Southern Survey Data Release 3 (DR3), in combination with 2MASS, VHS DR6, VIKING DR5, AllWISE, and CatWISE2020, as well as parallaxes and proper motions from Gaia DR2 and eDR3. We report 142 newly discovered Southern QSOs at 3.8 < z < 5.5, of which 126 have M145 < −27 ABmag and are found in a search area of 14 486 deg2. This Southern sample, utilising the Gaia astrometry to offset wider photometric colour criteria, achieves unprecedented completeness for an ultra-luminous QSO search at high redshift. In combination with already known QSOs, we construct a sample that is >80 per cent complete for M145 < −27.33 ABmag at z = 4.7 and for M145 < −27.73 ABmag at z = 5.4. We derive the bright end of the QSO luminosity function at restframe 145 nm for z = 4.7 − 5.4 and measure its slope to be β =−3.60 ± 0.37 and β =−3.38 ± 0.32 for two different estimates of the faint-end QSO density adopted from the literature. We also present the first z ∼ 5 QSO luminosity function at restframe 300 nm.