The formalism of relativistic partial wave expansion is developed for four-point celestial amplitudes of massless external particles. In particular, relativistic partial waves are found as eigenfunctions to the product representation of celestial Poincaré Casimir operators with appropriate eigenvalues. The requirement of hermiticity of Casimir operators is used to fix the corresponding integral inner product, and orthogonality of the obtained relativistic partial waves is verified explicitly. The completeness relation, as well as the relativistic partial wave expansion follow. Example celestial amplitudes of scalars, gluons, gravitons and open superstring gluons are expanded on the basis of relativistic partial waves for demonstration. A connection with the formulation of relativistic partial waves in the bulk of Minkowski space is made in appendices.