In the traditional observability analysis methods, only current and a few historical measurement values are considered. For the linear systems, the satisfactory analysis results can be obtained. However, for the nonlinear systems, the analysis results are not completely in accordance with the navigation errors since the high-order nonlinearity of the navigation system is not embodied on the observability matrix. In order to solve this problem, a fractional differentiation-based observability analysis method is developed. Considering the fact that the fractional differentiation with respect to time is characterized by long-term memory effects, the fractional derivative of the measurement model with respect to time is considered. By this means, more historical measurement data can be exploited. And then the fractional differentiationbased observability matrix, which reflects the high-order nonlinearity of the navigation system, is constructed. Finally, as the condition number is not directly proportional to the positioning error, to evaluate the navigation performance, the exponent weighted condition number is developed, where small condition numbers have large weight. The simulation results demonstrate that the fractional differentiation-based observability analysis method is sensitive to the orbital elements in the nonlinear X-ray pulsar navigation system, and has small calculation load. In addition, compared to the traditional observability analysis methods, it is observed that its analysis result matches well with the X-ray pulsar navigation performance.