In this paper, we study the passive problem of uncertain fractional-order neural networks (UFONNs) with time-varying delays. First, we give a sufficient condition for the asymptotic stability of UFONNs with bounded time-varying delays by using the fractional-order Razumikhin theorem. Secondly, according to the above stability criteria and some properties of fractional-order calculus, a delay-dependent condition that can guarantee the passivity of UFONNs with time-varying delays is given in the form of a linear matrix inequality (LMI) that can be reasonably solved in polynomial time using the LMI Control Toolbox. These conditions are not only delay-dependent but also order-dependent, and less conservative than some existing work. Finally, the rationality of the research results is proved by simulation.