Recent work has established the existence of stable quantum phases of matter described by symmetric tensor gauge fields, which naturally couple to particles of restricted mobility, such as fractons. We focus on a minimal toy model of a rank 2 tensor gauge field, consisting of fractons coupled to an emergent graviton (massless spin-2 excitation). We show how to reconcile the immobility of fractons with the expected gravitational behavior of the model. First, we reformulate the fracton phenomenon in terms of an emergent center of mass quantum number, and we show how an effective attraction arises from the principles of locality and conservation of center of mass. This interaction between fractons is always attractive and can be recast in geometric language, with a geodesiclike formulation, thereby satisfying the expected properties of a gravitational force. This force will generically be short-ranged, but we discuss how the power-law behavior of Newtonian gravity can arise under certain conditions. We then show that, while an isolated fracton is immobile, fractons are endowed with finite inertia by the presence of a large-scale distribution of other fractons, in a concrete manifestation of Mach's principle. Our formalism provides suggestive hints that matter plays a fundamental role, not only in perturbing, but in creating the background space in which it propagates.