We study inverse problems for the Einstein equations with source fields in a general form. Under a microlocal linearization stability condition, we show that by generating small gravitational perturbations and measuring the responses near a freely falling observer, one can uniquely determine the background Lorentzian metric up to isometries in a region where the gravitational perturbations can travel to and return. We apply the result to two concrete examples when the source fields are scalar fields (i.e., Einstein-scalar field equations) and electromagnetic fields (i.e., Einstein-Maxwell equations).