A first-principles approach to the unitarity problem for black holes is systematically explored, based on the postulates of 1) quantum mechanics 2) the ability to approximately locally divide quantum gravitational systems into subsystems 3) correspondence with quantum field theory predictions for appropriate observers and (optionally) 4) universality of new gravitational effects. Unitarity requires interactions between the internal state of a black hole and its surroundings that have not been identified in the field theory description; correspondence with field theory indicates that these are soft. A conjectured information-theoretic result for information transfer between subsystems, partly motivated by a perturbative argument, then constrains the minimum coupling size of these interactions of the quantum atmosphere of a black hole. While large couplings are potentially astronomically observable, given this conjecture one finds that the new couplings can be exponentially small in the black hole entropy, yet achieve the information transfer rate needed for unitarization, due to the large number of black hole internal states. This provides a new possible alternative to arguments for large effects near the horizon. If universality is assumed, these couplings can be described as small, soft, state-dependent fluctuations of the metric near the black hole. Open questions include that of the more fundamental basis for such an effective picture.