Short-depth algorithms are crucial for reducing computational error on near-term quantum computers, for which decoherence and gate infidelity remain important issues. Here we present a machine-learning approach for discovering such algorithms. We apply our method to a ubiquitous primitive: computing the overlap rs ( ) Tr between two quantum states ρ and σ. The standard algorithm for this task, known as the Swap Test, is used in many applications such as quantum support vector machines, and, when specialized to ρ=σ, quantifies the Renyi entanglement. Here, we find algorithms that have shorter depths than the Swap Test, including one that has a constant depth (independent of problem size). Furthermore, we apply our approach to the hardware-specific connectivity and gate sets used by Rigetti's and IBM's quantum computers and demonstrate that the shorter algorithms that we derive significantly reduce the error-compared to the Swap Test-on these computers.