We analyze the algebra of boundary observables in canonically quantised JT gravity with or without matter. In the absence of matter, this algebra is commutative, generated by the ADM Hamiltonian. After coupling to a bulk quantum field theory, it becomes a highly noncommutative algebra of Type II ∞ with a trivial center. As a result, density matrices and entropies on the boundary algebra are uniquely defined up to, respectively, a rescaling or shift. We show that this algebraic definition of entropy agrees with the usual replica trick definition computed using Euclidean path integrals. Unlike in previous arguments that focused on O(1) fluctuations to a black hole of specified mass, this Type II ∞ algebra describes states at all temperatures or energies. We also consider the role of spacetime wormholes. One can try to define operators associated with wormholes that commute with the boundary algebra, but this fails in an instructive way. In a regulated version of the theory, wormholes and topology change can be incorporated perturbatively. The bulk Hilbert space H bulk that includes baby universe states is then much bigger than the space of states H bdry accessible to a boundary observer. However, to a boundary observer, every pure or mixed state on H bulk is equivalent to some pure state in H bdry .