We develop a Bayesian nonparametric framework to analyze single molecule FRET (smFRET) data. This framework, a variation on infinite hidden Markov models, goes beyond traditional hidden Markov analysis, which already treats photon shot noise, in three critical ways: (1) it learns the number of molecular states present in a smFRET time trace (a hallmark of nonparametric approaches), (2) it accounts, simultaneously and self-consistently, for photo-physical features of donor and acceptor fluorophores (blinking kinetics, spectral cross-talk, detector quantum efficiency), and (3) it treats background photons. Point 2 is essential in reducing the tendency of nonparametric approaches to overinterpret noisy single molecule time traces and so to estimate states and transition kinetics robust to photophysical artifacts. As a result, with the proposed framework, we obtain accurate estimates of single molecule properties even when the supplied traces are excessively noisy, subject to photoartifacts, and of short duration. We validate our method using synthetic data sets and demonstrate its applicability to real data sets from single molecule experiments on Holliday junctions labeled with conventional fluorescent dyes.