We present a unified conceptual framework and associated software package for single molecule Förster Resonance Energy Transfer (smFRET) analysis from single photon arrivals leveraging Bayesian nonparametrics, BNP-FRET. This unified framework addresses the following key physical complexities of an smFRET experiment, including: 1) fluorophore photophysics; 2) continuous time dynamics of the labeled system with large timescale separations between photophysical phenomena such as excited photophysical state lifetimes and events such as transition between system states; 3) unavoidable detector artefacts; 4) background emissions; 5) unknown number of system states; and 6) both continuous and pulsed illumination. These physical features necessarily demand a novel framework that extends beyond existing tools. In particular, we propose a second order hidden Markov model (HMM) and Bayesian nonparametrics (BNP) on account of items 1, 2 and 5 on the list, respectively. In companion manuscripts II and III, we discuss the direct effects of these key complexities on the inference of parameters for continuous and pulsed illumination, respectively.