NMR data are collected as time- and ensemble-averaged quantities. Yet, in commonly used methods for structure determination of biomolecules, structures are required to satisfy simultaneously a large number of constrainsts. Recently, however, methods have been developed that allow a better fit of the experimental data by the use of time- or ensemble-averaged restraints. Thus far, these methods have been applied to structure refinement using distance and J-coupling restraints. In this paper, time and ensemble averaging is extended to the direct refinement with experimental NOE data. The implementation of time- and ensemble-averaged NOE restraints in DINOSAUR is described and illustrated with experimental NMR data for crambin, a 46-residue protein. Structure refinement with both time- and ensemble-averaged NOE restraints results in lower R-factors, indicating a better fit of the experimental NOE data.