The nucleosome remodeling deacetylase (NuRD) complex is a highly conserved regulator of chromatin structure and transcription. Structural studies have shed light on this and other chromatin modifying machines, but much less is known about how they assemble and whether stable and functional sub-modules exist that retain enzymatic activity. Purification of the endogenous Drosophila NuRD complex shows that it consists of a stable core of subunits, while others, in particular the chromatin remodeler CHD4, associate transiently. To dissect the assembly and activity of NuRD, we systematically produced all possible combinations of different components using the MultiBac system, and determined their activity and biophysical properties. We carried out single-molecule imaging of CHD4 in live mouse embryonic stem cells, in the presence and absence of one of core components (MBD3), to show how the core deacetylase and chromatin-remodeling sub-modules associate in vivo. Our experiments suggest a pathway for the assembly of NuRD via preformed and active sub-modules. These retain enzymatic activity and are present in both the nucleus and the cytosol, an outcome with important implications for understanding NuRD function.