We determine the total state dynamics of a dephasing open quantum system using the standard environment of harmonic oscillators. Of particular interest are random unitary approaches to the same reduced dynamics and system-environment correlations in the full model. Concentrating on a model with an at times negative dephasing rate, the issue of "non-Markovianity" will also be addressed. Crucially, given the quantum environment, the appearance of non-Markovian dynamics turns out to be accompanied by a loss of system-environment correlations. Depending on the initial purity of the qubit state, these system-environment correlations may be purely classical over the whole relevant time scale, or there may be intervals of genuine system-environment entanglement. In the latter case, we see no obvious relation between the build-up or decay of these quantum correlations and "Non-Markovianity".