Nanoscale devices, either biological or artificial, operate in a regime where the usual assumptions of a structureless Markovian bath do not hold. Being able to predict and study the dynamics of such systems is crucial and is usually done by tracing out the bath degrees of freedom, which implies losing information about the environment. To go beyond these approaches we use a numerically exact method relying on a matrix product state representation of the quantum state of a system and its environment to keep track of the bath explicitly. This method is applied to a specific example of interaction that depends on the spatial structure of a system made of two sites. The result is that we predict a non-Markovian dynamics where long-range couplings induce correlations into the environment. The environment dynamics can be naturally extracted from our method and shine a light on long-time feedback effects that are responsible for the observed non-Markovian recurrences in the eigenpopulations of the system.