Entanglement harvesting from the quantum field is a well-known fact that, in recent times, is being rigorously investigated further in flat and different curved backgrounds. The usually understood formulation studies the possibility of two uncorrelated Unruh-DeWitt detectors getting entangled over time due to the effects of quantum vacuum fluctuations. Our current work presents a thorough formulation to realize the entanglement harvesting from non-vacuum background fluctuations. In particular, we further consider single excitation field states and a pair of inertial detectors, respectively, in (1 + 1) and (1 + 3) dimensions for this investigation. Our main observation asserts that entanglement harvesting is suppressed compared to the vacuum fluctuations in this situation. Our other observations confirm a non-zero individual detector transition probability in this background and vanishing entanglement harvesting for parallel co-moving detectors. We look into the characteristics of the harvested entanglement and discuss its dependence on different system parameters.