This paper studies a four-dimensional (4D) memristive system modified from the 3D chaotic system proposed by Lü and Chen. The new system keeps the symmetry and dissipativity of the original system and has an uncountable infinite number of stable and unstable equilibria. By varying the strength of the memristor, we find rich complex dynamics, such as limit cycles, torus, chaos, and hyperchaos, which can peacefully coexist with the stable equilibria. To explain such coexistence, we compute the unstable manifolds of the equilibria, find that the manifolds create a safe zone for the hyperchaotic attractor, and also find many heteroclinic orbits. To verify the existence of hyperchaos in the 4D memristive circuit, we carry out a computerassisted proof via a topological horseshoe with twodirectional expansions, as well as a circuit experiment on oscilloscope views.