Motivated by comatrix coalgebras, we introduce the concept of a Newtonian comatrix coalgebra. We construct an infinitesimal unitary bialgebra on matrix algebras, via the construction of a suitable coproduct. As a consequence, a Newtonian comatrix coalgebra is established. Furthermore, an infinitesimal unitary Hopf algebra, under the view of Aguiar, is constructed on matrix algebras. By the close relationship between pre-Lie algebras and infinitesimal unitary bialgebras, we erect a pre-Lie algebra and a new Lie algebra on matrix algebras. Finally, a weighted infinitesimal unitary bialgebra on non-commutative polynomial algebras is also given.