In the present work, a pair of general inverse matrix series relations is established, and thereby a general class of matrix polynomials is introduced. This class generalizes the extended Jacobi polynomials and their particular cases such as the polynomials of Brafman, Jacobi, Chebyshev, and Legendre. It is further shown that this pair also gives rise to the matrix forms of the Wilson polynomials and the Racah polynomials. For these polynomials, the generating matrix function relations as well as the matrix summation formulas are deduced from their respective inverse pairs. Certain inverse pairs belonging to the Gould classes and the Legendre-Chebyshev classes due to John Riordan [An Introduction to Combinatorial Identities, Wiley, 1968] are also extended to matrix forms.