In this work, pyridine‐based low‐molecular‐weight polybenzimidazole (LMP) was synthesized (inherent viscosity: 0.52 dL g−1) by controlling the reaction time. The synthesized LMP was incorporated into polydimethylsiloxane (PDMS) matrix to impart proton conductivity. The composite membrane containing 30 wt % of LMP exhibited proton conductivity of 16 mS cm−1 at 100–120 °C. The LMP chains were bundled and formed clusters (aggregates) in the PDMS matrix as observed in field emission scanning electron microscopy. This is attributed to nonpolar (siloxane)–polar (polybenzimidazole) repulsion. An impressive weight loss of 8.6% was observed after 120 h Fenton's test that indicates the high oxidative stability of composite membranes. However, elongation of composite membranes was decreased compared to that of pristine PDMS, which is attributed to the incorporation of rigid LMP chains. The resultant composite membranes exhibited moderate proton conductivity, low moisture absorption, and good thermal stability. © 2019 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2019, 136, 48151.