The present work investigates the direct mixing of aqueous zeolitic imidazolate framework-8 (ZIF-8) suspension into a polyvinyl alcohol (PVA) and crosslinked with glutaraldehyde (GA) to form swelling-resistant, mechanically robust and conductivity retentive composite membranes. This drying-free nanofiller incorporation method enhances the homogeneous ZIF-8 distributions in the PVA/ZIF-8/GA composites to overcome the nanofiller aggregation problem in the mixed matrix membranes. Various ZIF-8 concentrations (25.4, 40.5 and 45.4 wt.%) are used to study the suitability of the resulting GA-crosslinked composites for direct alkaline methanol fuel cell (DAMFC). Surface morphological analysis confirmed homogeneous ZIF-8 particle distribution in the GA-crosslinked composites with a defect- and crack-free structure. The increased ionic conductivity (21% higher than the ZIF-free base material) and suppressed alcohol permeability (94% lower from the base material) of PVA/40.5%ZIF-8/GA resulted in the highest selectivity among the prepared composites. In addition, the GA-crosslinked composites’ selectivity increased to 1.5–2 times that of those without crosslink. Moreover, the ZIF-8 nanofillers improved the mechanical strength and alkaline stability of the composites. This was due to the negligible volume swelling ratio (<1.4%) of high (>40%) ZIF-8-loaded composites. After 168 h of alkaline treatment, the PVA/40.5%ZIF-8/GA composite had almost negligible ionic conductivity loss (0.19%) compared with the initial material. The maximum power density (Pmax) of PVA/40.5%ZIF-8/GA composite was 190.5 mW cm−2 at 60 °C, an increase of 181% from the PVA/GA membrane. Moreover, the Pmax of PVA/40.5%ZIF-8/GA was 10% higher than that without GA crosslinking. These swelling-resistant and stable solid electrolytes are promising in alkaline fuel cell applications.