Considering the variations in their pore characteristics and heterogeneity, the carbonization process of carboxylate‐functionalized polymers of intrinsic microporosity (PIM‐COOH) is investigated and the resulted membranes are tested for water treatment. The four distinct types of sub‐nanosized membranes with different oxygen‐to‐carbon ratios are prepared at four different temperatures: the pristine polymer membrane, crosslinked membrane via thermal decarboxylation, amorphous carbon membrane, and graphitic carbon membrane. Notably, the sub‐1 nm micropores of all the heat‐treated samples that do not exhibit a broadening pore size distribution are still retained. Nanofiltration performance is investigated using an aqueous MgSO4 solution (2000 ppm), pure water, and a series of the defect‐free, sub‐1 nm microporous membranes. While retaining the salt rejection, the water flux of the membranes increases with the pyrolysis temperature owing to their low friction property.