We present a powerful and versatile technique that enables exquisite spatial and temporal control over local solution chemistry in microfluidic devices. Using a microscope and a UV lamp, we use projection lithography to photopolymerize thin (10-25 m) hydrogel membrane ''microwindows'' (HMMs) into standard microfluidic devices. These microwindows are permeable to solute and solvent diffusion and to electric fields, yet act as rigid walls from the standpoint of fluid flow. Reservoirs of solution may thus be rapidly imposed, switched, and maintained on one side of a HMM using standard microfluidic techniques, provoking changes in solution conditions on the other side without active mixing, stirring, or diluting. We highlight three paradigmatic experimental capabilities enabled by HMMs: (1) rapid dialysis and swapping of solute and/or solvent, (2) stable and convection-free localized concentration gradients, and (3) local electric permeability. The functional versatility of hydrogel microwindow membranes, coupled with the ease and speed of their fabrication and integration into simple microchannels or multilayer devices, will open a variety of novel applications and studies in a broad range of fields.