To exploit tapered parallel plate waveguides for broadband terahertz (THz) spectroscopy, the impact of the waveguide geometry on transmission of terahertz pulses is investigated experimentally. We find that the approximation of single transverse electro-magnetic mode propagation is insufficient for describing the observed behavior. The TE02 mode plays a particularly important role. The mode composition, however, can be controlled by the gap between the waveguide plates, which affects the main loss mechanism, radiation leakage, and group velocity for the TE02 mode. Balancing the waveguide loss and coupling efficiencies results in an optimal gap for the tapered waveguide.