We propose and investigate theoretically the use of bi-directional propagation of light in silica integrated coupled waveguide structures for linear optical processing of fiber-coupled photons. We show that the class of linear operations that can be implemented in such a system is given by symmetric and unitary matrices. We present how an arbitrary 4×4 coupler of this form can be realized by a linear sequence of more fundamental 2 × 2 couplers and single-waveguide phase shifters and discuss in detail the implementation in a silica integrated platform utilizing direct UV-written waveguides and long-and short-period grating couplers with tilted gratings for optimized coupling efficiency.