Algorithms and architectures are described for optical matrix-matrix and matrix-vector processing. Implementations are based on quadratic residue number system coding which has the remarkable property that computations in the real and imaginary channels are performed in parallel without any interactions. This approach significantly reduces the complexity of implementation. Residue-based processing provides increased dynamic range and a reduced number of computations with high speed and parallelism of the optics fully utilized. Several architectures of optical processors described in this paper include systolic and data flow arrays.