Algebraic Stream Processing

Abstract

We identify and analyse the typically higher-order approaches to stream processing in the literature. From this analysis we motivate an alternative approach to the specification of SPSs as STs based on an essentially first-order equational representation. This technique is called Cartesian form specification. More specifically, while STs are properly second-order objects we show that using Cartesian forms, the second-order models needed to formalise STs are so weak that we may use and develop well-understood first-order methods from computability theory and mathematical logic to reason about their properties. Indeed, we show that by specifying STs equationally in Cartesian form as primitive recursive functions we have the basis of a new, general purpose and mathematically sound theory of stream processing that emphasises the formal specification and formal verification of STs. The main topics that we address in the development of this theory are as follows. We present a theoretically well-founded general purpose stream processing language ASTRAL (Algebraic Stream TRAnsformer Language) that supports the use of modular specification techniques for full second-order STs. We show how ASTRAL specifications can be given a Cartesian form semantics using the language PREQ that is an equational characterisation of the primitive recursive functions. In more detail, we show that by compiling ASTRAL specifications into an equivalent Cartesian form in PREQ we can use first-order equational logic with induction as a logical calculus to reason about STs. In particular, using this calculus we identify a syntactic class of correctness statements for which the verification of ASTRAL programmes is decidable relative to this calculus. We define an effective algorithm based on term re-writing techniques to implement this calculus and hence to automatically verify a very broad class of STs including conventional hardware devices. Finally, we analyse the properties of this abstract algorithm as a proof assistant and discuss various techniques that have been adopted to develop software tools based on this algorithm.