Deep Specifications and Certified Abstraction Layers

Abstract

Modern computer systems consist of a multitude of abstraction layers (e.g., OS kernels, hypervisors, device drivers, network protocols), each of which defines an interface that hides the implementation details of a particular set of functionality. Client programs built on top of each layer can be understood solely based on the interface, independent of the layer implementation. Despite their obvious importance, abstraction layers have mostly been treated as a system concept; they have almost never been formally specified or verified. This makes it difficult to establish strong correctness properties, and to scale program verification across multiple layers. In this paper, we present a novel language-based account of abstraction layers and show that they correspond to a strong form of abstraction over a particularly rich class of specifications which we call deep specifications. Just as data abstraction in typed functional languages leads to the important representation independence property, abstraction over deep specification is characterized by an important implementation independence property: any two implementations of the same deep specification must have contextually equivalent behaviors. We present a new layer calculus showing how to formally specify, program, verify, and compose abstraction layers. We show how to instantiate the layer calculus in realistic programming languages such as C and assembly, and how to adapt the CompCert verified compiler to compile certified C layers such that they can be linked with assembly layers. Using these new languages and tools, we have successfully developed multiple certified OS kernels in the Coq proof assistant, the most realistic of which consists of 37 abstraction layers, took less than one person year to develop, and can boot a version of Linux as a guest.

The HACMS program: using formal methods to eliminate exploitable bugs

“…Appropriate candidates include separation kernels, hypervisors, real-time operating systems (RTOSes), compilers, file systems, web broswers, sandboxes, cryptographic algorithms and garbage collectors. Perhaps surprisingly, the literature already includes artefacts that have been verified in all of these categories, albeit with somewhat limited functionality: the seL4 microkernel [ 33 ], the mCertiKOS hypervisor [ 35 ], the eChronos and ORIENTIAS RTOSes [ 27 , 36 ], the CompCert C Compiler [ 37 ], the FSCQ and BilbyFS file systems [ 38 , 39 ], the Quark web broswer [ 40 ], the RockSalt browser sandbox [ 41 ], various crytographic algorithms [ 42 , 43 ] and the Nucleus garbage collector [ 44 ]. The existence of two of these artefacts were crucial to DARPA’s decision to fund the HACMS program, serving as a basis of confidence that the program had some chance of succeeding: the seL4 microkernel and the CompCert verifying C compiler.…”

“… artefact size dev. time performance seL4 [ 33 ] 10K LoC, 480K LoP 13 PY 206 versus 227 cycles CompCert [ 46 ] 42K LoC+P 3 PY 2× speed of 7% slower than 12% slower than FSCQ File System [ 38 ] 24K LoC+P <5 PY 80% of xv6 file system certiKOS Hypervisor [ 35 ] 2K LoC, 18.5K LoP 1 PY <2× slowdown on most benchmarks SHA-256/HMAC [ 43 ] 407 LoC, 14.4K LoP n.a. equivalent to OpenSSL 0.9.1c Rocksalt Sandbox [ 41 ] 100 LoC, 10K LoP <2 PY 1M instructions per second faster than Google’s checker Nucleus GC [ 44 ] 6K LoC+P 0.75 PY ‘competitive’ Quark Web Browser [ 40 ] 5.5K LoC+P 0.5 PY 24% overhead w.r.t.…”

Formal design and verification of system task in intelligent transportation systems based on micro-kernel architecture

“…Flint created a new logic programming language known as VeriML Stampoulis ( 2012) with a strong formal description and safety, and studied verification methods of programs and systems in a concurrent environment Wang et al (2019); Gu et al (2018;. Flint used separation logic to verify the accuracy of memory sharing and data abstraction (Koenig and Shao 2018;Gu et al 2015). Meanwhile, virtual timeline was used to specify and reason for preemptive scheduling, and a novel compositional framework is proposed for reasoning Liu et al (2020).…”

CompCertELF: verified separate compilation of C programs into ELF object files

“…Various techniques for more general compositional compilation in CompCert have been proposed. Notable examples include Compositional CompCert Stewart 2015], CompCertX [Gu et al 2015], CASCompCert [Jiang et al 2019] and CompCertM [Song et al 2020]. All these extensions compile only to CompCert's assembly language.…”