Formally verified big step semantics out of x86-64 binaries

Roessle, Ian; Verbeek, Freek; Ravindran, Binoy

doi:10.1145/3293880.3294102

Cited by 8 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The K framework also enables one to represent our semantics as SMT theories, which allows others to easily reuse our semantics for their own purposes. Indeed, we have translated our semantics to Stoke [111] which can serve as a drop-in replacement of Heule et al's semantics [87] and can immediately benefit tools built on Stoke (e.g., [112]).…”

Section: Usablementioning

confidence: 99%

“…A contemporary work by Roessle et al [112] presents a method to extract the big step semantics of a binary program using the small step instruction semantics extracted mostly from Strata 1 plus some manually drafted support for branching instructions and stack operations. Like Strata, their specification is executable only for the non-floating-point instructions.…”

Section: Defining Formal Semantics Of X86-64mentioning

confidence: 99%

See 1 more Smart Citation

Scalable validation of binary lifters

Dasgupta

Dinesh

Venkatesh

et al. 2020

Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

The ability to directly reason about binary machine code is desirable, not only because it allows analyzing binaries even when the source code is not available (e.g., legacy code, closed-source software, or malware), but also avoids the need to trust the correctness of compilers. Binary analysis is generally performed by existing decompiler projects by (1) converting raw bytes from the binary into a stream of assembly instructions through disassembly, (2) translating machine code to an intermediate representation (IR) using a binary lifter, and (3) performing various analysis and transformations on the IR pertaining to the specific goals of the decompiler. Many binary analysis frameworks published in academia or as open-source code, use such a lifter as the first step in their pipeline. Validating the correctness of binary lifters is pivotal to gain trust in binary analysis, especially when used in scenarios where correctness is essential. Unfortunately, existing approaches focus on validating the correctness of lifting a single instruction and do not scale to full programs. I believe an effort in the direction would enable both the developers of binary translators, to validate their implementation, and the clients of those translators, to gain trust in their analysis results. The overall goal of my work is to develop formal and informal techniques to achieve high confidence in the correctness of binary lifting by leveraging the semantics of the languages involved (e.g., Intel's x86-64 and LLVM IR). Towards that goal, I made two broad contributions. First, I defined the most complete and thoroughly tested formal semantics of x86-64 to date. The semantics faithfully formalizes all the non-deprecated, sequential user-level instructions of the x86-64 Haswell instruction set architecture. The formal specification covers 3155 instruction variants, corresponding to 774 mnemonics. The semantics is fully executable and has been tested against the GCC C-torture test suite. Moreover, each instruction is individually tested against more than 7,000 instruction-level test cases. This extensive testing paid off, revealing bugs in both the x86-64 reference manual and other existing semantics, which are all acknowledged, and some are fixed. Also, I illustrated potential applications of the semantics in different formal analyses, and discuss how it can be useful for processor verification. Second, I show that formal translation validation of single instructions for a complex ISA like x86-64 is not only practical but can be used as a building block for scalable full-program validation. My work is the first to do translation validation of single instructions on an Coming up to this point is one of the biggest challenges I have ever pursued in my life. This thesis would not have been possible if I did not have the support of the following people. I owe my deepest gratitude to my adviser, Professor Vikram Adve, whose guidance, patience, and encouragement have been pivotal in the successful completion of this thesis. He is one of the ...

show abstract

Section: Usablementioning

confidence: 99%

Section: Defining Formal Semantics Of X86-64mentioning

confidence: 99%

Scalable validation of binary lifters

Dasgupta

Dinesh

Venkatesh

et al. 2020

Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

show abstract

“…This produced highly reliable semantics: they compared the semantics to manually written semantics based on the Intel reference manuals, and found that in the few cases where they differed the Intel manuals were wrong. Roessle et al embedded these semantics into the Isabelle/HOL theorem prover and tested the formal Isabelle semantics against live x86 hardware [49]. This formal machine model is the base of our verification effort.…”

Section: Verification Tools Usedmentioning

confidence: 99%

Highly Automated Formal Proofs over Memory Usage of Assembly Code

Verbeek

Bockenek

Ravindran

2020

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

We present a methodology for generating a characterization of the memory used by an assembly program, as well as a formal proof that the assembly is bounded to the generated memory regions. A formal proof of memory usage is required for compositional reasoning over assembly programs. Moreover, it can be used to prove low-level security properties, such as integrity of the return address of a function. Our verification method is based on interactive theorem proving, but provides automation by generating pre- and postconditions, invariants, control-flow, and assumptions on memory layout. As a case study, three binaries of the Xen hypervisor are disassembled. These binaries are the result of a complex build-chain compiling production code, and contain various complex and nested loops, large and compound data structures, and functions with over 100 basic blocks. The methodology has been successfully applied to 251 functions, covering 12,252 assembly instructions.

show abstract

“…The machine model must provide a step function that provides semantics for instructions. We have used the machine model of Roessle et al [26], which is built upon the work of Heule et al [9]. Heule et al used machine learning to derive semantics by executing instructions on an actual x86-64 machine.…”

Section: Case Study: Hermitcorementioning

confidence: 99%

“…Figure 1 shows an overview of the contribution's methodology. The approach disassembles a binary using an off-the-shelf disassembler, performs analysis on the binary to extract data for automation, and embeds it into a theorem prover using the symbolic execution toolchain of Roessle et al [26], the machine model of which is based on the work of Heule et al [9]. Within the theorem prover, two things need to be manually defined: an invariant and the set of regions that the function is allowed to write to.…”

Section: Introductionmentioning

confidence: 99%

Formal Verification of Memory Preservation of x86-64 Binaries

Bockenek

Verbeek

Lammich

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Formal verification of a binary can provide high software assurance, even when the source code is unavailable. It is, however, inherently hard due to the low level of abstraction involved; instead of verifying typed and structured source code, one has to verify machine code or reconstructed assembly. This paper presents a semi-automated methodology for formally verifying memory preservation, as well as register preservation, over disassembled binaries. The methodology is based on formal symbolic execution and Floyd-style verification. We show that the methodology is compositional on the function level, which is crucial for scalability. The methodology works for loops, recursion, and both optimized and non-optimized code. It can be used to expose preconditions required for non-exceptional behavior. We demonstrate applicability by verifying a set of functions from the HermitCore unikernel library.

show abstract

Formally verified big step semantics out of x86-64 binaries

Cited by 8 publications

References 22 publications

Scalable validation of binary lifters

Scalable validation of binary lifters

Highly Automated Formal Proofs over Memory Usage of Assembly Code

Formal Verification of Memory Preservation of x86-64 Binaries

Contact Info

Product

Resources

About