A New Approach to Instruction-Idioms Detection in a Retargetable Decompiler

Jakub Křoustek1, Fridolín Pokorný1 and Dušan Kolář1

  1. Faculty of Information Technology, IT4Innovations Centre of Excellence
    Brno University of Technology, Bozetechova 1/2, 612 66 Brno, Czech Republic
    ikroustek@fit.vutbr.cz, xpokor32@stud.fit.vutbr.cz, kolar@fit.vutbr.cz

Abstract

Retargetable executable-code decompilation is a one of the most complicated reverse-engineering tasks. Among others, it involves de-optimization of compiler-optimized code. One type of such an optimization is usage of so-called instruction idioms. These idioms are used to produce faster or even smaller executable files. On the other hand, decompilation of instruction idioms without any advanced analysis produces almost unreadable high-level language code that may confuse the user of the decompiler. In this paper, we revisit and extend the previous approach of instruction-idioms detection used in a retargetable decompiler developed within the Lissom project. The previous approach was based on detection of instruction idioms in a very-early phase of decompilation (a front-end part) and it was inaccurate for architectures with a complex instruction set (e.g. Intel x86). The novel approach is based on delaying detection of idioms and reconstruction of code to the later phase (a middleend part). For this purpose, we use the LLVM optimizer and we implement this analysis as a new pass in this tool. According to experimental results, this new approach significantly outperforms the previous approach as well as the other commercial solutions.

Key words

compiler optimizations, reverse engineering, decompiler, Lissom, instruction idioms, LLVM, LLVM IR

Digital Object Identifier (DOI)

https://doi.org/10.2298/CSIS131203076K

Publication information

Volume 11, Issue 4 (October 2014)
Special Issue on Advances in Systems, Modeling, Languages and Agents
Year of Publication: 2014
ISSN: 2406-1018 (Online)
Publisher: ComSIS Consortium

Full text

DownloadAvailable in PDF
Portable Document Format

How to cite

Křoustek, J., Pokorný, F., Kolář, D.: A New Approach to Instruction-Idioms Detection in a Retargetable Decompiler. Computer Science and Information Systems, Vol. 11, No. 4, 1337–1359. (2014), https://doi.org/10.2298/CSIS131203076K