Bobby Birrer

Metamorphic Program Fragmentation As a Software Protection Bobby Birrer

Publisher Marketing: Unauthorized reverse-engineering of programs and algorithms is a major problem for the software industry. Every program released to the public can be analyzed by any number of malicious reverse-engineers. These reversers search for security holes in the program to exploit or try to steal a competitor's vital algorithms. While it can take years and millions of dollars worth of research to develop new software, a determined reverser can reverse-engineer the program in a fraction of the time. To discourage reverse-engineering attempts, developers use a variety of software prote tions to obfuscate their programs. However, these protections are generally static, allowing reverse-engineers to eventually adapt to the protections, defeat them, and sometimes build automated tools to defeat them in the future. Metamorphic software protections add another layer of protection to traditional static obfuscation techniques. Metamorphic protections force a reverser to adjust their attacks as the protection changes. Program fragmentation combines two obfuscation techniques, outlining and obfuscated jump tables, into a new, metamorphic protection. Sections of code are removed from the main program flow and randomly placed throughout memory, reducing the programs locality. These fragments move while the program is running and are called using obfuscated jump tables, making program execution difficult to follow. This research assesses the performance overhead of a program fragmentation metamorphic engine and provides a qualitative analysis of its effectiveness against reverse-engineering techniques. Program fragmentation has very little associated overhead, with execution times for individual fragments of less than one microsecond. This low overhead allow a large numbers of fragments to be inserted into programs for protection. In addition, program fragmentation is an effective technique to complicate program of programs using two common disassembler/debugger tools. Thus, program fra