Enhancing Branch Monitoring for Security Purposes: From Control Flow Integrity to Malware Analysis and Debugging Academic Article uri icon


  • Malware and code-reuse attacks are the most significant threats to current systems operation. Solutions developed to countermeasure them have their weaknesses exploited by attackers through sandbox evasion and antidebug crafting. To address such weaknesses, we propose a framework that relies on the modern processors branch monitor feature to allow us to analyze malware while reducing evasion effects. The use of hardware assistance aids in increasing stealthiness, a key feature for debuggers, as modern software (malicious or benign) may be antianalysis armored. We achieve stealthier code execution control by using the branch monitor hardwares inherent interrupt capabilities, keeping the code under execution intact. Previous works on branch monitoring have already addressed the ROP attack problem but require code injection and/or are limited in their capture window size. Therefore, we also propose a ROP detector without these limitations.

published proceedings


altmetric score

  • 1

author list (cited authors)

  • Botacin, M., De Geus, P. L., & Gregio, A.

citation count

  • 10

complete list of authors

  • Botacin, Marcus||De Geus, Paulo Licio||Gregio, Andre

publication date

  • February 2018