Shadow attacks: automatically evading system-call-behavior based malware detection Academic Article uri icon

abstract

  • Contemporary malware makes extensive use of different techniques such as packing, code obfuscation, polymorphism, and metamorphism, to evade signature-based detection. Traditional signature-based detection technique is hard to catch up with latest malware or unknown malware. Behavior-based detection models are being investigated as a new methodology to defeat malware. This kind of approaches typically relies on system call sequences/graphs to model a malicious specification/pattern. In this paper, we present a new class of attacks, namely "shadow attacks", to evade current behavior-based malware detectors by partitioning one piece of malware into multiple "shadow processes". None of the shadow processes contains a recognizable malicious behavior specification known to single-process-based malware detectors, yet those shadow processes as an ensemble can still fulfill the original malicious functionality. To demonstrate the feasibility of this attack, we have developed a compiler-level prototype tool, AutoShadow, to automatically generate shadow-process version of malware given the source code of original malware. Our preliminary result has demonstrated the effectiveness of shadow attacks in evading several behavior-based malware analysis/detection solutions in real world. With the increasing adoption of multi-core computers and multi-process programs, malware writers may exploit more such shadow attacks in the future. We hope our preliminary study can foster more discussion and research to improve current generation of behavior-based malware detectors to address this great potential threat before it becomes a security problem of the epidemic proportions. 2011 Springer-Verlag France.

published proceedings

  • Journal in Computer Virology

author list (cited authors)

  • Ma, W., Duan, P. u., Liu, S., Gu, G., & Liu, J.

citation count

  • 54

complete list of authors

  • Ma, Weiqin||Duan, Pu||Liu, Sanmin||Gu, Guofei||Liu, Jyh-Charn

publication date

  • May 2012