Toward an Approach on Probability Distribution for Polymorphic Malware Analysis

Abstract

Nowadays, computer security is a serious issue
which attracts the interest from many nations. To identify
malware, most of industry approaches still center the well-known
technique of signature matching. However, modern polymorphic
malwares use packer to obfuscate their malicious actions. A
sophisticated packer can generate virtually variants of a viral
code, making the signature-based technique easily defeated.
Naturally, applying stochastic approach prompts a potential
solution to handle polymorphic virus. This paper studies an
approach of applying probability distribution for tackling the
two important problems in analyzing polymorphic malware,
which are to identify a potential malware and to detect packer
which malware adopts. For the first goal, we derive a new
frequency-based weight to identify most specific instructions for
each malware family, known as instruction frequency-inverse
malware frequency (􀀁􀀂°􀀁􀀄􀀂) . For the second one, we propose a
new term, obfuscation technique frequency-inverse packer
frequency (􀀅􀀆􀀂°􀀁􀀇􀀂) for evaluating the importance of obfuscation
techniques in packers. We have performed the experiment on
4194 real malware and the result is very promising.