May 16, 2006

Moving to automated behavioral classification for malware detection

These days I am jealous of some of the neat security research being done over on the Microsoft campus. Although I don't have any plans to ever work there, there are a few teams that are doing some interesting stuff I wish I could be part of... if nothing more but to learn.

Today I saw a great blog post by Tony Lee, a virus researcher on the Microsoft Antimalware team. Apparently one of the growing challenges Microsoft is facing today is the large number of active malware samples, totaling in the order of tens of thousands, and increasing rapidly that his team has to sift through. The traditional manual analysis process they are using is not adequate in dealing with malware of this order of magnitude, and they are thinking about new ways to automate this process.

Tony and Jigar J. Mody (a colleague on the anti-malware team) wrote and presented a paper on "Behavioral Classification" in which they dive into tackling the challenge of the classifcation process. I was quite engaged as I read through it, as they have approached it in an interesting manner.

When it comes to this sort of analysis, in the past it has been perceived that human heuristics will always trump automated methods because the attack vectors can easily change. However, what Microsoft is seeing is that the classification can be weighed in such a manner that by using runtime events and machine learning, classification is indeed possible with automation. Some of their initial research shows up to 84% success rate. Of course, what wasn't detailed is if the failure rate is in misclassifcation, or if they are not being able to classify it at all.

It will be interesting to see where they go with this. As more hostile code is created, developing an automated classification process that can leverage behaviorial learning capabilities will be critical for malware intelligence and defense.

Interesting stuff. You should check out their paper on the topic.

Happy reading!

Posted by SilverStr at May 16, 2006 01:34 PM | TrackBack

Semi-related thought:

I think perhaps we are past the days of signature based malware defenses. My suspiscion is that the number of executable malware variants (I include spyware/adware/virus/trojan/rootkit and so on in this) may by now actually exceed the number of goodware available! I could be wrong in this, but ...

The essential strategy of signature based malware defenses is: I have a List of Disallowed Things. I must regularly scan everything, and if I spot a Disallowed Thing, I then take some remediating action.

This is exactly opposite the best practice in security strategy, which is essentially: I have a List of Allowed Things. I sit in front of Protected Resource, and I only let Allowed Things access Protected Resource.

I've recently started using the free/beta version of Prevx, which employs the second strategy. I had tried Prevx a year or two back, and it wasn't very good at the time, but they've made some great strides since then. I highly recommend it (and LUA)!

Posted by: Bryan at May 16, 2006 06:56 PM