To the admins: if you consider this post to be against the rules - please delete.

Hi to all the members of this board. I'm member of a team that does the testing of antivirus programs (the link to the site will be posted if that won't be considered as spam).

As we are doing a pretty big test of recognition of packers, we are trying to collect all the packers/protector/cryptors that can be found.
As for now, we have all that Google can provide, but there is a lot of stuff not available for download on the net.
Here I ask all of you who want to help in preparing this test to get me informed.

As for now, I'll ask the admins of this board to get the decision if this thread will remain, or it will be deleted. After that, we can talk further.

I apologize for my English

Hi,

Don't worry about it, there are no violations of our rules and AV is in the best interests of all. I can't think of a better raison d’être for RCE than anti-malware...

Regards,
Kayaker

Thanks

I'll try to make a list of known packers (protector, cryptors) in a form of checklist (what we have/don't have) ant to post it here.
I hope someone is able to provide some old or rare 'species' that I can't find.

The link to our site is: http://www.mc-antivirus-test.com
If someone want to take a role in the testing/preparing the tests - please let me know.

There are already websites that have performed tests with compressed malware (e.g., www.rokop-security.de). They might be able to help.

If you compress malware with a packer, crypter or protector for purposes of your test it is imperative that you execute the malware sample before you put it into your test archive. If you do not do this and create non-working samples the test results will be flawed.

Consequently, it is quite burdensome to create a compressed test archive.

In addition, you need to understand WHY a scanner detects a compressed malware sample in order to properly evaluate your test results. For example, certain scanners will have no problems to detect compressed malware samples simply because they use "insecure" signatures taken from the resource section (which frequently remains uncompressed).

Moreover, if you want to know whether a specific scanner can unpack a specific packer or not you can simply ask me. I know them all

Old stuff huh? There are some old com/exe packers here, as well as some new AV related files that might be of some use

Ralph Roth
http://come.to/rose_swe

You might find some other related material at this very somewhat idle mailing list..

http://groups.yahoo.com/group/exeml/
http://groups.yahoo.com/group/exeml/messages/1

The EXE-mailing-list is a mailing-list for EXE/COM - packers, protectors,
unpackers and detection-tools. All new stuff is send via this mailing-
mailing-list, e. g. HackStop, CrackStop, MESS, TEU, GTR, ChkEXE etc.

I also collect such stuff. But the main problem is not to collect a few 100 packers but to create & test the samples. You need at least 5 to 10 samples for each packer and you should exactly know about the specifics of each of your samples (e.g., does it use an embedded .dll, does it have a resource section and so on) in order to create a test archive that allows you to come to well-founded conclusions in respect of a scanner's unpacking engine.

I would estimate that it will take you a few hundred hours to create a reasonably sized test archive.

@N8di8
I know that I should test if the sample can still be executed, thats why I need help.
First thing is to get a virus-sample that is recognized by all of the programs on the test.
After that, I need to pack the sample, and to test it...

I would really apreciate if someone want to help me with this.

@Kayaker
Thanks for the links

========edit=======
@N8di8
Can you help me to find some samples that are good for being packed?
I know that Delphi programs are problematic in some cases, so I need either an ASM or C++ sample.
I dont know the situation for DOS programs. Can you give me some instructions there?

@Kayaker
No help from the EXE-mailin-list, all the links are dead, and the attachments in mesagess are not stored (missing)

bobby:

Do you consider it mandatory to use viruses for you test? Most viruses are already crypted which makes it difficult to compress/crypt them with a standard executable packer. By contrast, if you take a worm like Mydoom things are much easier. Possibly the best way to go is to use non-replicating malware (aka trojans). There are numerous trojans that can be easily compressed and that can be easily cleaned after execution (i.e., you do not always need to restart your computer after the execution of a malware sample). For instance, the following trojans can be easily compressed: AnalFTP, Bionet, Lithium, Optix Lite, Theef Lite, CIA and Beast. These trojans are detected by most AV/AT scanners. It shouldn't be too difficult for you to find the trojans via google.

@N8di8
No need to search for them. The test collection is compsed from about 90.000 malware-samples, and I have those for sure.

What about DOS, BAT, VBS, JS and others, what I need to take care about there?

Now I also have a question: where did you obtain the test collection from? Is it the vxheavens collection?

DOS,BAT,VBS,JS...forget it (for the time being). Start with normal Win PE files (.exe files).

vxheaven collection has <40000 samples. Yes, I did downloaded it.
I got the rest from my (and someone others) honeypot (about 10.000 samples)
I got 2 private collections from ex-VX-traders
One CD that was on sale in the mid '90 (~12.000 samples, mostly DOS-related)
The rest is from P2P (you can get more than 20.000 samples on P2P)

There was a lot of duplicates between those collections.

About packing, yes, I'll start with PE executables, but I also want to have other formats too.
I have almost finished the collection of archivers (some are realy exotic, so I still need to think if there is a sense to include them on test)
I also have a collection of Installers (Inno, NSIS and dosen of others) because many of them have a specific storage algorithm.

Apart of those scaning-tests, I have also done programs for the testing of real-time engines (access malware files and see the reaction of AV program).

Just one additional hint (unless you already know): in order to determine a scanners' detection rate you should not simply look at the scanners' log file. This is because a scanner frequently counts compressed malware twice. IMHO, the preferrable way is to let the scanner delete (a copy) of the entire test archive and then use an application like DirLister in order to create your own log (= undected files). If you then compare the log with the orginal test archive you can figure out the detected files.

I did saw that problem last year, during the May '05 testing.
Now I have different approach - there won't be any archive or any kind of container file (Installers etc) on the test of detection rate.

Deleting is not a good way of testing. I'll give some examples:
- if the file is in a kind of archive (say RAR), and AV has just the UnRAR module - the virus will detected, but not removed or deleted.
- if the malware is a virus (infector), the file won't be touch in a lot of cases
- just the plain malware files will be deleted foru sure (e.g. trojans)
- if a file contains multiple malware (say some Installer), and AV detects one of them, the it delete the whole file - we won't know if AV have detected all the files in container

I'm preparing the collectin in such way that there won't be such situations that the log file can give wrong info.

Other formats: there are certain html "crypters" (you would have to google). Same applies to other web formats. Old packers can be used to pack outdated DOS executables (although I do not understand the benefit of a test using DOS malware).

The use of commercial installers is highly relevant. They only wrap the file (i.e., the on-access scanner can still detect the sample after the installer has installed it and before the malware has been executed) but there are certain tricks that can be employed with the help of an installer. For example, the installer can firstly delete the autostart registry keys of the antivirus, install the trojan and then reboot the machine. Or it can write malicious exclusion lists or switch off the on-access scanner via the registry. (This depends of course on the individual scanner.)

" I have also done programs for the testing of real-time engines (access malware files and see the reaction of AV program)."

Could you please provide me with additional information? Sounds interesting.

As regards deleting: I fully agree. My comment merely related to packed executables that qualify as non-replicating malware.

Testing of real-time engine:
One have a folder full of malware. The test-program enters the folder and try to access every file with OpenForRead API. AV program should react.
2nd test is CopyFile test.
The program counts the number of blocked/succesful operations

There is also HTTP, FTP and Mail download test.
This have two aspects, one is - do the AV program scans the transfers or it watches the HDD for changes.
Interesting stuff is that some AV programs react >15 seconds after the download is finished.

That is to be expected, given the size of the AV's signature database. It has to do a compare against every single signature, so an AV that scans very quickly is probably worthless (some fake antivirus programs just flog the disk for a few seconds and then pretend to find some virus in a file that doesn't even exist, just to coerce you to buy the full version - which is likely to do the same thing).

@bobby

One further thing you should bear in mind: after the finalization of your tests with compressed malware you will probably come to the conclusion that Kaspersky has the best unpacking engine (unless you wrongly interpret the results for McAfee ;-). You should be careful, however, to conclude that Kaspersky's scan technology is "superior". In fact, it comes at a bitter price. For example, you may try the following: take a Bionet 3.18 sample (uncompressed). It is of course detected by Kaspersky. Now change the entry point from 0009F20C to 0009F20D with the help of a PE editor. The sample will still execute as normal (because it's a Delphi app) but ...

My point is: you need to have a basic understanding of the scan engine of the various scanners in order to properly interpret the test results. In the case of Kaspersky the signature match is done at a location relative to the OEP. Doh!

@N8di8
Thanks for the tip, it would be a good test-case

@LLXX
I guess I didn't explained well.
For the same virus, on OpenForRead test - the AV reacts imediately, but on download the virus from the net - the reaction is after 15 seconds.
It is not the same like downloading with browsers, because browser will download first to the cache, and then it will copy to destination folder. There is at least one CopyFile (if WinAPI is used for copy) or OpenForRead (if streamed copy is used).
In the case of 'pure' download, only thing that can be monitored are FolderChange or similar events. Here goes the late reaction.

Some AV programs monitors even HTTP transfer. There will come to reaction even before the download is finished.