=============================================== File 6BDOSFIL.TXT DOS.I: Detailed results of File Virus Detection of on-demand scanner tests under DOS: =============================================== (Formatted with non-proportional font: Courier) The following *13* products (versions) participated in DOS (File, Boot and Macro virus) tests (for details of AV producers: see A2SCNLS.txt, which also includes several scanners which didnot install or execute at all): ----------------------------------------------------------------- Products submitted for aVTC test under DOS: ----------------------------------------------------------------- AVA V7.70 Database: VPS 7.70-47, Dec.4,2000 AVG 6.220 Database: 105 AVK Version 3.0 Build 133 Database: Dec.01,2000 AVP Version 3.0 Build 133 DRW 4.21 Database: Sep.25,2000 FPR 3.08b Database: Dec.11,2000 INO V4.5 n(s) Database: Dec.11,2000 MR2 1.14 (Dec.2000) Database: Dec.11,2000 NAV Database: Dec.07,2000 NVC 4.90.00 PAV 3.0 Build 131 Database: Dec.08,2000 SCN v4.12.0 Database: Dec.06,2000 VSP 12.02.2 ----------------------------------------------------------------- There have been serious problems with several AV products (esp. concerning handling of the very large file virus testbed where subdirectories were not touched in the overall test, so that many post-scans were needed). For details of problems, see 8PROBLMS.TXT. When even postscans were unsuccessful, related results are missing in the related tabels. The following tables summarize detection and identification quality concerning FILE viruses as well as selected FILE MALWARE, both in full "zoo" virus collection and for viral ITW testbed. Additionally, test results are reported concerning detection of (6*10,000) viruses in a testbed with generations of 6 polymorphic file viruses, as well as a subset of 10,706 viruses generated from VKIT virus construction kit. Moreover, results for detection of viruses in files compressed with *6* popular packing methods are also given. Finally, a special test was performed concerning "false positive" virus detection of selected files which were deliberately chosen from available CD-ROMs and which were definitively clean of viruses. For discussion of results, see 6ASUMOV.TXT and 7EVAL.TXT. Results may be influenced by problems experienced during tests; such problems are documented in 8PROBLMS.TXT. Index of tables: ---------------- FDOS.F1: "FileVirus 1": Results of "full" Zoo test for file viruses FDOS.F2: "FileVirus 2": Results of "In-The-Wild" test for file viruses FDOS.FA: "Polyfile-Test": Results of Polymorphic test FDOS.FB: "VKIT Test": Results of VKIT file virus test FDOS.F3V: "Comparison of Detection Rate of Packed Viruses": Results of Detection Rate of ITW file viruses packed with PKZIP, LHA, ARJ, RAR, WinRAR and CAB FDOS.F3F: "Comparison of Detection Rate of Packed Viral Objects": Results of Detection Rate of objects infected with ITW file viruses and packed with PKZIP, LHA, ARJ, RAR, WinRAR and CAB FDOS.F3a: "PKZIP-Packed File Viruses": Results of Detection of ITW File Viruses Packed wiith PKZIP FDOS.F3b: "LHA-Packed File Viruses": Results of Detection of ITW File Viruses Packed with LHA FDOS.F3c: "ARJ-Packed File Viruses": Results of Detection of ITW File Viruses Packed with ARJ FDOS.F3d: "RAR-Packed File Viruses": Results of Detection of ITW File Viruses Packed with RAR FDOS.F3e: "WinRAR-Packed File Viruses": Results of Detection of ITW File Viruses Packed with WinRAR FDOS.F3f: "CAB-Packed File Viruses": Results of Detection of ITW File Viruses Packed with CAB FDOS.F4: "False Positive" detection: Results of "full" Zoo test for Non-viral (clean) samples detected as "False Positives" FDOS.F5: "File Malware": Results of "full" Zoo test for File-related malware FDOS.F6: "Exotic Malware" detection: Results of detection of replicative and non-replicative malware for several other platforms (Linux, Unix et al) Table FDOS.F1: "FileVirus 1": Results of "full" Zoo test for file viruses: =============================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20564 100.0 150703 100.0 ---------------------------------------------------------- AVA 19575 95.2 884 4.3 183 0.9 145269 96.4 AVG 16846 81.9 571 2.8 373 1.8 129293 85.8 AVK 20498 99.7 579 2.8 54 0.3 150466 99.8 AVP 20527 99.8 589 2.9 30 0.1 150554 99.9 FPR 20111 97.8 24 0.1 59 0.3 148739 98.7 INO 18714 91.0 679 3.3 274 1.3 138067 91.6 NAV 18673 90.8 1590 7.7 665 3.2 139047 92.3 PAV 20539 99.9 595 2.9 13 0.1 150584 99.9 SCN 20515 99.8 639 3.1 4 0.0 150640 100.0 ----------------------------------------------------------- Table FDOS.F2: "FileVirus 2": Results of "In-The-Wild" Test for file viruses: ====================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 20 100.0 2 10.0 0 0.0 409 100.0 AVG 20 100.0 4 20.0 0 0.0 409 100.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 20 100.0 0 0.0 0 0.0 409 100.0 FPR 20 100.0 0 0.0 0 0.0 409 100.0 INO 20 100.0 1 5.0 2 10.0 407 99.5 MR2 12 60.0 1 5.0 1 5.0 332 81.2 NAV 20 100.0 3 15.0 0 0.0 409 100.0 NVC 20 100.0 2 10.0 0 0.0 409 100.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 9 45.0 3 15.0 1 5.0 295 72.1 ----------------------------------------------------------- Table FDOS.FA: "Polyfile-Test": Results of Polymorphic test: ============================================================ This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Maximum 6 100.0 60000 100.0 ---------------------------------------------------------- AVA 6 100.0 1 16.7 1 16.7 59999 100.0 AVG 6 100.0 0 0.0 0 0.0 60000 100.0 AVK 6 100.0 0 0.0 0 0.0 60000 100.0 AVP 6 100.0 0 0.0 0 0.0 60000 100.0 DRW 6 100.0 0 0.0 0 0.0 60000 100.0 FPR 6 100.0 1 16.7 0 0.0 60000 100.0 INO 6 100.0 2 33.3 0 0.0 60000 100.0 MR2 6 100.0 3 50.0 2 33.3 59797 99.7 NAV 6 100.0 1 16.7 0 0.0 60000 100.0 NVC 6 100.0 1 16.7 0 0.0 60000 100.0 PAV 6 100.0 0 0.0 0 0.0 60000 100.0 SCN 6 100.0 1 16.7 0 0.0 60000 100.0 ---------------------------------------------------------- Remark: For 6 polymorphic viruses (with Maltese Amoeba, MTE.Encroacher.B, NATAS, TREMOR, One-Half and Tequila as in the previous test), 10,000 generations each were produced with VTCs dynamic polymorphic generation and test engine. For each virus, 100 directories including infected objects with goat files of lengths ranging from 1 kByte to 100 kByte were generated. Table FDOS.FB: "VKIT Test": Results of VKIT file virus test: ============================================================ This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 10706 100.0 104640 100.0 ---------------------------------------------------------- AVA 10706 100.0 1642 15.3 23 0.2 104595 100.0 AVG 10137 94.7 806 7.5 117 1.1 97780 93.4 AVK 10706 100.0 1196 11.2 0 0.0 104640 100.0 AVP 10706 100.0 1194 11.2 0 0.0 104640 100.0 DRW 10706 100.0 1007 9.4 1 0.0 104639 100.0 FPR 10706 100.0 1439 13.4 3 0.0 104636 100.0 INO 10703 100.0 1250 11.7 8 0.1 104592 100.0 MR2 10704 100.0 7519 70.2 1 0.0 104636 100.0 NAV 10696 99.9 654 6.1 120 1.1 103947 99.3 NVC 10704 100.0 6191 57.8 323 3.0 102073 97.5 PAV 10706 100.0 1194 11.2 0 0.0 104640 100.0 SCN 10706 100.0 1168 10.9 0 0.0 104640 100.0 VSP 10638 99.4 5925 55.3 71 0.7 103416 98.8 ----------------------------------------------------------- Remark: A testbed of 10,706 viruses generated with the VKIT virus generator (out of about 14,000 viruses which can be generated) was used to test detection quality. This test was separated from the "normal" file virus test as 1) there is no agreement between AV producers whether viruses from VKIT should be counted just as 1 or as 14,000 different viruses (boasting number of detected viruses to over 40,000), and 2) because of the large size of this special testbed. Table FDOS.F3V: "Comparison of Detection Rate of Packed Viruses": Results of Detection Rate of ITW file viruses packed with PKZIP, LHA, ARJ, RAR, WinRAR, CAB ================================================================ This includes Viruses detected per packer ------------------------------------------------------------------------- ZIP % LHA % ARJ % RAR % WRAR % CAB % ------------------------------------------------------------------------- TestBed 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 ------------------------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 AVP 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 DRW 20 100.0 0 0.0 20 100.0 20 100.0 20 100.0 0 0.0 FPR 20 100.0 0 0.0 20 100.0 20 100.0 20 100.0 0 0.0 INO 19 95.0 0 0.0 20 100.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 NAV 20 100.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 SCN 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 20 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 ------------------------------------------------------------------------- Table FDOS.F3F: "Comparison of Detection Rate of Packed Viral Objects": Results of Detection Rate of objects infected with ITW file viruses and with PKZIP, LHA, ARJ, RAR, WinRAR, CAB ======================================================================== This includes Viral objects detected per packer ------------------------------------------------------------------------------- ZIP % LHA % ARJ % RAR % WRAR % CAB % ------------------------------------------------------------------------------- TestBed 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 ------------------------------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 AVK 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 AVP 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 DRW 409 100.0 0 0.0 409 100.0 409 100.0 409 100.0 0 0.0 FPR 409 100.0 0 0.0 409 100.0 409 100.0 409 100.0 0 0.0 INO 387 94.6 0 0.0 407 99.5 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 NAV 409 100.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 PAV 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 SCN 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 ------------------------------------------------------------------------------- Table FDOS.F3a: "PKZIP-Packed File Viruses": Results of Detection of ITW File Viruses Packed with PKZIP: ================================================================= This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 20 100.0 0 0.0 0 0.0 409 100.0 FPR 20 100.0 0 0.0 0 0.0 409 100.0 INO 19 95.0 1 5.0 1 5.0 387 94.6 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 20 100.0 3 15.0 0 0.0 409 100.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F3b: "LHA-Packed File Viruses": Results of Detection of ITW File Viruses Packed with LHA: =============================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 0 0.0 0 0.0 0 0.0 0 0.0 FPR 0 0.0 0 0.0 0 0.0 0 0.0 INO 0 0.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F3c: "ARJ-Packed File Viruses": Results of Detection of ITW File Viruses Packed with ARJ: =============================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 20 100.0 0 0.0 0 0.0 409 100.0 FPR 20 100.0 0 0.0 0 0.0 409 100.0 INO 20 100.0 1 5.0 2 10.0 407 99.5 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F3d: "RAR-Packed File Viruses": Results of Detection of ITW File Viruses Packed with RAR: =============================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 20 100.0 0 0.0 0 0.0 409 100.0 FPR 20 100.0 0 0.0 0 0.0 409 100.0 INO 0 0.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F3e: "WinRAR-Packed File Viruses": Results of Detection of ITW File Viruses Packed with WinRAR: ================================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 20 100.0 0 0.0 0 0.0 409 100.0 FPR 20 100.0 0 0.0 0 0.0 409 100.0 INO 0 0.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F3f: "CAB-Packed File Viruses": Results of Detection of ITW File Viruses Packed with CAB: =============================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 20 100.0 409 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 20 100.0 1 5.0 0 0.0 409 100.0 AVP 20 100.0 1 5.0 0 0.0 409 100.0 DRW 0 0.0 0 0.0 0 0.0 0 0.0 FPR 0 0.0 0 0.0 0 0.0 0 0.0 INO 0 0.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 20 100.0 1 5.0 0 0.0 409 100.0 SCN 20 100.0 1 5.0 0 0.0 409 100.0 VSP 0 0.0 0 0.0 0 0.0 0 0.0 ---------------------------------------------------------- Table FDOS.F4: "False Positive" detection: Results of "full" Zoo test for Non-viral (clean) samples detected as "False Positives": ============================================================ False This includes Virus ---- unreliably ---- Files Scanner Alarm identified detected detected ---------------------------------------------------------- Maximum 27 100.0 664 100.0 ---------------------------------------------------------- AVA 0 0.0 0 0.0 0 0.0 0 0.0 AVG 0 0.0 0 0.0 0 0.0 0 0.0 AVK 0 0.0 0 0.0 0 0.0 0 0.0 AVP 0 0.0 0 0.0 0 0.0 0 0.0 FPR 0 0.0 0 0.0 0 0.0 0 0.0 INO 0 0.0 0 0.0 0 0.0 0 0.0 MR2 0 0.0 0 0.0 0 0.0 0 0.0 NAV 0 0.0 0 0.0 0 0.0 0 0.0 PAV 0 0.0 0 0.0 0 0.0 0 0.0 SCN 0 0.0 0 0.0 0 0.0 0 0.0 ----------------------------------------------------------- Remark: within 27 non-viral directories and totally 664 non- viral objects, at least one sample in N directories was falsely detected (N = number in column 1) Table FDOS.F5 "File Malware": Results of "full" Zoo test for File-related malware: ======================================================== This includes Malware ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 6250 100.0 12160 100.0 ---------------------------------------------------------- AVA 3188 51.0 110 1.8 185 3.0 6403 52.7 AVG 3166 50.6 62 1.0 213 3.4 5566 45.8 AVK 6000 96.0 397 6.4 49 0.8 11636 95.7 AVP 6023 96.4 397 6.4 46 0.7 11675 96.0 FPR 5872 94.0 13 0.2 125 2.0 11326 93.1 INO 2993 47.9 89 1.4 56 0.9 5218 42.9 MR2 2619 41.9 148 2.4 339 5.4 3924 32.3 NAV 2818 45.1 85 1.4 156 2.5 5354 44.0 PAV 6015 96.2 399 6.4 45 0.7 11666 95.9 SCN 5653 90.4 214 3.4 28 0.4 11304 93.0 VSP 2734 43.7 152 2.4 161 2.6 4302 35.4 ----------------------------------------------------------- Table FDOS.F6 "Exotic Malware" detection: Results of detection of replicative and non-replicative malware for several other platforms (Linux, Unix et al) ============================================================== This includes Viruses ---- unreliably ---- Files Scanner detected identified detected detected ---------------------------------------------------------- Testbed 115 100.0 274 100.0 ---------------------------------------------------------- AVA 17 14.8 0 0.0 4 3.5 64 23.4 AVG 6 5.2 0 0.0 0 0.0 46 16.8 AVK 103 89.6 3 2.6 0 0.0 251 91.6 AVP 104 90.4 3 2.6 1 0.9 252 92.0 DRW 43 37.4 1 0.9 3 2.6 150 54.7 FPR 76 66.1 1 0.9 4 3.5 129 47.1 INO 29 25.2 2 1.7 1 0.9 126 46.0 MR2 2 1.7 0 0.0 1 0.9 2 0.7 NAV 14 12.2 1 0.9 3 2.6 50 18.2 NVC 75 65.2 2 1.7 5 4.3 150 54.7 PAV 106 92.2 3 2.6 1 0.9 254 92.7 SCN 81 70.4 8 7.0 2 1.7 220 80.3 VSP 30 26.1 0 0.0 11 9.6 94 34.3 ----------------------------------------------------------