========================================== File 2PROLOG.TXT Prologue of VTC Millennium test "2000-04": ========================================== Formatted with non-proportional font (Courier) As the flow of documents and other forms of "active content" (such as Java applets) via Internet continues to increase at growing speed and with growing impact esp. on networks, macro viruses, worms and several types of malware, esp. including trojan horses and hostile applets, gradually become a major threat for users and business. Consequently, there is growing need for AntiMalware software to effectively protect users from such threats. Moreover, AntiMalware software must also detect malicious (viral and non-viral) code in objects packed with compression methods often used to optimize transfer costs. VTC has therefore upgraded its previous tests to include significantly more viral and non-viral malware, and to include testing the detection quality of 4 popular compression tools found widely in Internet usage, namely ARJ, LHA and ZIP and RAR. Concerning AntiMalware software, only few very specialised products (e.g. to filter hostile Java applets/viruses) are presently available. Fortunately, most AntiVirus products have been adapted to detect some forms of malware including esp. trojan horses. Admittedly, techniques used in detecting viruses are not ideally suited to identify trojanized software; but instead of waiting for some "scientifically solid" definition of AntiMalware and for some theoretical foundation of adequate methods (into which VTC invests some efforts), it is worthwhile to determine the degree of ability of contemporary AntiVirus products in detecting and warning also of such (non-viral) threats. In past tests, VTC has determined the ability of AV products to detect also non-viral forms of malware; such test were performed as far as AV manufacturers did not explicitly contradict (in test "1998-10", this applied to 5 manufacturers). As VTC malware tests demonstrated that almost all AV products are able to detect a significant part of malware, VTC now considers its malware test as *mandatory part of VTC tests*. Indeed, all essential AntiVirus manufacturers agreed that their product would also be tested against VTC´s malware testbeds. VTC test "2000-04" has also been upgraded to include a significant portion of polymorphic file viruses. Since some time, VTC has an engine which dynamically produces generations of polymorphic file viruses and tests the ability of AV products to detect all related samples; this engine which so far generated several 100,000 viral samples (not all of which were detected by scanners-in-test) was used to generate 10,000 samples of 6 essential polmorphic file viruses each to be included in a special testbed. Finally, a special test is devoted to the ability of scanners to detect any of about 11,000 viruses generated with VKIT file virus generator. This was included as separate (VKit) test, first because of better handling of this particularly large testbed, and second as AntiVirus producers disagree whether these viruses should be counted as different instantiations of just 1 virus or as different viruses (thus boosting the number of detected viruses). Generally, we welcome any comment which helps us developing our tests further to give interested users more information about the tools which they use. On behalf of the VTC team: Klaus Brunnstein (May 26, 2000) ********** Different Prologues of previous VTC Tests: ************** -Prologue of VTC Tests "1999-03" and "1999-09" similar to "2000-04"- ---------------- Prologue of VTC Test "1998-10" -------------------- With growing flow of documents of software via Internet, macro viruses and some forms of malware, esp. including trojan horses, become a major threat for users. Moreover, AntiMalware software must also detect malicious (viral and non-viral) code in packed objects. VTC has therefore upgraded its previous tests to include significantly more viral and non-viral malware, and to include testing the detection quality of 4 popular compression tools found widely in Internet usage, namely ARJ, LHA and ZIP and RAR. VTC regrets that some manufacturers didnot agree that their product is tested for malware detection. We understand that techniques used in contemporary AntiVirus products are not well adapted to also detect non-viral malware, but we sincerely hope that AN producers try to also protect their customers against growing threats of malware streaming into local systems in growing numbers from the Internet. ---------------- Prologue of VTC Test "1998-02" -------------------- As malicious software evolves becoming a major threat for IT and Network users, evolution of AV tests has to take several directions at once: - as the multiplicity of platforms grows, AV products must be tested against broadly sued platforms, including DOS, Windows 95 and Windows NT; - as the multiplicity of viruses grows, testbeds for boot, file and macro viruses must equally be adaptes to match the actual status of potential threats; - as kinds and numbers of non-viral malicious software ("malware") grow equally, relevant tests should also check whether AV products detect other forms of malware which users need to detect, including trojan horses, droppers of malicious code, intended (though not properly self- replicating) viruses, worms, as well as hostile agents, worms and other attacks on networks. VTC test "1998-02" follows the described trends and requirements: - 3 platforms are tested: DOS, Windows 95 and Windows NT; - the virus databases were significantly updated; - the file and macro malware databases (first in last VTC test "1997-07") were significantly updated. It is not VTCs goal to blackmail any AV producer. Our basic as- sumption is that almost all AV producers try their best to protect their customers (both present and future ones) against malicious and especially viral software. We therefore try to help AV producers to improve their products, and to help users to compare their preferred product with others. Any advice and remark which helps us to achieve our determined goals will be welcomed. On behalf of the VTC Team: Klaus Brunnstein (March 16, 1998) ----------------- Prologue of VTC Test "1997-02" ------------------- "In ol' times when Vesselin Vladimirov Bontchev was active in testing AV products and Morton Swimmer was around developing his Virus Intrusion Detection Expert System (VIDES), and with many more students at the Virus Test Center of Hamburg University`s Faculty for Informatics..." Although these "ancient times" are not so far back (Vesselin left in July 1995 to work with Fridrik Skulason, and Morton left in January 1996 for IBMs High Integrity Computing Labs), significant changes have appeared. The number of boot/file viruses has more than doubled (to reach more than 11,000 file viruses and 700 boot viruses at the end of November 1996). A new species of viruses has appeared: the MACRO viruses, which soon reached world-wide distri- bution within about 1 year, with unlucky assistance of MicroSoft. Far beyond, the fast development of Local and Wide Area Networks (esp. of Internet) has been accompanied by more serious threats, including massive automated scanning of sites, mail bombing, spoofing, sniffing and data hijacking, to mention only few. More recently, malicious agents and "hostile applets" (assumed to be impossible by adherents of "SECURE JAVA") enlarge Pandora`s Box of malevolent anomalies. The importance of single-system threats, esp. including "computer viruses" has therefore relatively decreased, though these threats grow in absolute figures and in their damaging potential. With views of their future duties, students are more interested in Network Test Center (NTC) organized in parallel to VTC for those concentrating on studies on IT Security and Safety offered in 4- semester courses at Hamburg University`s Faculty for Informatics (for details, see VTC/NTC homepage). This is one essential reason that AV Product tests were only resumed 1996 when fresh interested students joined VTC asking for new activities. Fortunately, VTC's virus database could be updated to again reflect the actual status of the threats. Macro viruses provided interesting methods and future job demands, so allocation of related knowledge and methods seemed promissing. In this situation, the ol' VTC activities were restarted, with fresh aims. As VTCs databases are comparatively large, this test was explicitly set-up to assess not only detection of viruses, both generally and "In-The-Wild". Moreover, we try to assess the precision and reliability of virus detection. Both aspects are of major concern for users, esp. as they are prerequistite for any reliable cleaning. These text files result from a a first round of testing on-demand scanning on media. It is intended to enlarge the scope of our tests step-by-step, to also cover testing on-access scanners, virus cleaning as well as virus detection in memory. Moreover, we also plan to test virus detection on other platforms such as Windows 95. As usual in scientific work, we very much welcome critical and con- structive comments. Though we did our best to avoid errors, some may hard to be avoided, as our insight into related products may be in- sufficient (e.g. due to missing or ill-understood documentation). We will properly analyse any suggestion and critical comment IF adequate forms and ways are used, though we will not react on any indecent or flaming attacks. In presenting these test results, it is NOT our goal to blame any AV producer for problems of their product. Nor is it our goal to help any marketing expert in selling poducts which reach beneficial results. Indeed, it is outside our possibilities to influence such side-effects. But besides collecting methodical insights into such test processes, it is our ESSENTIAL GOAL to help customers orient themselves in jungles of mis-information. If this test may help some customer in overcoming or avoiding related problems, we would regard our goals to have been successfully reached. On behalf of the VTC Test Team: Klaus Brunnstein (February 14, 1997) brunnstein@rz.informatik.uni-hamburg.d400.de