Sophos is getting good publicity for its latest sales pitch virus test on Windows 7. This tells us:

We grabbed the next 10 unique samples that arrived in the SophosLabs feed to see how well the newer, more secure version of Windows and UAC held up. Unfortunately, despite Microsoft’s claims, Windows 7 disappointed just like earlier versions of Windows. The good news is that, of the freshest 10 samples that arrived, 2 would not operate correctly under Windows 7.

Unfortunately Chester Wisniewski from Sophos is vague about his methodology, though he does say that Windows 7 was set up in its default state and without anti-virus installed. The UAC setting was on its new default, which is less secure (and intrusive) than the default in Windows Vista.

My presumption is that he copied each virus to the machine and executed it – and was apparently disappointed (or more likely elated) to discover that 8 out of 10 examples infected the machine.

It might be more accurate to say that he infected the machine, when he copied the virus to it and executed it.

I am not sure what operating system would pass this test. What about a script, for example, that deleted all a user’s documents? UAC would not attempt to prevent that; users have the right do delete their own documents if they wish. Would that count as a failure?

Now, it may be that Wisniewski means that these executables successfully escalated their permissions. This means, for example, that they might have written to system locations which are meant to be protected unless the user passes the UAC prompt. That would count as some sort of failure – although Microsoft has never claimed that UAC will prevent it, particularly if the user is logged on with administrative rights.

If this were a serious study, we would be told what the results were if the user is logged on with standard user rights (Microsoft’s long-term goal), and what the results were if UAC is wound up to its highest level (which I recommend).

Even in that case, it would not surprise me if some of the malware succeeded in escalating its permissions and infecting system areas, though it would make a more interesting study. The better way to protect your machine is not to execute the malware in the first place. Unfortunately, social engineering means that even skilled users make mistakes; or sometimes a bug in the web browser enables a malicious web site to install malware (that would also be a more interesting study). Sometimes a user will even agree to elevate the malware’s rights – UAC cannot prevent that.

My point: the malware problem is too important to trivialise with this sort of headline-grabbing, meaningless test.

Nor do I believe the implicit message in Wisniewski’s post, that buying and installing Sophos will make a machine secure. Anti-virus software has by and large failed to protect us, though undoubtedly it will prevent some infections.

See also this earlier post about UAC and Windows security, which has links to some Microsoft statements about it.