Performance testing Vista: misleading reporting of inadequate tests.

Is Vista really half the speed of XP? That is what CNET reports here, in an article which does not bother to link to the source of the tests, as far as I can see.

The tests seem to be based mainly on scripting Office through OLE automation. This is certainly interesting, but it would be more helpful to have a performance breakdown, looking at various aspects of OS performance. Otherwise, how do we know whether this is more to do with, say, OLE performance than Vista performance?

Second, Vista has richer graphics than XP. Think of it like a game which offers difference graphics options to suit your hardware. Typically, you can set varying levels of background detail, shadow effects, etc, in order to find the right compromise between appearance and performance. Vista sets this higher than XP by default, so you would expect screen operations to be slower.

Third, Vista’s UAC security imposes a significant performance overhead. Again you would expect that, since it applies additional checks to numerous operations.

You could conduct a more useful test by configuring Vista to work as much as possible like XP. Turn off UAC, turn off Aero, turn off the indexer, turn off visual effects. I would also suggest checking what is running in the background and matching it as closely as possible on the two machines. I would be surprised if the performance difference is so striking after doing these things.

Perhaps you will argue that this is not the real-world experience. In practice, users take the machine as supplied and start using it; they do not open Control Panel and look at Performance Information and Tools, or get rid of all that third-party foistware. It’s a fair point, though on a corporate network admins can set these things on the user’s behalf.

When it comes to real-world performance, I also have gripes about Vista. However, I have little problem with Vista’s overall performance, say when working in Office, aside from Outlook 2007 which is a disgrace. What bugs me is unexpected delays. For example, I click the Start button, then All Programs, then scroll down to a program group and click. Vista just stops. I tried it just now, and it took 12 seconds to open the group I chose. I’ve had to train myself not to keep clicking. What’s going on there?

There are real issues here, but not helped by misleading reporting of inadequate tests.

Buzzword gets word count, easier sharing

Adobe’s online, Flash-based word processor has been revised and now includes word count, handy for journalists and essay writers:

buzz_count

It also has easier sharing: simply copy the URL at the top of the document you are editing to create a link. I intended to demo this here; but cannot because Buzzword does not yet support public sharing. You can only share with other specified Buzzword users identified by their email address. Buzzword says public sharing is “a feature that we’re actively investigating.”

Amusing: I noticed that my document had a number of flagged words, wiggly underlines that indicate likely spelling errors. Two of these unrecognized terms were “Google” and “Zoho”. “Buzzword” on the other hand is in the dictionary.

Technorati tags: , , , ,