The tests seem to be based mainly on scripting Office through OLE automation. This is certainly interesting, but it would be more helpful to have a performance breakdown, looking at various aspects of OS performance. Otherwise, how do we know whether this is more to do with, say, OLE performance than Vista performance?
Second, Vista has richer graphics than XP. Think of it like a game which offers difference graphics options to suit your hardware. Typically, you can set varying levels of background detail, shadow effects, etc, in order to find the right compromise between appearance and performance. Vista sets this higher than XP by default, so you would expect screen operations to be slower.
Third, Vista’s UAC security imposes a significant performance overhead. Again you would expect that, since it applies additional checks to numerous operations.
You could conduct a more useful test by configuring Vista to work as much as possible like XP. Turn off UAC, turn off Aero, turn off the indexer, turn off visual effects. I would also suggest checking what is running in the background and matching it as closely as possible on the two machines. I would be surprised if the performance difference is so striking after doing these things.
Perhaps you will argue that this is not the real-world experience. In practice, users take the machine as supplied and start using it; they do not open Control Panel and look at Performance Information and Tools, or get rid of all that third-party foistware. It’s a fair point, though on a corporate network admins can set these things on the user’s behalf.
When it comes to real-world performance, I also have gripes about Vista. However, I have little problem with Vista’s overall performance, say when working in Office, aside from Outlook 2007 which is a disgrace. What bugs me is unexpected delays. For example, I click the Start button, then All Programs, then scroll down to a program group and click. Vista just stops. I tried it just now, and it took 12 seconds to open the group I chose. I’ve had to train myself not to keep clicking. What’s going on there?
There are real issues here, but not helped by misleading reporting of inadequate tests.