Why Outlook 2007 is slow: Microsoft’s official answer

A knowledgebase article published last week acknowledges performance problems with Outlook 2007, though it says these only occur with mailboxes larger than 2GB:

You may experience one or more of the following performance problems when you are working with items in a large Personal Folder file (.pst) or in a large Offline Folder file (.ost) in Microsoft Office Outlook 2007 … Note When you perform the same operations on the large .pst or .ost file in earlier versions of Outlook, the same performance problems do not occur. These problems may occur if the .pst or .ost file is larger than 2 GB. Additionally, the performance problems are more pronounced when the .pst or .ost file is larger than 4 GB.

I think this is optimistic and that smaller mailboxes are slower too; nevertheless, it does confirm that that the size of the local store is the key issue.

If you use Exchange, the local store is the .PST or .OST file on your workstation or laptop. If you do not use Exchange, a local .PST store is all you have.

Here’s what Microsoft says is the reason:

To accommodate new features, Outlook 2007 introduced a new data structure for .pst and .ost files. In this new data structure, the frequency of writing data to the hard disk increases as the number of items in the .pst or .ost files increases.

Intriguing, especially as I had thought the .pst format was the same in Outlook 2003 and 2007. The big change was from Outlook 2000 to Outlook 2003, when Unicode was introduced and the maximum size increased to 20GB.

I’d also like to know whether Microsoft is just stating the obvious here (bigger file, more disk access); or whether there is some exponential increase in disk writes, suggesting a design fault in the software. I have already noticed that if you show the I/O columns in Task Manager’s performance tab, Outlook 2007 shows some extraordinarily large numbers.

So what’s the fix? The news is not too good. In essence, you have to reduce the size of the local store. You can archive or move items to separate .pst files, or switch off cached mode so you always work online to Exchange.

The article doesn’t say it, but there are significant problems with switching off cached mode. These include hugely increased network traffic, problems with junk mail filtering, and loss of all your mail when using a laptop disconnected from the network.

The most imaginative suggestion is to filter the sychronization. For example, you could filter out messagse with large attachments, or all messages from last year or earlier. These messages will still exist in Exchange, but not in the local store.

Worth a try, but none of the workarounds is really satisfactory. Outlook 2003 worked fine with large mailboxes, Outlook 2007 does not. That’s a blunder.

 

Vista – worth having?

Now that Vista is on the shelves, people are asking: is it worth having?

I’ve been testing it for a while now, using it for most of my work and as a media center.

If there is a “Wow” in Vista, it is in the Windows Presentation Foundation, not the core operating system. And WPF is available for XP as well; and there aren’t yet many applications which use it. So forget the wow for now.

That said, it is mostly an improvement. Why mostly? Mainly because of driver quality. For example, I’ve been chasing an Intel display driver bug for a couple of weeks. It made certain games unplayable and also caused problems when more than one display was active. Last week Intel posted an update that fixes the problem. That’s on a laptop; on my desktop the sound card doesn’t work as it should – because Creative’s Vista drivers are still in beta and far from production quality. I get stuttering sound from a supposedly high-end X-Fi card.

These issues will gradually disappear as the hardware vendors properly support Vista. That said, I have a scanner that will probably never work. It’s old enough that the vendor has no incentive to come up with a driver.

The other major issue is software compatibility. Everything has to work with XP, but Vista is new and there may be problems. Most of these are caused by the new security feature called User Account Control. In reality I have not had many problems. If you have a few key applications you depend on, it makes sense to verify whether or not they run on Vista before making the switch.

Upgrade? Buy new?

Now a few specifics. Would I upgrade a laptop? No, not unless you enjoy techie problems or can get a supported upgrade pack from the vendor. Laptops are stuffed with devices, updating the hardware is near-impossible, and things like sleep and resume are prone to go wrong.

Would I upgrade a desktop? Possibly, if it is no more than a couple of years old. It’s still somewhat risky. I’d plan to upgrade the RAM to 1GB or more, update the motherboard BIOS, and buy a new graphics card. You might get away without; but my impression is that Vista is more demanding (ie. slower) on the same hardware than XP.

Would I buy a new compter – desktop or laptop – with Vista rather than XP? Yes, provided you’ve established that you can run or replace the applications you depend on and the hardware you intend to plug in. 

Vista is a better version of Windows, more logically organized, more pleasant to use, more secure. The best feature for usability is the search box on the start menu. No more hunting through the fly-out menus; just click Start, type the first few letters of what you want and hit Enter.

Security

How much more secure? Unfortunately the blizzard of hype and counter-hype has obscured the security changes in Vista. A substantial industry has been built on security weaknesses in Windows, and this industry is desperate to persuade us that we still need its services, while journalists everywhere are keen to find and publicise any security problems; and undoubtedly there are and will be problems to find.

The key change is that users by default run without local administrator permissions. This brings Windows into line with standard practice on other operating systems including Linux and Mac OS X. In consequence system files are protected unless the user passes a dialog approving a change. Some claim that these dialogs pop up frequently and are annoying. I can’t substantiate that – I don’t often see them, and when they do appear I don’t find them particularly objectionable though there are cases when I’m not sure why admin rights are needed.

Of course if a virus comes along in an email attachment and says, “I’m an important update from Microsoft, please run me”, and you click Allow, then Vista isn’t going to help you.

Another less publicized change is Internet Explorer’s Protected mode, again on by default. This means IE runs with even more limited rights, and should help to prevent silent installs of malicious software. Arguably, this makes IE more secure than FireFox on Vista. 

In reality, this is a process. The changes in Vista mean that software vendors might actually stop producing applications that breach basic Windows guidelines. A side-effect will be better separation of application code and data, which will help with backup as well as security. It will make sense to set Vista to a higher level of security, where you have to enter an admin password to make system changes, and the intrusive dialogs will appear less often.

 

Technorati tags: , , , ,

Help! I’ve been sent an ODT file

The document format wars are upon us. I know this, because I got a tech query from someone at another desk. She couldn’t open the attachment she’d been sent. The file had an .odt extension. Someone had saved a document from Open Office using its defaults, and emailed it, probably without realising that this could cause problems for the recipient.

ODT is Open Document Text, the XML document format supported by Open Office and heavily promoted by IBM, Sun and everyone not in the Microsoft camp. The solution? There are converters around, some of which don’t work properly, but the easy answer is to go along to openoffice.org and download the free Open Office suite. In Windows, this sets up the requisite file association so you can double-click an .odt document and it opens. Once open, you can edit it in Open Office or use the clipboard to copy the contents into Word or other applications.

Installing Open Office is painless. The main caveat is that you might want to stop the thing loading itself at startup. Otherwise you’ll find a process called soffice.bin occupying large amounts of memory even when you are not using it. Right-click the OpenOffice icon in the system tray, uncheck “Load OpenOffice.org during system startup”, then choose Exit Quickstarter. Next time you restart, you should not be troubled by soffice.exe or soffice.bin until you actually want to use Open Office. Of course you might prefer Open Office to Microsoft Office. In that case, by all means leave the quickstarter in place.

What’s interesting here is how effective document format frustrations are in persuading, almost forcing users to install new software. Those who follow the above advice now have two office suites on their system. If they find themselves receiving lots of .odt files, or get many requests for documents in that format, they might switch, just to make it easier to get their work done.

What about the other scenario, where users receive .docx attachments? This is Microsoft’s Open Office XML, and is the default save format in Word 2007. It’s not too bad for existing Office users; they just download an add-in from Microsoft which, unlike the ODT converter, works smoothly in my experience. Only those with Office 97 or earlier will run into problems. It’s not so good for those who do not currently use Office, or for Mac users, though free utilities like this Mac example are turning up. Note that whereas Open Office is a complete solution for .odt, most converters have shortcomings and tend to lose some of the formatting or content of the original.

The key difference here is easily stated. Users who need to deal with .odt files will install Open Office. Users who need to deal with .docx files will be more inclined to get a converter – because buying Office 2007 is expensive, or not available at all for those who do not run Windows. In other words, the document format wars will increase the installed base of Open Office, but this will be less true of Microsoft Office.

Personally I prefer Microsoft Office, though in fairness it’s a year or so since I took a careful look at Open Office. On the other hand, Open Office is free and pretty good. Many users of word processors and spreadsheets don’t stress the products at all; where this is the case, it is hard to see how Microsoft Office is worth the extra cost. That said, most people use Microsoft Office anyway, simply because it is the de facto standard. That position is now being eroded.

Google Maps puzzler

Ran into this puzzler today:

Google Maps showing Mansfield on the map, but unable to find it for directions.

I could not persuade Google Maps that it knew where Mansfield is. I mean Mansfield in Nottinghamshire; but even when I typed in “Mansfield, Nottinghamshire, UK”, Google Maps professed ignorance – although I could easily scroll to the actual location on the map itself by the strategy of searching for another place first. No doubt there are other Mansfields in the world – but why not a disambiguation menu?

Eventuall I tried Mansfield, UK and it worked. Except that the position is miles out:

I guess Google Maps just doesn’t like Mansfield. Microsoft’s local.live.com had no problems.

I’m sure a postcode would have worked; but these are not always conveniently at hand.

Technorati tags: , ,

IE7 script madness

Ever seen this guy?

Stop running this script dialog in IE7

I’m writing a piece on Javascript. In the new world of AJAX, web applications may run large amounts of client-side code in the browser. I’m having a look at performance issues, so I wrote some code that does some processing in a tight loop and tested it in IE7, FireFox 2.0 and Flash 9.

Getting timings was difficult, because IE7 pops up this “Stop running this script” dialog when my code is running. Nor will it let go. You click “No”, and 1 second later the dialog pops up again. And again. And again.

I’ve trawled through the IE7 options looking for a way to switch this thing off, but cannot find one. I’m hoping I’ve missed it, or that there is a secret registry key I can change, because it is really annoying.

I don’t understand why there is no option for “don’t ask me again”, or “allow long-running scripts at this site”. After all, this scenario is going to get increasingly common. Neither FireFox nor Flash suffers from this problem.

I appreciate that IE7 is trying to be helpful here. There is though a fine line between helpful and annoying. Without any obvious way to prevent it, this falls in the latter category.

That said, I did find a way to get my timings, because of my experience with the htmleditor.  If you host Mshtml in an application, you can implement the COM interface IDocHostShowUI. This has a ShowMessage function which IE calls when it wants to show a dialog. This enables you to catch the over-helpful “stop this script” message and not show it.

Unfortunately this solution isn’t something users can easily apply. It requires creating your own customized version of IE. There must be some easier way and I look forward to learning what it is.

One last comment: why does Microsoft still come up with poorly thought-out UI elements like this? It is easy to think of better ways than a brutal modal dialog. How about a “stop script” toolbar button that appears only when scripts are taking too long or grabbing too much CPU?

Update

FireFox does exactly the same thing, also with a modal dialog, “A script on this page may be busy” …

Still, two benefits to FireFox. First, the timeout is set to a more reasonable 10 seconds. Second, you can easily amend it. Navigate to about:config. Find the entry dom.max_script_run_time. Change it from 10 to whatever you like. 

Further update

A comment has pointed me to this knowledgebase article.

Here’s the fix:

  1. Using a Registry Editor such as Regedt32.exe, open this key:
    HKEY_CURRENT_USER\Software\Microsoft\InternetExplorer\Styles

    Note If the Styles key is not present, create a new key that is called Styles.

  2. Create a new DWORD value called “MaxScriptStatements” under this key and set the value to the desired number of script statements.

    By default the key doesn’t exist. If the key has not been added, Internet Explorer 4 defaults to 5,000,000 statements executed as the trigger for the time-out dialog box.

Technorati tags: , , ,

Steve Jobs on DRM: sense and nonsense

Kudos – mostly – to Steve Jobs for his remarks on Apple and DRM. I like his closing comments:

Convincing [big music companies] to license their music to Apple and others DRM-free will create a truly interoperable music marketplace.  Apple will embrace this wholeheartedly.

Yes please. But while I applaud these remarks, I have to note some curious logic in the rest of his defence of Apple’s DRM policy. Remember, the essence of the complaint against Apple is that it will neither license its FairPlay DRM to others, nor support other DRM schemes in its iTunes store. The consequence is that iTunes customers are locked to Apple’s software, and for portable devices, largely to its hardware as well.

Jobs says Apple doesn’t license FairPlay because it could compromise its “secrets”:

The most serious problem is that licensing a DRM involves disclosing some of its secrets to many people in many companies, and history tells us that inevitably these secrets will leak.

However, Jobs has already stated that such secrets often get cracked anyway. The intransigent problem is that the keys reside on the user’s own machine:

In other words, even if one uses the most sophisticated cryptographic locks to protect the actual music, one must still “hide” the keys which unlock the music on the user’s computer or portable music player.

This is a greater impediment to FairPlay’s security than licensing it would be. Further, any iTunes purchase can be burned to CD and ripped to unprotected files, albeit with loss of quality if you choose a compressed format. I also note that DVD Jon (as far as I’m aware) achieved his success at cracking DRM by reverse engineering, not industrial espionage.

So this statement makes no sense:

Apple has concluded that if it licenses FairPlay to others, it can no longer guarantee to protect the music it licenses from the big four music companies.

Apple has actually concluded that it can’t “guarantee to protect the music” anyway, irrespective of whether it licenses FairPlay.

Further quibbles: Jobs sees a “a very competitive market”, where others see Apple’s unhealthy dominance, particularly in portable music players.

Another. Jobs says:

Since 97% of the music on the average iPod was not purchased from the iTunes store, iPod users are clearly not locked into the iTunes store to acquire their music.

No Mr Jobs, they are not locked into the iTunes store (yet). They are locked into the iPod to play this music back. Well, subject to the caveats already discussed. And what about iTunes exclusives?

Finally, Jobs notes that “The music companies sell the vast majority of their music DRM-free”, referring to the continuining importance of CD sales, which greatly exceed online sales.

Yet CD sales are declining and will continue to do so. We are having this discussion because we know that those figures will swing, probably quite fast, and that online or subscription sales will dominate the music business.

Users would love to see more legal, DRM-free downloads. In the meantime, Apple’s refusal to interoperate its DRM with others remains anti-competitive.

Technorati tags: , , ,

Windows web server market share grows

Netcraft reports that Windows/IIS has a growing share of the web server market:

Microsoft-IIS gains 935K sites, continuing an advance that has seen Microsoft steadily chip away at what once seemed an insurmountable lead for Apache. In our Feb. 2006 survey, Apache held 68% market share, giving it lead of 47.5% over Windows (20.5% share). In this month’s survey, Microsoft’s share has improved to 31.0%, narrowing Apache’s advantage to 27.7%.

Netcraft counts sites, which means that its monthly figures are hugely influenced by the actions of a few big players in the web hosting market; many “sites” are no more than parked domains. It’s also worth noting that the total number of sites is constantly increasing. Apache probably has more users than ever before, despite the above “decline”. Other web servers have a miniscule market share.

Even with these caveats, it seems that Microsoft is at least holding its own in the web server market. My hunch is that this has to do with the high quality of ASP.NET, and the fact that Windows Server 2003 has won a decent reputation for security as a web server. I am not saying it is more secure than say Linux-Apache; just that security isn’t the deal-breaker that it tends to be on the desktop.

Congratulations to Scott Guthrie and his team.

Technorati tags: , ,

Open Document to Office Open XML converter: not good

The first full release of the Open XML to Open Document Format translator is available for download. Great news for interoperability – or is it?

I like to try things out before writing about them, so here’s what I did. I downloaded the Word 2007 add-in and ran the setup. Then I opened Word, and opened the document I was working on, which happens to be called Using DigiKam.docx. This is just under 800 words long and contains no graphics. I went to Home – Save As, and looked for Open Document in the list of document types. No deal. Puzzled, I looked again at the Home menu in Word 2007. Ah, there it is. A separate top-level entry for ODF with Open and Save As menu items. Not ideal in terms of integration, but never mind.

Note: there is an important issue here. Imagine you are an organization that has decided to mandate ODF for your documents, but to continue using Microsoft Office. What you want to do is to fiddle with Group Policy and have Word default to opening and saving ODT (Open Document Text). As far as I can tell, this is not possible with this version 1.0 release. In fact it is worse than that. If you have a new document, and choose ODF – Save As, you get the following error:

Please save your document before exporting to ODF. So instead of just clicking Save, users have to save twice, first as .docx, next as ODT. Ugly. It gets worse, read on.

OK, so I decided to save my current document as ODF. A wait message appeared: it took the converter about 30 seconds to save the document. I don’t like to think what would happen to a 10,000 word report full of charts and tables.

Next, I closed the document, went to ODF – Open, and chose the document I just saved. Another 30 seconds later I get this message about lost elements:

If I go into details, it tells me that the header dimensions and document creation and modification dates might have been lost. Fair enough, nothing drastic – unless perhaps I am laying out a booklet for publication. Of course you would be mad to use a document converter like this in such circumstances – but let’s not forget the implications of potential inflexible government legislation that might mandate such a thing.

I notice a curious thing. My opened document has been renamed to Using DigiKam_tmp.docx. Let me get my head round this. Let’s say I want always to save in ODF. I have to save as .docx, then export to ODF. Then I open the ODF document, which now has _tmp appended. I make some changes, and want to export it as ODF. I get, you guessed it, the “Please save before exporting” message. So I click save, and get a view of all my temporary documents, because the converter puts the imported document in my temp folder. If I try to save it directly, I get a “this file is read-only” error. So I save it to My Documents, then I go to ODF – Save As. Next session, I go to ODF – Open and guess what. My file is now called Using DigiKam_tmp_tmp.docx.

So the message is: don’t even think about using this converter as a means of standardising on Open Document while still using Word. It will cause immense and unnecessary hassle. However, it could still be useful for importing and exporting documents interchanged with others using, say, Open Office.

Not the same

That said, I noticed something else about my round-tripped document. It was different. In Word, I have my Normal style set with no space before or after. After round-tripping, these paragraphs had 10pt space after applied.

It gets worse. The converter lost all my paragraph styles – not the formatting, but the style tagging. This is a deal-breaker for me, as I depend on paragraph styles; but I am probably in a minority. Still, it prompted me to look at the list of unsupported features. Casting my eye down the page I came across this item:

In Open XML in real spacing between two consecutive paragraphs is the biger [stet]. For example first paragraph style has spacing after 10pt and second has spacing before 20pt the real spacing is 20pt. In Open Document Format real spacing is sum. In our example the real spacing is 30pt.

Is that my spacing problem? It could be related; but this is not what I would call a model of clarity. Let’s just say that the ODF converter will mess up your paragraph spacing.

Question: why was I warned that I might lose “header dimensions”, but these more significant issues – no paragraph styles, messed up spacing – went unmentioned?

Not professional quality

I realise that despite the flaws this converter could be a life-saver if you get a document that would otherwise be unreadable, or if you are forced by regulation to send a document in ODF format. However it does not merit Microsoft’s effusive press release, nor Brian Jones enthusiatic blog entry. It falls far short of the standards set by Microsoft Office. Perhaps I am judging too swiftly; but you will understand my scepticism considering the design flaws noted above, the extreme performance problems, and the fact that it somewhat messed up my short document without any graphics.

Practical considerations

In closing, some practical notes. If you really want to work with Open Document, don’t use Microsoft Office. If you want to use Microsoft Office, don’t use the converter except in an emergency, not in this release at least. For Word documents, RTF is the least bad option and macro-free; or failing that, the Office binary formats are actually well understood by third-party applications.

What if you use an application that supports Open Document and want to distribute richly formatted documents to others? Well, in the real world Microsoft Office is everywhere, so the same applies: RTF or Microsoft Office binary formats will help the recipients to get their work done.

Update: I spoke to Microsoft’s Jean Paoli about a number of Office Open XML issues – see here for the interview. He acknowledged there are some issues but said that performance is usually better than I found it to be. I’m sceptical but will try to do some more testing.

Technorati tags: , , , , , , ,

CD loudness: the pro perspective

I noticed this comment on my piece on CD mastering, from the webmaster at APRS (Association of Professional Recording Services):

By making CDs Extremely Limited we are not only destroying natural dynamics, we are also reducing the clarity and making CDs which are far more difficult and tiring to listen to…I’ve lost arguments and jobs over this a number of times. You master something and make it sound as good as possible, just the way the client and musicians agree they want it to sound; you transfer it at the maximum level possible without destroying it, then they take it home and complain that it’s “just not loud enough”…

I have sympathy for professionals who are in an impossible position. They have to make CDs that sound worse than they should, sometimes much worse, or risk losing their jobs.

It’s a bad situation, but there is hope. Publicising the issue must help the pros win more arguments. Technology may solve this too. ReplayGain and Apple’s Sound Check are both techniques for matching the replay volume of music automatically, removing any supposed advantage from “loud” mastering. Broadcasters already use compression and limiting on their output. There is no need to compromise the source. 

Technorati tags: ,

Miguel de Icaza on ODF vs OOXML

Novell’s Miguel de Icaza has an important and unusual perspective on Microsoft technology. Unlike many open source advocates, he is deeply familiar with the Microsoft platform because of his work on Mono, the open source implementation of the .NET Framework. I therefore read with interest his comments on the war against Microsoft Open Office XML now being waged by the sponsors of the rival Open Document Format. 

As de Icaza observes, it is “hard to articulate” the difference between OOXML and ODF. They are XML schemas, inpenetrable to non-technical folk. Both appear to do the same thing, yet in detail they have little in common. Here’s a key comment:

The high-level comparisons so far have focused on tiny details (encoding, model used for the XML). There is nothing fundamentally better or worse in those standards like there is between XML Schema and Relax NG. ODF grew out of OpenOffice.org and is influenced by its internal design. OOXML grew out of Microsoft Office and it is influenced by its internal design. No real surprises there.

I agree. But isn’t the OOXML specification too bulky and verbose, as its opposition claims?

If Microsoft had produced 760 pages (the size of ODF) as the documentation for the “.doc”, “.xls” and “.ppt” that lacked for example the formula specification, wouldn’t people justly complain that the specification was incomplete and was useless?

Quite possibly. And I am unimpressed by the efforts of Rob Weir and others at IBM in taking pot shots at flaws in OOXML rather than being constructive in helping Microsoft transition from proprietary binary document formats to XML formats with a standardised specification.

That said, OOXML and ODF do have different aims, something which Weir does not recognize. He writes in his response to de Icaza:

OOXML, on the other hand, matches to an inane degree the internals of a single vendor’s legacy application, with no concessions to platform-neutrality.

The point Weir misses is that (as I understand it) the rationale behind OOXML is to be able to represent all the world’s immense archive of Microsoft Office documents in an XML format with a published specification and without loss of information. In that sense, its goals are less lofty than those of ODF, which wants to be the one true office document specification for the world.

That means OOXML has a huge legacy burden to carry. It also implies that much of the cruft in OOXML is not there to be used by new applications, but rather to document what has to be done to support old stuff in Office.

My background is in software development, and I’ve explored the intricacies of RTF (Rich Text Format), the non-XML specification for Word documents and pretty much what you had to use prior to OOXML. I found the documentation inadequate, too closely tied to versions of Word, and difficult to work with. OOXML is delightful in comparison. The ability to generate and consume Office documents in XML substantially benefits developer productivity.

Another benefit is in working with Office documents on the server. Ugly solutions like automating Office applications on a server in order to create or process documents are no longer necessary.

I therefore disagree that OOXML has no value.

A single Office XML format for the world would have been nice. If the ODF folk had got Microsoft on board in the early days of the specification that might have been possible, though the scenario was politically implausible. What we have instead is two formats; but at least they are both XML and therefore amenable to programmatic manipulation and conversion. I think that’s progress, though it falls short of the ideal. Furthermore, it likely would not have happened without the existence of Open Office and ODF. They have won the argument for open document formats; no need to spoil it by obstructing the standardisation process for which they fought.

 

Technorati tags: , , , ,