Tim Anderson’s ITWriting

More details are emerging about the ODF converter which will be in Office 2007 SP2, set for release next year. Doug Mahugh’s post outlines the architecture and explains how the converter will deal with compatibility issues.

I want to highlight the section that describes how the converter is implemented:

Word, Excel and PowerPoint have a Model-View-Controller design. The in-memory representation of the document, or Model, is designed to facilitate document revision and display functions and includes concepts which are never saved to the file, such as the insertion point and the selection.

The persistence code converts this in-memory representation to and from some sort of the disk file based representation. Office 2007 already had code to support a number of angle-brackety persistence formats including HTML and OOXML. When we built in support for ODF, we added it in that area of our code.

That suggests deep integration, and a converter that has good chances of working smoothly and quickly – unlike the clunky open source translator which Microsoft sponsored, which is based on XSLT.

Users also have the option of Sun’s plug-in, which is based on Open Office code.

It seems that Microsoft is aiming to make Office 2007 a good ODF citizen. Will that impede adoption of Microsoft’s own OOXML format? Well, I guess OOXML will still be the default, and will have the best support for Office-specific features. Another thing to bear in mind is that OOXML was designed with Office specifically in mind. Potentially troublesome creatures like very large Excel spreadsheets may well perform best with OOXML. Another point is that Microsoft’s server products, where they exploit XML documents, are likely to work best with OOXML. In other words, there are likely to be advantages in OOXML within enterprises that use Microsoft’s platform.

Still, strong support for ODF by default in Office will be a significant boost for the OASIS format. Microsoft is protecting its very profitable Office sales against the risk of being dropped for lack of ODF compatibility.

Jack Schofield at The Guardian links to Matthew Paul Thomas’ post on Why Free Software has poor usability.

For most of his points you could drop the free from his title.

Thomas makes the point about most programmers not being good designers:

Some programmers are also great designers, but most aren’t. Programming and human interface design are separate skills, and people good at both are rare. So it’s important for software to have dedicated designers.

True; but software can be over-designed too – I guess this happens more in the commercial world. Sometimes usability is sacrificed for the sake of good looks.

Mega-budgets don’t always help. How did Office 2007 get through its usability testing, when there is no indication that you have to click on the “Office button” to access essential features like New, Save As and Print? What about Outlook 2007, with one of the world’s most obscure and difficult UIs? Opening another user’s mailbox – how many clicks is that, and how would you possibly find it without being told?

Good code is hard; good design is harder.

I mentioned my annoyance that wi-fi with WPA encryption did not work on the Asus 901 Eee PC with Linux which I am reviewing. I later found that even a WEP connection, while it worked a bit, tended to drop out after few minutes. Worse still, the Asus repositories are messed up so getting updates is tricky.

I’ve managed to fix it. For the record, here’s what I did; though I can’t promise it will work for you; try it at your own risk. I should think there will be an easier way to install an updated driver soon.

One of the irritations about the repository problem is that attempted updates fill up your /dev/sda1. I wanted as much space as possible (without removing the recovery partition) so I reset the system with F9 on boot. Then I went into /etc/apt/sources.list and removed the default repositories, replacing them with these two:

deb http://xnv4.xandros.com/xs2.0/upkg-srv2 etch main contrib non-free

deb http://download.tuxfamily.org/eeepcrepos/ p701 main

(This is probably not a good arrangement long-term; but having the Asus repositories enabled right now makes it hard to install anything).

Then I did:

apt-get update

apt-get install build-essential

I also installed ksmserver and kicker to get access to the full desktop; but this wasn’t needed to get wi-fi working.

Next, I downloaded the latest driver source for the 901’s Ralink wi-fi card:

http://www.ralinktech.com.tw/data/drivers/2008_0708_RT2860_Linux_STA_v1.7.0.0.tar.bz2 (Update: don’t use this code - see Update 3 below).

Extracted it; then edited /os/linux/config.mk and changed:

HAS_WPA_SUPPLICANT=y

HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=y – but see update 2 and update 3 below for the problems this caused.

I also removed –DDBG from WFLAGS

WFLAGS += –DCONFIG_STA_SUPPORT

In order to compile you need the kernel source. I adapted the instructions here. In essence, download the source from http://dlsvr01.asus.com/pub/ASUS/EeePC/701/Linux_Kernel_071127.rar, extract, install the deb, unpack the source in /usr/src, delete the .bz2 to save space, build the kernel with:

make oldconfig

make

(don’t actually install it), then create the following symbolic link:

ln -s /usr/src/linux-source-2.6.21.4-eeepc /lib/modules/2.6.21.4-eeepc/build

Now I was able to compile and install the ralink driver by switching to the RT2860 directory and typing:

make

make install

Wi-fi now worked with WPA; at least, it did on my second attempt after disabling native WPA supplicant support. I’ve also found it perfectly stable so far – much better.

I’m happy now, but… what is Asus thinking?

I doubt the Eee brand would have ever taken off, except that the first 7xx releases made excellent use of Linux and open source applications to win people over; you did not even need to know that it was Linux.

Now we get this; the support for the 901 is dire; updates don’t work, the wi-fi doesn’t work properly, and the only fix I’ve found is a hack. Wi-fi is critical for a gadget like this, which is not much use without an Internet connection.

Together with short supply of the Linux version, this does suggest that Asus is keen to favour the Windows XP variant.

Update: I’ve made the binary (rt2860sta.ko) available for download here. Download includes RT2860STA.dat which goes in /etc/Wireless/RT2860STA. I guess we need an install script; I’ll have a go when I get a moment.

Update 2: Unfortunately I’m not quite there yet. If I compile with HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=y then WEP works but WPA does not. If I compile with HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=n then WPA works but (I now discover) WEP does not. Weird. I’m going to seek advice. If anyone wants the binary that works great with WEP but not WPA, let me know.

Update 3: Thanks to Ralink tech support I now have a fully working driver. I’ve updated the download above. The new code defines EEEPC_SPECIAL_SETTING and has some edits in common/mlme.c. It now works OK with HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=y. Interesting point: the code is dated 15th July; which suggests that the problem was both known and fixed 2 and half weeks ago; it’s a shame nobody shared this with the community.

Update 4: I suggest you delete the old copy of rt2860sta.ko in /lib/modules/2.6.21.4-eeepc/rt2860 (if it exists).

PS: I’ve posted instructions for the binary install, with download links, here.

The team behind the open source SCM (Software Configuration Management) tool Subversion released version 1.50 last month. Karl Fogel, president of the Subversion Corporation, says:

Measuring by new features alone, Subversion 1.5 is our biggest release since version 1.0 became available in February 2004.

I am a contented Subversion user, so took a look at the changes. Top of the list is “merge tracking”, though it is described as “foundational” which means that although the basic support is there, there is performance and feature work which remains to be done. From the user’s perspective, the difference is that branching and merging is just easier than before, as explained by Ben Sussman:

Notice how I never had to type a single revision number in my example: Subversion 1.5 knows when the branch was created, which changes need to be synced from branch to trunk, and which changes need to be merged back into the trunk when I’m done. It’s all magic now. This is how it should have been in the first place.

Other changes include sparse checkouts (when you only want to grab a small part of a repository), and changelists, a client feature which lets you tag a set of files under a changelist name and work on them as a group. There are also improvements aimed at making Subversion better suited to large-scale deployments using multiple servers. Subversion is still a centralized rather than a distributed SCM system, but 1.5 is better suited for use in a distributed manner. No doubt the Subversion team is aware of the increasing interest in Git, a distributed system. There are also numerous bug-fixes and performance tweaks. The changes are described here.

I want to move to Subversion 1.5 but it is not that easy. Compatibility is good, in that older clients work with 1.5 servers and vice versa, the main proviso being that you cannot mix several versions of the Subversion client with the same working copy. That is not likely to be a problem for most users.

The difficulty I encountered is that mainstream Linux distributions still have older versions of Subversion in their stable repositories. Ubuntu, for example, has version 1.4.4. My most-used Subversion repositories are on a Debian server, which also has an old version. I don’t want to switch the server to sid, Debian’s development distribution, and mixing packages is often problematic. I could do a manual installation I guess; but instead I will be patient.

I did install 1.50 on Windows, for an intranet repository. I used the Collabnet download. All I needed to do was to inform the installer of the location of my existing repository, and then copy a few lines from the old Apache 2.0 configuration file to the new Apache 2.2 configuration file. Everything works fine. I also updated TortoiseSVN on the Windows clients.

One of the advantages of Subversion (or any SCM) repositories over synched folders like those in Microsoft’s Live Mesh or Apple’s MobileMe (as I understand it) is that you get version history. I regard this as a key feature. The problem with synchronization is that you might overwrite a good copy with a bad one. It is easy to do; it might be caused by user error, or a bug in your word processor, or a failing hard drive. Automatic synch (un)helpfully replicates the bad copy all over. Versioning means you can just rollback to the good one.

Lengthy but worthwhile post on what is it like to write a technical book, from O’Reilly author Baron Schwartz.

Two things that particularly interest me here. One, it is a good account of how much work is involved in writing something worthwhile. Two, the discussion of authoring tools is fascinating: bugs and inadequacies in Open Office and Word.

I have done a couple of quick books; they were low-budget affairs and the process was simpler, but they were still a considerable labour. The first was well received and sold remarkably well. In the second, I misjudged the readership and it bombed. I’d like to do more but I need to block out a large chunk of time, as well as coming up with a winning idea and execution. I’m happy to take a chance on the latter, but finding time is the biggest problem.

For my books I used Pagemaker, and although there were a few annoyances, it never skipped a beat. I’d be quite inclined to use it again, even though it is deprecated in favour of InDesign. Pagemaker’s long document support is excellent. For collaborative authoring though, DocBook might be better.

Congratulations to the Wine community which has released Wine 1.0:

The Wine team is proud to announce that Wine 1.0 is now available. This is the first stable release of Wine after 15 years of development and beta testing.

Wine is a compatibility layer that enables Windows applications to run on Linux, Mac OS X, and other Unix-like systems.

Although in beta, Wine has been used in numerous commercial applications over the years. Two I can think of: WordPerfect 9 from Corel, and the Kylix IDE from Borland. More accurately, Kylix used Winelib, which uses the Wine API at compile-time rather than at runtime.

Another notable example is Google’s Picasa for Linux.

Despite this landmark, Wine seems less significant now than it did a few years back. For one thing, it is easier to write a cross-platform application. For another, web applications have grown in importance, reducing the number of native applications we need to run. Finally, virtualization is now a better, more compatible route to running Windows apps on Unix in many scenarios.

If you check out the installation instructions for SproutCore you will notice that Windows developers are not really catered for (though I am sure it can be done). I also noticed some Windows issues in the Sproutcore forum. More evidence of the Windows/Unix disconnect.

Having been impressed by VirtualBox recently, I figured it might be easier to use VirtualBox and Ubuntu.

Here’s how it went. I ran up a new VirtualBox image and installed Ubuntu. Very easy; just point VirtualBox to the Ubuntu install iso. This worked, except that Ubuntu did not recognize the video driver and defaulted to 800 x 600 resolution. Solution: install dkms in Synaptic (or using apt-get); then install the VirtualBox Linux Additions. That gave me 1024 x 768, which is good enough.

Next, you need to install Ruby (developer version) (use sudo or root)

apt-get install ruby

apt-get install rubygems

apt-get install ruby1.8-dev

and some build tools as one of the SproutCore dependencies (hpricot) calls make when installed:

apt-get install build-essential

I thought that would be enough, but my first go at installing SproutCore failed, apparently because gem is not on the path. Although it seemed to work, I got “command not found” when running sproutcore. So:

export PATH=$PATH:/var/lib/gems/1.8/bin

Now you can install SproutCore:

gem install sproutcore

Several components, such as hpricot, ask which version to install. I chose the variant designated (ruby).

After that, the Hello World tutorial ran exactly as expected.

By the way, although this example makes it look as if SproutCore is a dynamic server framework, it is not. The usual way to deploy a SproutCore application is as static files; the server arrangement is for debugging.

Note: the command above only amends the path for the current session. To make it permanent, I added the following to lines to the end of /etc/profile (needs root permissions to edit)*:

PATH=$PATH:/var/lib/gems/1.8/bin
export PATH

*See comment below for an even better suggestion.

There’s recently been a bit of hype (RoughlyDrafted.com) for Charles Jolley’s Sproutcore, yet another JavaScript framework, mainly because Apple is using it as its “Cocoa for the Web”, according to AppleInsider.

I tried the sample controls demo in IE7 but it didn’t work quite right. For example, the Picker pane opened but would not close. Tried again in Firefox 3.0 and everything was fine.

I’ve got no idea what the problem is with IE7; it is probably because of weak standards support in IE. However, it illustrates the advantages of a plug-in like Flash, Silverlight or Java. With these platforms, the application is largely insulated from differences between browsers.

The snag with the proprietary plug-in approach is that the vendor may not support every platform equally. Microsoft is entrusting the bulk of Silverlight Linux support to a third party. There are also issues of control. Apple most likely does not want any runtimes on iPhone because they open up a route to application deployment that bypasses its App Store and 30% revenue share. Google seems wary of Flash; RoughlyDrafted says that is because of the risk of content being turned into “opaque binaries” that are beyond the reach of its contextual advertising analysis, but it may just be  reluctance to cede such an important part of its platform to a third party.

Still, as a developer in search of a predictable app platform I’d rather target a plug-in than trust the browser vendors to be sufficiently consistent, and the Javascript libraries sufficiently smart, to enable my code to run reliably everywhere. It is easier to get away with a requirement for, say, Flash 9, than to insist that users choose a particular browser or operating system.

There are other factors of course. On the Javascript + HTML side, there are advantages in that it extends rather than replaces the HTML model. Things like clipboard support just work. Plus, it runs on iPhone.

On the plug-in side, you get the fast execution of a JIT compiler, and easy use of graphical and multimedia effects that take effort to do in JavaScript, or can’t be done at all.

I would be interested in comments from developers about what RIA platform you are choosing, and why.

This amused me:

In case you can’t see the image, it shows the release notes for the Google App Engine SDK, which I’m trying right now. The notes say:

Fixed behavior with source files that have Windows line-endings or missing line-endings.

With a nice self-referential touch, the notes reveal the very problem they are describing. On Windows, a line ending is usually encoded by two characters (carriage return and line feed), whereas on Unix (and similar systems like Linux and Mac OS 10) a single line feed character is used. Notepad is the default viewer for text files on Windows, and displays these Google release notes with scrambled formatting. A programmer’s editor like CodeWright¹ understands both conventions and the file displays fine.

It is a minor annoyance, but exposes a deeper problem. There are two factors at play here. One is that the open source world has tended to use Unix-like operating systems (maybe because open source operating systems are Unix-like); the other is that the favourite client OS among Web 2.0 geeks is the Mac. It’s likely therefore that most or all the folk working on App Engine do not use Windows and do not see this issue. They have taken the trouble to support Windows, but the Windows platform is just a little alien. The poor experience you get when double-clicking a readme is a symptom.

Another example is in the notes describing the options for the dev_appserver.py script, used to run the SDK. It says:

–datastore_path=PATH      Path to use for storing Datastore file stub data. (Default /tmp/dev_appserver.datastore)

What was that default again? There is no /tmp directory on Windows. I am sure the script uses a sensible default on Windows; but it will be different from what these notes say.

Let me add that the Unix directory structure is generally nicer to work with than Windows, with its archaic drive letters; and that I see little sense in the carriage return character being needed to end a line (it’s a throwback to the typewriter). Windows is also more wrong that right about using the backslash as a path divider (all IBM’s fault, apparently).

As a developer, I perceive these things as a small nudge that I might not be using the best OS for the task in hand. That’s unfortunate for Microsoft, and I’m not sure how it can fix it - though getting Notepad to respect Unix line endings would be a start.

¹CodeWright is/was an excellent editor that was acquired and killed by Borland; unfortunately it does not work properly on Vista though I still find it handy occasionally.

Microsoft’s John Lam reports:

IronRuby dispatched some simple requests through an unmodified copy of Rails a few days ago. Today, we’re going to show off our progress live at RailsConf.

He adds that performance is terrible; so you might not want to migrate your project just yet. Why bother? Mainly, to get Rails productivity plus access to .NET libraries - in other words, integration with Microsoft’s platform.