All posts by onlyconnect

Bringing paper and ebook together

There’s a buzz building about a session at the O’Reilly Tools of Change for Publishing Conference (which looks great, I wish I could have attended) from Manolis Kelaidis on his “blueBook”. Kelaidis is a designer at the Royal College of Art in London. His idea is to bring electronics to the book, rather than making books virtual. Here is the brief, from his presentation:

  • Design a book consisting of sheets of paper with printed buttons (hyperlinks), which when touched allow the user to access and control digital information.
  • Information accessed in this way could then be stored either locally (within the book) or remotely (PC, handheld devices, Web, other books).
  • The book should have the look-and-feel of a regular book, with flipping pages and conventional binding, while technology should be non-intrusive, portable and robust.
  • Manufacturing should be based on traditional bookbinding techniques minimizing complexity and costs.

Using conductive ink and embedded electronics, such a book can include multimedia (play music or video) and live links to web content or interactive discussions, perhaps in conjunction with a wireless-connected PC. Printed words become hyperlinks. The presentation is amazing and thought-provoking. See also Tim O’Reilly’s post on the subject and its comments.

Update: Presentation seems to have been removed; if it reappears I imagine it will do so here.

Technorati tags: , , ,

SQLite, test-driven development, and the inscrutable SQL standard

I interviewed Dr D Richard Hipp, the main author of SQLite, for the Guardian Newspaper.

Among the things I found interesting is that he attributes the high reliability of his database engine to the extensive test suite included in the code. I’m not sure whether he practices test-driven development as such, but it is a great case study for the advantages of integrating tests with your code. One of the points he made was that the test suite enables him to replace entire subsystems and be confident that nothing gets broken. By contrast, I have heard of cases where key sections of code in large, old applications is marked “do not touch” because nobody dares to risk the consequences.

I also asked him about the importance of standards in software development. He gave me an answer that somewhat surprised me:

When I coded up SQLite I did not refer to any official SQL standard. I used the PostgreSQL documentation. That was my reference. If you’ve ever picked up a copy of one of the official SQL standards you will find it largely inscrutable. They are next to impossible to make sense of. Even for particular details of syntax you can study it, and they are so vague that you can’t really understand what they mean. So a strategy we’ve used when there’s some question about how something should work is we write a little test script and run it on lots of popular SQL database engines, PostgreSQL, MySQL, Oracle, and try and find a consensus. Then we code to make SQLite work the same as everybody else does. Clearly that’s not the right way to do a standard, but in practice the implementations vary so widely that it’s the only practical thing to do.

Many companies boast about how they respect and observe software standards; sometimes the reality is more pragmatic than you might have thought.

CD swapping with HitFlip

I took a look at the CD swapping site hitflip.co.uk, after seeing it recommended in The Independent newspaper.

Here’s the deal:

Disposing of CDs

You list all your unwanted CDs (DVDs, games, etc). Each one is assigned a value in “flips”, the currency of hitflip. Currently a flip is worth £2.30; CDs tend to be assigned from 2 flips up. If someone wants a CD, they request it, and you send it off to them. You get the value in flips credited to your account. You can’t cash in your flips; you have to use them to acquire other items.

Acquiring CDs

You browse the library for an item you want. When you find one, you request it. If it is immediately available, it is posted to you. You pay the value in flips, plus a cash fee to hitflip, currently £0.79 for a low value item, increasing for more valuable items.

My Hitflip review

I don’t much like it. Main problem: there’s not much immediately available that you are likely to want. The obvious risk is that you merrily post off all your best items, then find there is nothing you want that is actually available. You get stuck with a pile of useless flips.

The problem is that you are not really swapping. Instead, you are selling for virtual money. That’s more flexible than real swapping, but lacks the advantage of a real swap, where you do not approve the trade unless you get something you want in return.

There seem to be several flaws in the hitflip system:

  • You can’t set your own price in flips, you have to accept what the system assigns.
  • There’s no way I can see to describe your item, its condition, special features etc. Everything is just meant to be in good condition.
  • You can run a wish list, but you can’t add items of your own description, only select from the hitflip library which is not remotely comprehensive.
  • Privacy: it appears that you can get anyone’s address by offering an item on their wishlist. I realise that you need someone’s address in order to send them something (duh!). But I’d have thought there should be an approval step. X is offering the item, do you want to accept? If X is new, or has a bad community rating, you might want to decline.
  • Bugs: The system originated in Germany, and I found German language messages popping up now and again.
  • User interface: browsing the library is laborious and the UI overall is not great.

It’s not all bad. It all hinges on finding items you want. If you succeed, you get them in return for your unwanted items at the cost of the fee.

Overall though, I find it hard to recommend.

Technorati tags: ,

The declining price of recorded music

I recently purchased a second-hand copy of Physical Graffiti, Led Zeppelin’s tour de force from 1975.

These sell for anything from £1.50 – £8.00 or so on eBay. Mine was the old unremastered release which usually sells at the upper end of the range, as there are fewer of them circulating and some people prefer the sound.

I was interested to find the original sales receipt inside. The double CD was purchased from HMV in Portsmouth in 1993. The price: £22.99.

Prices have risen by about 40% in that time. So £22.99 in 1993 equates to about £32.00 today.

Amazon UK has this CD (remastered) new for £7.97 – 25% of the 1993 price at HMV.

OK, so bricks and mortar shops are more expensive than online, and CDs are often reissued at budget price. So try a new double CD: Bruce Springsteen’s Live in Dublin is £11.98 at Amazon UK, or complete with a DVD at £17.99, still way below the 1993 price.

It is not just the decline in CD sales that is hurting the music industry, but the fact that they are selling for much less money.

How Microsoft changed its mind about Office XML standardization

My interview with Microsoft’s XML Architect Jean Paoli back in April was not the first time I had spoken to him. I also talked to him in February 2005. At that time Microsoft had no intention of submitting its Office XML specification to a standards body. I thought it should do so, and asked Paoli why not:

Backward compatibility. We have today 400 million users of Office, which means billions of documents. So we went and did a huge job of documenting electronically all these features and we put that into this WordML format. Well we need to maintain this damn thing, and we need to maintain this big format, we have like 1500 tags. Who is going to maintain that? A standard body? It doesn’t know what is inside of Word. That’s the problem. So we said we are going to give you a license, open and free… [Jean Paoli, February 2005].

Microsoft was forced to change its mind, because important customers (mostly governments) indicated their preference for standardised document formats. The quote remains relevant, because it says a lot about the goals of Office Open XML, which is an evolution of WordML and SpreadsheetML.

While on the subject, I also want to mention Simon Jones’ piece in the August 2007 PC Pro (article not online), perhaps a little one-sided but he does a good job of debunking some of the common objections to OOXML and exposing some of the politics in the standardisation process. He adds:

I’m not saying there aren’t any problems with the ECMA-376 standard. Nor am I saying ODF is bad. I do, however, believe OOXML is technically superior to ODF in many ways, and I want to see both as ISO standards so people can have the choice.

Technorati tags: , , , ,

The version problem of today: browser compatibility

David Berlind reports on a case where 35% of developer time is spent on browser compatibility issues.

It’s a huge problem, though I’m cautious about attaching too much weight to a singe anecdotal report. Of course it’s nothing new. Browser compatibility issues are as old as the Web; it was getting better, until AJAX and a new focus on the web-as-platform meant greater stress on advanced browser features. For that matter, version issues are as old as computing. Yesterday, DLL Hell. Today, web browsers.

What’s the solution? All use the same browser? Not realistic. The browser developers could fix the incompatibilities? It’s happening to some degree, but even if Microsoft came out with a 100% FireFox-compatible IE8 tomorrow, there’s still a big legacy problem. My web site stats for this month:

IE7 24%

IE6 22%

IE5 4%

FireFox 2.x 22%

FireFox 1.x 3%

Opera 3.9%

Safari 2.3%

etc

Interesting that the FireFox folk seem to upgrade more quickly than those on IE – but even so, there are a lot of older browsers still in use. I suspect a lot of those IE6 users are corporates with conservative upgrade policies.

Another idea is to use AJAX libraries that hide the incompatibilities. That makes a lot of sense, though if you stress the libraries you might still find compatibility issues.

Finally, you can bypass the browser and use some other runtime, most likely Java or Flash. Unfortunately this doesn’t remove all version issues, but at least it means you are mainly dealing with one vendor’s evolving platform (Sun or Adobe). Silverlight could help as well, though its “cross-platform” only means Windows or Intel Mac at the moment, which is not broad enough.

This will be an important factor in the RIA (Rich Internet Application) wars.

Office Open XML vs COM automation

Looking at the new Open XML API, introduced by Kevin Boske here, makes you realise that old-style COM automation wasn’t so bad after all.

There are two distinct aspects to working programmatically with OOXML. First, there’s the Packaging API, which deals with how the various XML files which make up a document get stored in a ZIP archive. Second, there’s the XML specification itself, which defines the schema of elements and attributes that form the content of an OOXML document.

The new wrapper classes really only deal with the packaging aspect. You still have to work out how to parse and/or generate the correct XML content using your favourite XML parser. And it’s a lot more complex then HTML.

By contrast, the old COM automation API for Office presents a programmatic object model for the content, and you don’t have to worry much about how the document gets stored – you just tell Word or Excel to save it.

The (very big) downside of the COM object model is that it depends on the presence of Microsoft Office. High resource requirements, version problems, Windows-only, and inappropriate for server apps.

We seem to have traded one problem for another. What Microsoft needs to provide is wrapper classes for the content, rather than just its packaging.

Technorati tags: , , , ,

Why doesn’t Adobe’s AIR dev guide mention SQLite?

I’ve been trying out the Adobe AIR (formerly Apollo) SDK.  It’s a confusing business. There are two varieties of AIR apps, Flex, or HTML. The HTML kind is essentially a browser app that runs in WebKit, as wrapped by the AIR runtime, instead of in the browser, while the Flex kind compiles Adobe’s MXML into a Flash SWF which again runs within AIR. The AIR SDK only supports HTML AIR apps, so for the full experience you also need the Flex 3 beta SDK.

But I digress. I have a long-standing interest in SQLite so one of the first things I looked for was how Adobe is using this in AIR. It is there: it’s mentioned in the press release, which emphasizes that AIR has some of that open source fairy dust:

Key elements of Adobe AIR are open source, including the WebKit HTML engine, the ActionScript™ Virtual Machine (Tamarin project) and SQLite local database functionality.

However, you wouldn’t know it from the docs. The word SQLite does not appear in either the Flex or the HTML developer guides. Here’s how it introduces the “local SQL databases” section:

Adobe Integrated Runtime (AIR) includes the capability of creating and working with local SQL databases. The runtime includes a SQL database engine with support for many standard SQL features.

The SQLite library itself appears to be compiled into the main AIR runtime library, Adobe AIR.dll.

Why do I mention this? A few reasons.

First, it stinks. Let me emphasize: Adobe is entirely within its rights in not crediting SQLite in its docs. The main author of SQLite, Dr D Richard Hipp, has disclaimed copyright. So it is not illegal, but it is discourteous. By contrast, here’s how the Google Gears docs introduce the database module:

The Database module provides browser-local relational data storage to your JavaScript web application. Google Gears uses the open source SQLite database system.

Second, it’s unhelpful. As a developer familiar with SQLite, I want to see an explanation of how Adobe’s build of SQLite differs from what I am used to – what is added, what if anything is taken away. I also need to know how easily I can access the same database from both AIR and from another application, using the standard SQLite library.

Third, I’m increasingly sceptical of Adobe’s claim that it is somehow “aligning” its API in AIR with that in Gears. Here’s what Michele Turner, Adobe’s VP of developer relations, told me:

Adobe, Google, Mozilla and others will be working to align the APIs used to access local database storage for offline applications, so this functionality will be consistent for developers both in the browser and via Apollo on the desktop.

Perhaps, but there’s really no sign of this in the current beta. The AIR database API and the Gears API are totally different. The full text search extension which is part of Gears seems to be missing in AIR. Another key difference is that unlike Gears, AIR makes no attempt to isolate databases based on the origin of the application. In AIR, a SQLite database may be anywhere in the file system, and it’s equally available to any AIR application – a big hole in the AIR sandbox.

This is all beta, of course, so it can change. I hope it does. Here’s my wish list:

  • Proper credit for SQLite in the docs.
  • Use the Gears code – full text search could be very useful – and deliver on the promise of aligning the API.
  • Failing that, set out exactly how AIR’s SQLite differs from the standard build.
Technorati tags: , , ,

The problem of old Java runtimes

The August PC Pro arrived this morning, and I enjoyed Steve Cassidy’s rant (page 174) on old versions of Java that typically litter PCs:

I’ve made it my habit to go round all the LAN’s I visit removing all older versions of Java from the machines, because the Java updater doesn’t remove them automatically.

It reminded me that I’d intended to post about this dialog, encountered when installing Accurev for a short review:

The decision here is whether to let AccuRev install its own version of the JRE (Java Runtime Environment), or to use one you already have, in which case you have to identify it. It’s a tough decision. If you follow the recommendation to install a private version, you end up with multiple different versions of Java which will likely never get updated except by the application vendor, if indeed you choose to upgrade. I understand why vendors do this: it simplifies testing and installation, and gives apps a predictable platform on which to run.

Unfortunately the downside is substantial too. In the AccuRev case it was slightly unfortunate, since the supplied JRE was incompatible with Vista and broke Aero graphics. A more painful example was when the JRE installed with APC’s PowerChute utility failed because of an expired cryptographic certificate; the consequences were extreme, and in many cases affected systems would no longer boot. See here for the gory details.

I prefer the way Microsoft handles the .NET runtime, where more than one version can be installed, but they are system files for which Microsoft takes responsibility through Windows Update. Sun installs an updater with its JRE that works for web browsers and other applications that use a shared JRE, but there are still many apps like AccuRev that install private versions.

Technorati tags: , , ,

How to speed up Vista: disable the slow slow search

What’s the biggest problems with Vista? Not the buggy drivers, which are gradually getting sorted. Not the evil DRM, which I haven’t encountered directly, though it may be a factor in increasing the complexity and therefore the bugginess of video and audio drivers. Not User Account Control security, which I think is pretty good. Not the user interface, which I reckon improves on Windows XP though there are annoyances.

No, my biggest complaint is performance. This morning I noticed that if I clicked the Start button and then Documents, it took around 15 seconds for the explorer window to display, fully populated. Doing this with Task Manager monitoring performance, I could see CPU usage spike from below 10% to between 55% and 60% while Explorer did its stuff.

Explorer gets blamed for many things that are not really its fault. Applications which integrate with the desktop, such as file archive utilities, hook into Explorer and can cause problems. I tried to figure out what was slowing it down. I opened up Services (in Administrative Tools) and looked at what was running. It didn’t take long to find the main culprit – Windows Search:

Windows Search in Services

You will notice that the above dialog shows that the service is not running. That’s because I stopped it. The difference is amazing. The Documents folder now shows in less than a second. When I click the Start button, the menu displays immediately instead of pausing for thought. Everything seems faster.

Looking at the description above, it is not surprising that there is a performance impact. The indexer gets notified every time you change a file or receive an email (if you are using Outlook or Windows Mail). The same service creates virtual folder views in Explorer, a poor man’s WinFS that should make the real location of files less important. Notice that the explanatory text warns me that by stopping the service I lose these features and have to “fall back to item-by-item slow search”.

I think it should say, “If the service is started, Explorer will take fifteen times longer to open and your system will run more slowly.”

Desktop search is a great feature, but only if it is unobtrusive. In Vista, that’s not the case.

This kind of thing will vary substantially from one system to another. Another user may say that Windows Search causes no problems. I also believe that the system impact is much greater if the indexer has many outstanding tasks – such as indexing a large Outlook mailbox, for example. Further, disabling Windows search really does slow down the search function in Explorer.

Turning off Windows search is therefore not something to do lightly. It breaks an important part of Vista.

Still, sometimes you need to get your work done. That fifteen seconds delay soon adds up when repeated many times.

In truth, we should not be faced with this decision. Microsoft should know better – it has plenty of database expertise, after all. There’s no excuse for a system service that slows things down to this extent.

By the way, if you have understood all the caveats and still want to run without Windows Search, until Microsoft fix it, then you must set the service to disabled. Otherwise applications like Outlook will helpfully restart it for you.

Update

See comments below – a couple of others have reported (as I expected) that search works fine for them. So what is the issue here? In my case I think it is related to Outlook 2007, known to have performance problems especially with large mailboxes like mine. But what’s the general conclusion? If you are suffering from performance problems with Vista, I recommend experimenting with Search – stop and disable it temporarily, to see what effect it has. If there’s no improvement, you can always enable it again.

It strikes me that there is some unfortunate interaction between Explorer, Search, and Outlook; it’s possible that there are other bad combinations as well.

Technorati tags: , , ,