Category Archives: software development

software, software development

SQLite, test-driven development, and the inscrutable SQL standard

2 Comments

I interviewed Dr D Richard Hipp, the main author of SQLite, for the Guardian Newspaper.

Among the things I found interesting is that he attributes the high reliability of his database engine to the extensive test suite included in the code. I’m not sure whether he practices test-driven development as such, but it is a great case study for the advantages of integrating tests with your code. One of the points he made was that the test suite enables him to replace entire subsystems and be confident that nothing gets broken. By contrast, I have heard of cases where key sections of code in large, old applications is marked “do not touch” because nobody dares to risk the consequences.

I also asked him about the importance of standards in software development. He gave me an answer that somewhat surprised me:

When I coded up SQLite I did not refer to any official SQL standard. I used the PostgreSQL documentation. That was my reference. If you’ve ever picked up a copy of one of the official SQL standards you will find it largely inscrutable. They are next to impossible to make sense of. Even for particular details of syntax you can study it, and they are so vague that you can’t really understand what they mean. So a strategy we’ve used when there’s some question about how something should work is we write a little test script and run it on lots of popular SQL database engines, PostgreSQL, MySQL, Oracle, and try and find a consensus. Then we code to make SQLite work the same as everybody else does. Clearly that’s not the right way to do a standard, but in practice the implementations vary so widely that it’s the only practical thing to do.

Many companies boast about how they respect and observe software standards; sometimes the reality is more pragmatic than you might have thought.

internet, software development, web authoring

The version problem of today: browser compatibility

1 Comment

David Berlind reports on a case where 35% of developer time is spent on browser compatibility issues.

It’s a huge problem, though I’m cautious about attaching too much weight to a singe anecdotal report. Of course it’s nothing new. Browser compatibility issues are as old as the Web; it was getting better, until AJAX and a new focus on the web-as-platform meant greater stress on advanced browser features. For that matter, version issues are as old as computing. Yesterday, DLL Hell. Today, web browsers.

What’s the solution? All use the same browser? Not realistic. The browser developers could fix the incompatibilities? It’s happening to some degree, but even if Microsoft came out with a 100% FireFox-compatible IE8 tomorrow, there’s still a big legacy problem. My web site stats for this month:

IE7 24%

IE6 22%

IE5 4%

FireFox 2.x 22%

FireFox 1.x 3%

Opera 3.9%

Safari 2.3%

etc

Interesting that the FireFox folk seem to upgrade more quickly than those on IE – but even so, there are a lot of older browsers still in use. I suspect a lot of those IE6 users are corporates with conservative upgrade policies.

Another idea is to use AJAX libraries that hide the incompatibilities. That makes a lot of sense, though if you stress the libraries you might still find compatibility issues.

Finally, you can bypass the browser and use some other runtime, most likely Java or Flash. Unfortunately this doesn’t remove all version issues, but at least it means you are mainly dealing with one vendor’s evolving platform (Sun or Adobe). Silverlight could help as well, though its “cross-platform” only means Windows or Intel Mac at the moment, which is not broad enough.

This will be an important factor in the RIA (Rich Internet Application) wars.

microsoft, software development, windows

Office Open XML vs COM automation

2 Comments

Looking at the new Open XML API, introduced by Kevin Boske here, makes you realise that old-style COM automation wasn’t so bad after all.

There are two distinct aspects to working programmatically with OOXML. First, there’s the Packaging API, which deals with how the various XML files which make up a document get stored in a ZIP archive. Second, there’s the XML specification itself, which defines the schema of elements and attributes that form the content of an OOXML document.

The new wrapper classes really only deal with the packaging aspect. You still have to work out how to parse and/or generate the correct XML content using your favourite XML parser. And it’s a lot more complex then HTML.

By contrast, the old COM automation API for Office presents a programmatic object model for the content, and you don’t have to worry much about how the document gets stored – you just tell Word or Excel to save it.

The (very big) downside of the COM object model is that it depends on the presence of Microsoft Office. High resource requirements, version problems, Windows-only, and inappropriate for server apps.

We seem to have traded one problem for another. What Microsoft needs to provide is wrapper classes for the content, rather than just its packaging.

Technorati tags: , , , ,
internet, software development, web authoring

Why doesn’t Adobe’s AIR dev guide mention SQLite?

8 Comments

I’ve been trying out the Adobe AIR (formerly Apollo) SDK.  It’s a confusing business. There are two varieties of AIR apps, Flex, or HTML. The HTML kind is essentially a browser app that runs in WebKit, as wrapped by the AIR runtime, instead of in the browser, while the Flex kind compiles Adobe’s MXML into a Flash SWF which again runs within AIR. The AIR SDK only supports HTML AIR apps, so for the full experience you also need the Flex 3 beta SDK.

But I digress. I have a long-standing interest in SQLite so one of the first things I looked for was how Adobe is using this in AIR. It is there: it’s mentioned in the press release, which emphasizes that AIR has some of that open source fairy dust:

Key elements of Adobe AIR are open source, including the WebKit HTML engine, the ActionScript™ Virtual Machine (Tamarin project) and SQLite local database functionality.

However, you wouldn’t know it from the docs. The word SQLite does not appear in either the Flex or the HTML developer guides. Here’s how it introduces the “local SQL databases” section:

Adobe Integrated Runtime (AIR) includes the capability of creating and working with local SQL databases. The runtime includes a SQL database engine with support for many standard SQL features.

The SQLite library itself appears to be compiled into the main AIR runtime library, Adobe AIR.dll.

Why do I mention this? A few reasons.

First, it stinks. Let me emphasize: Adobe is entirely within its rights in not crediting SQLite in its docs. The main author of SQLite, Dr D Richard Hipp, has disclaimed copyright. So it is not illegal, but it is discourteous. By contrast, here’s how the Google Gears docs introduce the database module:

The Database module provides browser-local relational data storage to your JavaScript web application. Google Gears uses the open source SQLite database system.

Second, it’s unhelpful. As a developer familiar with SQLite, I want to see an explanation of how Adobe’s build of SQLite differs from what I am used to – what is added, what if anything is taken away. I also need to know how easily I can access the same database from both AIR and from another application, using the standard SQLite library.

Third, I’m increasingly sceptical of Adobe’s claim that it is somehow “aligning” its API in AIR with that in Gears. Here’s what Michele Turner, Adobe’s VP of developer relations, told me:

Adobe, Google, Mozilla and others will be working to align the APIs used to access local database storage for offline applications, so this functionality will be consistent for developers both in the browser and via Apollo on the desktop.

Perhaps, but there’s really no sign of this in the current beta. The AIR database API and the Gears API are totally different. The full text search extension which is part of Gears seems to be missing in AIR. Another key difference is that unlike Gears, AIR makes no attempt to isolate databases based on the origin of the application. In AIR, a SQLite database may be anywhere in the file system, and it’s equally available to any AIR application – a big hole in the AIR sandbox.

This is all beta, of course, so it can change. I hope it does. Here’s my wish list:

  • Proper credit for SQLite in the docs.
  • Use the Gears code – full text search could be very useful – and deliver on the promise of aligning the API.
  • Failing that, set out exactly how AIR’s SQLite differs from the standard build.
Technorati tags: , , ,
java, software development

The problem of old Java runtimes

The August PC Pro arrived this morning, and I enjoyed Steve Cassidy’s rant (page 174) on old versions of Java that typically litter PCs:

I’ve made it my habit to go round all the LAN’s I visit removing all older versions of Java from the machines, because the Java updater doesn’t remove them automatically.

It reminded me that I’d intended to post about this dialog, encountered when installing Accurev for a short review:

The decision here is whether to let AccuRev install its own version of the JRE (Java Runtime Environment), or to use one you already have, in which case you have to identify it. It’s a tough decision. If you follow the recommendation to install a private version, you end up with multiple different versions of Java which will likely never get updated except by the application vendor, if indeed you choose to upgrade. I understand why vendors do this: it simplifies testing and installation, and gives apps a predictable platform on which to run.

Unfortunately the downside is substantial too. In the AccuRev case it was slightly unfortunate, since the supplied JRE was incompatible with Vista and broke Aero graphics. A more painful example was when the JRE installed with APC’s PowerChute utility failed because of an expired cryptographic certificate; the consequences were extreme, and in many cases affected systems would no longer boot. See here for the gory details.

I prefer the way Microsoft handles the .NET runtime, where more than one version can be installed, but they are system files for which Microsoft takes responsibility through Windows Update. Sun installs an updater with its JRE that works for web browsers and other applications that use a shared JRE, but there are still many apps like AccuRev that install private versions.

Technorati tags: , , ,
apple, internet, software development, web authoring

Apple iPhone needs Google Gears

1 Comment

At its developer conference Apple announced that the forthcoming iPhone will support Web 2.0 applications. In this context, “Web 2.0” means at a minimum an embedded web browser (Safari) that runs JavaScript, but that’s no big deal; we expected nothing less. It’s at least a little more than that though:

Developers can create Web 2.0 applications which look and behave just like the applications built into iPhone, and which can seamlessly access iPhone’s services, including making a phone call, sending an email and displaying a location in Google Maps.

The emphasis is mine. This implies some sort of hole in the sandbox, but web apps on the iPhone needs more than just the ability to make phone calls if they are going to be useful. They need to work offline. In fact, a mobile phone (ironically) is one environment where offline web apps will be particularly useful. Nobody is always-on when travelling; it varies from mostly on (urban travel) to mostly off (trains, planes). As a regular train traveller, I find attempting to run web apps on a mobile utterly frustrating.

Fortunately Google has come up with an answer to this with its Gears initiative. Here’s how you write a good Gears app:

  1. Write your app to work offline.
  2. Add synchronization with the server that happens transparently when connected.

This is perfect for a mobile app. Running web apps rather than local apps also bypasses one of the main obstacles to mobile development: the need to get your binaries approved by a telecom provider before they can be installed.

Now, I have no idea whether Apple plans to include Google Gears, or an equivalent, in the iPhone (I’m not at WWDC). But I do think it is a great idea, for this or any mobile device. Combine it with Flash or Silverlight and we will wonder why we ever wanted more.

internet, software, software development, web authoring

Google’s new model of app development

3 Comments

I was fascinated by this slide shown at the recent global developer day, which I’m reproducing with Google’s permission:

Four blocks captions Ads, Standards, Mashups, Open Source

The image doesn’t make sense without the caption, which I’ve used as the title of this post: The New Model of App Development. You can see the slide in context in this Register piece. Two things in particular interest me. One is the appearance of ads as an integral part of the development model. This makes sense for Google’s own development, but does it make sense for others? Given that much of the software industry is slogging away at internal business applications, that seems a stretch. It may be true for consumer apps. Ad-funded applications have not been a big success on the desktop, but we have somehow become tolerant of ads flashing round the screen when working on the Web.

Another issue is one we tried to capture in the caption for this image at the Reg. The main goal of developer day was to get developers to integrate Google services into their applications, by using Google Maps and the other APIs on show at Google code. The company is even keen to host your gadgets on its own servers. Google wants to be an indispensable building block in app development, even though it left itself out of the illustration.

How about open source? Google uses and sponsors open source software, and has posted the code for Gears, but where’s the code for Docs & Spreadsheets? Closed source is an important part of Google’s own app development model, as it is for most others.

internet, software development, web authoring

Adobe Live in London – quick report

1 Comment

I attended Adobe Live yesterday. This was in two parts, an exhibition with a schedule of presentations/tutorials, and a developer day focused on Rich Internet Applications – Apollo and Flex – as well as ColdFusion 8.

I’m told that around 1800 attended, though most of these were for the main exhibition, whereas I spent almost the whole day in the developer sessions.

It was a worthwhile though low-key event and I picked up some insights into Apollo and Flex 3 as well as getting an update on ColdFusion. I’ll be reporting in more detail shortly. A couple of quick comments though.

First, this struck me as primarily a designer crowd, Flash folk interested in applications, rather than developers new to Adobe. These are just impressions so I could be wrong, but it strikes me as in interesting issue. Equally, when I went along to Microsoft’s Mix07 earlier this year my perception was that many delegates were primarily Visual Studio folk interested in web design, rather than vice versa. If I’m right, Adobe and Microsoft have inverse cultural and marketing problems here. Still, at least Adobe put on a proper developer schedule this year; it didn’t exist at last year’s London event.

Second, I found the Apollo stuff though-provoking in the light of Google’s Gears announcement as well as what Microsoft is offering with WPF. I knew that Apollo was sandboxed, but hadn’t appreciated the extent of the sandboxing. As I understand it, Apollo apps can do file i/o on the local machine, but in other respects it is locked down in a similar way to web apps running in the browser. There is no access to external libraries, OS scripting, or custom native code extensions.

That’s good from a security perspective, but it limits the extent of OS integration. So the key question: if you can’t integrate with the OS, beyond a few trivia like Start menu or Dock shortcuts, then why bother with a desktop app? Especially now that Gears has a solution to the offline problem. Maybe it is worth it just to get out of the browser window, but some at least will not see the point.

Third, I asked what if anything Adobe intends to do with Google’s open-source Gears code, but apparently I may not have the full story yet – more when I get the information.

Technorati tags: , , , , ,
internet, software, software development, web authoring

FireFox team not sure about Google Gears adoption

3 Comments

During Google Developer Day I had the impression that Mozilla was right on board with Google Gears, the plug-in which which enables offline applications. Here’s Aaron Boodman and Erik Arvidsson from the Gears team:

We are releasing Gears as an open source project and we are working with Adobe, Mozilla and Opera and other industry partners to make sure that Gears is the right solution for everyone.

It seems that things are not as clear-cut as Boodman and Arvidsson imply. LinuxWorld.com quotes Mozilla’s Mike Shaver:

We’re talking to Google engineers and looking at how these two models — ours and theirs — compare. This is in the open now, and going forward we’ll see what we can learn from each other,” Shaver said. “But there’s a lot of work that’s been done already [on Firefox 3.0], and we’re not planning to throw that work away.

According to the article, Shaver is particularly doubtful about using SQLite for persisting web application data locally, and is inclined to stick with the more limited DOM Storage API already planned for FireFox 3.0.

A related point is that Adobe’s commitment to Gears is not absolute. Here’s Adobe’s Michele Turner says, quoted by David Berlind:

…For example, they’re using SQLite and we were already incorporating SQLite into Apollo. So, now we’re aligning our efforts with Google on things like the synchronous and asynchronous calls that must be made to the SQLite database in order to enable the offline capability. 

My impression is that Adobe is aiming for a measure of API compatibility, but will ship its own build of SQLite rather than using one installed by Gears, with inevitable version and customization differences.

There is a world of difference between using a similar API on the one hand, and sharing common components and/or common source code on the other. It looks to me as if Apollo, FireFox and Google are going to provide three independent and isolated mechanisms for handling local storage.

Of course Gears installs into FireFox anyway, and there is nothing to stop Flash developers from using Gears.

I will quiz Adobe on this subject at tomorrow’s Adobe Live event.

internet, software development, web authoring

Google: Don’t let your kids use Gears

8 Comments

More Google Gears Terms and Conditions madness. Gears is licensed under New BSD terms; yet before you can install the runtime you have to agree to onerous terms and conditions. Here’s clause 2:

2. Accepting the Terms

2.1 In order to use Google Gears, you must first agree to the Terms. You may not use Google Gears if you do not accept the Terms.

2.2 You can accept the Terms by:

(A) clicking to accept or agree to the Terms, where this option is made available to you by Google in the user interface for any Services; or

(B) by actually using Google Gears. In this case, you understand and agree that Google will treat your use of Google Gears as acceptance of the Terms from that point onwards.

2.3 You may not use Google Gears and may not accept the Terms if (a) you are not of legal age to form a binding contract with Google, or (b) you are a person barred from receiving the Services under the laws of the United States or other countries including the country in which you are resident or from which you use the Services.

I’m puzzled. If Gears is BSD licensed, how can Google insist that the mere act of using it binds me to these terms (which I dislike for other reasons too)? And 2.3 is bewildering: you may not use Google Gears apps if you are not an adult?

What if someone else installs Gears on your machine, and you then use a Gears-enabled app? How can terms like this possibly apply in such cases? Note that the agreement does not refer only to installing Gears, but specifically to using Gears.

By the way, you can download the source for Gears, and compile it if you can figure out how, without assenting to any such agreement.

I think Google is letting its legal team get out of hand.

Technorati tags: , ,