Category Archives: open source

ThoughtWorks bemoans excessive software complexity, advocates small, focused services

ThoughtWorks has released its latest Technology Radar, an opinionated analysis of software development trends.

Things the folk at ThoughtWorks like include automated build and deployment, essential for Continuous delivery; NOSQL database managers especially Neo4j; mobile-first development; the AppCode IDE for Apple’s Objective-C; the Graphite realtime graphing tool for creating dashboards; Clojure and Scala for programming.

I meet some of the ThoughtWorks team at developer conferences from time to time, and generally find them smart and though-provoking to talk to. They must be the despair of the big enterprise software vendors, with a liking for open source and an aversion to heavyweight high-maintenance systems.

This remark particularly caught my eye:

Simple architectures—Simple continues to gain traction, including both techniques for building and composing applications, as well as infrastructure-based techniques to enable simple deployment, failover and recovery. This theme is a recurring one for us, but we have not yet seen the usage shifts we believe are necessary.

I asked consultant James Lewis and practice lead Sam Newman to expand on that. Why do we continue to choose complexity over simplicity?

“A lot of people like to stay inside their big box, and don’t understand the complexities that then creates,” said Newman. “There’s a lack of critical thought given to how services talk to each other. A lot of them are driven by whatever the vendor says you do. Java makes RMI very easy. [Microsoft] .NET makes binding to WSDL [SOAP] schemas very easy. All these tools make bad things very easy to do.

“When you start talking to organisations about smaller services that are focused on doing one thing well, they have the horror associated with now having more than one box to manage and operate. So it’s hard to talk about moving from one big monolithic complicated box that is hard to change, to lots of little boxes, without also having conversations with those clients about how they get better at managing multiple services.

“Netflix has 300 services. Each service runs on at least six machines. They are very good at deploying those services. Yet they are not an overly complicated domain compared to some of our clients.”

“Amazon were talking about this in 2004,” adds Lewis, “the idea that you join up development of these small, simple applications with the operational control, so the same people who build them are also the people who run them. Now that we’re seeing both private and public clouds, and the ability to spin up machines becoming more and more prevalent, its starting to become more attractive.”

This is a consistent theme from ThoughtWorks. Break up complex solutions into many small services, think about how they talk to each other (with REST and HTTP favoured), and think about the infrastructure and how to automate it as well as the software itself.

“In many organisations these conversations are happening,” Newman told me. “I go to clients now, and they talk about the Enterprise Service Bus as being something they’d love to get rid of if they only knew how. Five to ten years ago, to even mention the Enterprise Service Bus as being a problem, they’d look at you with daggers in their eyes.”

“It’s almost like we’re now able to fulfil the promise of service orientation,” says Lewis. “It needed these additional practices, around things like automated deployment, automated rollback, and an understanding that people and process are tied intrinsically with it.”

Another issue, claims Lewis, is that software architects simply get out of touch with best practice.

“Most architects who build big systems are quite a long way from their codebases. They sit in rooms talking to other architects. They might have last written a line of code five or ten years ago. What they do is to design systems as they would have done ten years ago. People do get divorced from the latest trends and perpetuate less effective ways of doing things.”

Finally, here’s something for the Microsoft platform people who read this site. ThoughtWorks is not altogether averse to Microsoft and mentions the Azure cloud platform as something which is becoming interesting. But Windows Phone:

Despite a promising start to Windows Phone, a well thought-out user interface, and probably the best development experience of any mobile platform, we have seen several stumbles in the execution of the platform strategy by Microsoft and its partners. This makes us less optimistic about the future of the platform than we were in the last radar.

Translation: nice mobile platform, but nobody’s buying it. Then again, on Monday next week Windows Phone 8 will be properly unveiled. Still hope?

How Adobe turned on a pin to embrace the web (and Google)

Adobe’s Create the Web world tour – which came to London yesterday – is in the public unveiling of of Adobe’s new wave of tools, the first since it turned away from Flash and towards open web standard, hardly a year ago.

image 

Michael Chaize is a developer evangelist at Adobe. I asked him when it became clear to him personally that Adobe was no longer a Flash platform company.

“The main shift happened November last year [2011]” he told me. “It happened when we, for the Flash part, decided to just focus on video games and premium video, and invest in HTML tooling and specifications with a team of engineers. It was synced with the decision to stop developing Flash in mobile just to focus on apps with Adobe AIR.

“Now we are almost a year later, and Create the Web is an opportunity to showcase the work that has been done. All the product that have been launched, the Edge tools and service, just started in November of last year.”

The timing was confirmed by Adam Lehman, product manager for Edge Code, a tool built on Bracket, which is an open source project created by Adobe to provide a lightweight, code-centric editor for HTML 5 technologies. I asked him when work on Brackets started. Research started in mid-2011, he said, but “we got the team together in December 2011 and started coding.”

image
Adam Lehman

The Edge tools are intended as focused, lightweight product each targeting a specific small part of web design, in contrast to typical Creative Suite products such as Dreamweaver which encompass a large area of functionality; a valid approach but one which inevitably leads to huge tools that take an age to load and a lifetime to learn. Edge is also being used as a not-to-subtle way to promote Adobe’s subscription-based Creative Cloud, since the tools are only available by that route. As a further sweetener, you can get some of the tools as part of the free subscription tier.

It is remarkable that Adobe has navigated the difficult transition from Flash to HTML, and the difficult transition from shrink-wrap to subscription, with so little pain.

That said, perhaps the transition from Flash to HTML is not as profound as it first appears. The Flash runtime was always free, while Adobe made its money from design tools, and as the web become more capable, designing for the Web looks increasingly similar to designing for Flash.

Even the community is the same. “When it deals with expressive web, motion design, we feel that the Flash community can reuse their skills,” said Chaize.  “Being a Flash developer is not just about the language, it’s a knowledge, it’s a culture. Agencies tell me, ‘When I need to hire a motion designer for HTML, I hire a Flash guy.’

That said, HTML 5 is still inferior to Flash in some respects. I watched a slightly jerky animation showing off HTML 5 capabilities and could not help thinking that it would run more smoothly in Flash (of course it was all preview software). It will get there though. This is why Adobe is working to bring specifications like CSS shaders and CSS regions to the official standards.

There is another thing I noticed at Create the Web, which is the extent to which Adobe’s new tools are built on Google’s platform. Many of the Edge tools are made with the Chrome Embedded Framework; the browser used for demonstrations is Chrome Canary, a preview build implementing the newest standards, and if you look at the code you see abundant use of the WebKit prefix which designates features currently specific to the WebKit browser engine used by Apple, Google and others. There is also extensive use of WebGL, popular with designer but contentious because some browser vendors consider it a security risk and it is not an official web standard.

Lehman insists that there is no intention to go down a Google-specific route. “It was more of a technology stack we went with,” he says, explaining that the intent for Brackets is that it will one day run in the browser, in which case it will have to support Mozilla, Opera and Microsoft browsers as well.

The reason for adopting so much Google stuff is partly the excellent fit with what Adobe needed, and partly the low friction. “We didn’t have to go to a meeting, it was just published” said Lehman, referring to the Chromium Embedded Framework which let you run HTML5 applications on the desktop.

Brackets looks great, has real community adoption already, and Adobe has interesting plan for its future. Along with browser hosting, Lehman talks about proper debugging support with breakpoint, JavaScript macros, an embedded node.js engine, and more.

When Apple rejected Flash in iOS it put Adobe in a difficult spot – another reason for the company’s warmth towards Google and Android – but since then the transition has been remarkable.

Adobe using Google Chromium Embedded Framework for Edge tools

Adobe has published a mission statement which is worth a read if only to demonstrate how far the company has moved away from Flash, once positioned at the heart of its ecosystem – remember the Flash Platform?

The mission statement essentially declares the web as the new heart of Adobe’s platform and it is working to bring HTML, CSS and JavaScript up to the level of richness and interactivity that is possible in Flash.

This even extends to apps and applications, and I was interested in this statement:

The web platform also lives outside of browsers. It’s used by apps, particularly on mobile devices, where the richness of the web platform makes it possible to deliver great user experiences. Adobe will continue to invest in the Apache Cordova project, and Adobe’s distribution of it, Adobe PhoneGap™. When appropriate, Adobe is using the web platform to build tools and services. For example, Brackets, Edge Code and Edge Reflow are built using HTML, CSS and JavaScript using the CEF open source project, to which Adobe is contributing.

CEF is the Chromium Embedded Framework, which is a web browser control based on the open source version of Google’s Chrome browser. It is a C/C++ project but third parties have created wrappers for .NET, Delphi, Java and Python.

It is not long ago that Adobe would be looked to AIR, based on Flash, for a project like this. Incidentally, AIR is also able to host a WebKit-based browser control so would have been viable. Using CEF means getting to use Google’s V8 JavaScript engine rather than ActionScript.

Adobe Creating the Web, offers Edge animation tool for free

It is less than a year ago that Adobe pivoted wholeheartedly from Flash to HTML, a moment that to mind was marked by the acquisition of Nitobi, the PhoneGap company, announced at MAX in October 2011.

Yesterday Adobe clarified its plans for its new wave of web design tools branded Edge. These are as follows:

Edge Animate

A motion and interactive design tool for animating web content with HTML, JavaScript and CSS.

image

Edge Inspect

Preview HTML content on mobile devices for test and debug.

image

Edge Code

This is a commercial product based on the open source Brackets project – a similar relationship to the one that exists between Adobe PhoneGap and the open source Cordova project.

Edge Code adds Adobe integrations such as with Edge web fonts and Typekit, and with PhoneGap Build.

image

Edge Reflow

image

Design tools for CSS, preview expected by end of 2012.

Edge Web Fonts

Free web font service for open source fonts.

TypeKit

Commercial font library service.

PhoneGap Build

Package web apps as native apps for mobile platforms, without needing to install SDKs on your own machine.

PhoneGap Build is free for open source apps, or costs $9.99 per month for up to 25 private app builds.

The Edge tools are only available through Creative Cloud, Adobe’s subscription service, cementing the company’s move to a subscription model for its products. As a tempter, Edge Animate is currently available even to those with the base, free subscription – though you have to agree to be on a marketing list for email, mail and telephone.

image

Will the Edge tools replace Dreamweaver, the web design tool in Creative Suite? I was told not, and that an update for Dreamweaver is in preparation. Dreamweaver is the “one production tool” as opposed to the Edge tools each of which focus on one narrow area of features. Adobe describes this as task-focused tools.

More information in yesterday’s San Francisco keynote here.

Microsoft Project Austin: superb C++ code sample for Windows 8

No time to blog in detail about this; but developers with any interest in Windows 8 should check out Project Austin, a sample project for Windows 8 whose quality exceeds most of what is currently available in the Windows Store.

This is a simple note-taking app but beautifully rendered and with support for adding photos, sharing via Charms, and more:

It’s amazing how useful just a pen and a paper are by themselves. But when you take that concept to the computer realm and expand it to do things like add photos and annotate them right on the spot, and digitally share what you create, then the possibilities are endless.

say the project’s creators. More to the point:

Austin aims to demonstrate with real code the kind of device-optimized, fluid and responsive user experience that can be built with our newest native tools on the Windows8 platform.

Most of the code has been put on CodePlex under the Apache 2.0 license. It demonstrates C++ AMP, the new parallel library for GPU computing, as well as C++ coding for the Windows Runtime.

I downloaded and built the project with few issues, following the helpful guidance here. I did have to add the boost libraries manually to the include path.

image

It is not completely stable but is already very pretty.

image

Immersive UI of course. The tools disappear while you are working. I am still not sure how well I like this, but it is good to see examples of how Microsoft thinks this should work.

Adobe’s Roy Fielding patches Apache to ignore IE10 Do Not Track privacy request

Adobe’s Roy Fielding, who is also the original author of the W3C’s Tracking Preference Expression draft, has patched Apache, the open source web server, to ignore the Do Not Track header sent by Microsoft’s Internet Explorer 10, the browser in Windows 8:

image

Under the heading “Apache does not tolerate deliberate abuse of open standards,” Fielding’s patch sets Apache to remove the Do Not Track request header if IE10 is the web browser.

Fielding’s argument, one presumes, is that IE10 breaches clause three in the Tracking Preference Expression draft:

Key to that notion of expression is that it must reflect the user’s preference, not the choice of some vendor, institution, or network-imposed mechanism outside the user’s control. The basic principle is that a tracking preference expression is only transmitted when it reflects a deliberate choice by the user. In the absence of user choice, there is no tracking preference expressed.

However the document goes on to say (highlighting is mine):

We do not specify how tracking preference choices are offered to the user or how the preference is enabled: each implementation is responsible for determining the user experience by which a tracking preference is enabled. For example, a user might select a check-box in their user agent’s configuration, install an extension or add-on that is specifically designed to add a tracking preference expression, or make a choice for privacy that then implicitly includes a tracking preference (e.g., Privacy settings: high). The user-agent might ask the user for their preference during startup, perhaps on first use or after an update adds the tracking protection feature. Likewise, a user might install or configure a proxy to add the expression to their own outgoing requests.

Here is what happens in Windows 8 after startup. This is among the first screens you see when installing Windows 8, before you get full access to the operating system:

image

One of the settings specified is “Turn on Do Not Track in Internet Explorer. If you click Learn more about express settings you get this:

image

If you click Customize you get this:

image

Does this respect the user’s preference? It seems to me a reasonable effort. The only objection I can see is if you consider that any user agent that defaults to setting Do Not Track on cannot be respecting the user’s preference. The draft specification does not state what the default should be.

It is also worth noting that clause 3 in the Tracking Preference Expression draft has changed; the wording about “not the choice of some vendor” was inserted in the 7th September draft, after Windows 8 was released to manufacturing. Here it is in the latest (March 2012) W3C Working draft:

Key to that notion of expression is that it must reflect the user’s preference, not the preference of some institutional or network-imposed mechanism…

Even if you agree with Fielding’s views on browser defaults, quietly patching the world’s most used web server to ignore the IE10 setting looks hard to defend, especially on a matter that is far from clear cut. Fielding is personally involved, not only as the author of the Tracking Preference Expression document, but also as an employee of Adobe, which specialises in digital marketing and may be more aligned with the vendors and their brands which may want to track user activity wherever their ads appear, rather than with end users.

Of course Apache is an open source project and Fielding’s patch has attracted the attention of the Apache community and may not survive.

It is also possible that a future draft of the Tracking Preference Expression document will state that Do Not Track must be off by default; but even if it does, patching the web server to ignore the browser’s header strikes me as a contentious solution.

Finally, it is worth noting that sending the Do Not Track header has little effect on whether or not your activity is tracked, since its meaning is unclear and respecting its value is a a choice made by third-parties, so this is a debate with little practical impact for the time being.

Nokia sells Qt cross-platform software framework to Digia

Nokia has sold its Qt business to Helskinki-based Digia. According to the press release the sales includes:

… all the Qt activities formerly carried out by Nokia. These include product development, as well as the commercial and open source licensing and service business. Following the acquisition, Digia plans to quickly enable Qt on Android, iOS and Windows 8 platforms.

Digia had already acquired the Qt commercial licensing business from Nokia in March 2011. Up to 125 people will transfer from Nokia to Digia.

This is a move that makes complete sense, given that Qt is no longer central to Nokia’s plans. I once hoped that Nokia would port Qt to Windows Phone and the Windows Metro runtime in order to unify its development platform but it seems that if anyone will do that, it is Digia. It is not clear from the release whether Digia’s Windows 8 plans include the new runtime, but you would imagine that it does since as far as I know Qt already works fine on the Windows 8 desktop side.

See Digia’s Qt site for more info.

Valve announces Steam-powered apps beyond games as well as embracing Linux

Steam maker Valve has announced that it is expanding beyond games, to sell software titles that “range from creativity to productivity”.

image

The Steam software is more than just a store. The platform handles updates, digital rights management, and supports multiplayer gaming. It also forms a chat network. The Steam overlay lets users access Steam features while playing a full-screen game.

Users can install a Steam title on multiple computers but can only play while logged in, and can only log in on one device.

Steam launched first on Windows, but also has clients for Mac and, via Wine compatibility, on Linux. There are also mobile clients for Android and iOS, and some support for PlayStation 3, though these have limited features. The mobile clients do not let you buy and run games for the mobile device itself.

With Apple, Google and now Microsoft backing their own app stores for their respective platforms, Valve has some tricky manoeuvring ahead if it is to avoid being squeezed out. Valve founder Gabe Newell made headlines recently by calling Windows 8 a “catastrophe”, though he is hardly a disinterested party. Note that he should not worry too much about Windows 8 in the short term, since Microsoft’s store does not support desktop titles other than by links to third-party sites, including Steam. However the general trend towards locked-down platforms with software installed only through an official store must be a concern to Newell.

Valve is turning towards Linux as a possible solution. It is talking at the Siggraph conference this week in Los Angeles about its work on OpenGL and Linux, and it seems that a native Linux Steam client is in prospect.

Could Windows gamers, or others disillusioned with Windows 8, turn to Linux in significant numbers as an alternative? While this is possible, it seems more likely that the Mac would benefit. You would also imagine that skilled gamers will be smart enough to operate the Windows 8 Start menu and discover that most of their stuff still runs fine on the new desktop.

The Steam platform is a strong one though, and with Microsoft not supporting desktop apps through its own Store, Valve has a good opportunity to extend its reach.

According to its own stats, Steam has peaked at over 4 million concurrent users this month.

image

Adobe Brackets: a different type of HTML and JavaScript code editor. Interview with Adobe’s Adam Lehman.

On Adobe’s Tools and Services page there is an intriguing remark about the company’s plans for a code editor. “We think there’s a need for a different type of code editor – we’re working on something and will have more to share soon.”

image

That something is Brackets, a code editor written in HTML and JavaScript (which means, as with all the best tools, that you can code Brackets in itself).

Advertisement

Although Brackets is written in HTML and JavaScript, it is not yet a web application. Instead, it runs on the desktop using Google’s Chromium Embedded Framework (CEF), which lets you embed the Chrome (strictly, the Chromium) browser engine in a desktop application. In the case of Brackets, the wrapper is lightweight, the intention being that in future Brackets may be fully browser-hosted. The consequence though is that currently you need Google Chrome installed and it only runs on Windows and Mac.

The project is open source under the MIT license; anyone can grab the code from Github. Brackets also depends on another open source project, CodeMirror, which is a JavaScript editor component for browsers. I installed it on Windows and soon had it up and running. Note that you should pull brackets_app if you want to run it, as this brings down the Brackets code as well.

image

I spoke to Brackets Product Manager Adam Lehman. “This might be the first project we started with the intent of being open source from day one,” he said.

“Our general intent is that we wanted to provide an editor that web developers felt that they could own. In the past we might have built something in Eclipse, and there would have been this giant gap between the person who knew HTML, JavaScript and CSS, and then having to write a Java-based Eclipse plug-in to extend the editor.

“When we start talking to developers, they’re going back to just text editors, things that don’t do much more than edit and manage a document, and as complex as HTML and JavaScript apps are getting these days, it seemed crazy that our tools weren’t keeping up with us. So the idea was to start this project, add a little bit of our own ideas, and have the community supply their own ideas.”

But how does Adobe intend to use Brackets in its own products, and what is the business model?

“We believe there are two spaces for the editor market. There is the larger IDE, but there’s also these lightweight text editors. We’re finding that the traditional JavaScript and HTML developer, CSS developer, was heading towards the lightweight text editor and not towards the larger IDE. We don’t see Dreamweaver and Brackets as direct competitors because they service two different tastes. It wasn’t a matter of could we add a feature here or there that was going to get people to use Dreamweaver. It was that difference between our larger tool and a much lighter weight tool. That’s where Brackets come in.”

How then will Brackets tie in with other Adobe products?

“That’s the key for Brackets. We wanted to see if we could innovate in the space and we also wanted to have a common language that we could start targeting. When you say, I want to open up an editor from Adobe Edge and start coding, we needed to define what that editor would be, so Brackets would come into that.

“We’re also saying we need to do better tooling around PhoneGap. A lot of people are fine with the command line, but we want to take a step beyond that and so Brackets is the obvious place where we’ll start to build an extension where we can tie into PhoneGap Build, or extensions around the PhoneGap APIs.

“We’ve got a lot of ideas around using Brackets to bring a lot of our HTML efforts together, not only our core HTML products but also a lot of the W3C and WebKit work that we’re doing. Brackets is a great place to put tooling, that isn’t quite ready for mass consumption yet, but we could actually build extensions for something like Shaders where those people who are interested in it can get in and start playing around with it.

“The beauty of building on the web platform is that we can go wherever the web platform goes.”

Initial prototypes of Brackets ran entirely in the browser, which would be interesting for future versions of Adobe’s Creative Cloud as well as other scenarios, but Lehman said this got mixed reactions.

“While we believe that the future of development is heading towards the cloud, and the general consensus from developers is the same, we also heard that it is not ready yet. We decided to focus on a desktop version first, with the idea that towards the end of this year or beginning of next year we’ll start to supplement with other targets, whether it be in the cloud or  a tablet, or embedded in a tool like Edge,”

says Lehman. He adds that a browser-hosted Brackets could end up integrated with the PhoneGap Build site. PhoneGap Build is a service for compiling cross-platform HTML and JavaScript into mobile apps for a variety of devices.

Since Brackets is built on Google’s Chrome/Chromium platform, what are the implications for cross-browser compatibility?

“There’s two pieces to it. There’s our container that Brackets runs in, and that is running on Chromium.

“The other part is that we have this live preview system which is tied directly to Chrome on the desktop. We happened to just start with Chrome, mostly because there is a remote debugging API that’s pretty fleshed out there.

“With Firefox and Internet Explorer, it’s a little bit different. We talked to Mozilla and they’re just now starting to work on that remote debugging API and trying to get it inline with where Chrome is, so we’re expecting to hear from them in August that they actually might have an API where we can start to build that same functionality, which is our intent.

“We’ve already started engaging with Microsoft about Internet Explorer. Right now their remote debugging API is somewhat private and in the form of a COM object which is not ideal coming from a JavaScript perspective, but we’ve showed them what we’re after, and we’ve started discussion of what a remote API from IE might look like that didn’t require COM. We’re exploring those options. Those are our priorities right now.

“If we build a cloud-based version then it’s going to be a question of what browsers this is going to run in. Our intent is to run in the modern major browsers. We aren’t building anything that’s Chrome-specific, we’re doing our best to stay as browser-agnostic as we can, but we are likely to require a more modern browser. We feel it would be OK to require the latest versions of Firefox, IE or Chrome.”

I asked Lehman whether Brackets might be useful for server-side as as client-side code. He said that Brackets is focused on the client, though a community extension is under way for node.js. He adds that since Brackets is fully extendable, others may do plug-ins for languages such as PHP.

Why is Brackets at Github and not Apache?

“We have a lot of people at Adobe who work for Apache now, and we talked to them before we released Brackets. Our sense is that Apache might be too much of a turn-off for the individual contributor, who just wants to hack and fix a bug and submit it back,”

Lehman told me. Although there are external contributors, all the committers are currently at Adobe, though there are plans for adding external committers by the end of the year. “We don’t want this to be 100% Adobe controlled.”

When will Brackets get to version 1.0?

“We’re being as agile as we can. Every bit we add, it comes closer to being a 1.0 for somebody. The things that I think are missing that you would expect out of a core editor are around code-completion for CSS and JavaScript, and solid and advanced search and replace. In the web world, that’s how we refactor code. We’re hoping to drive those in by November time. But we are on 2.5 week sprints and things change rapidly.”

I also asked about plans for a mobile app version of Brackets. Lehman says that is planned for next year, though the community is working on getting a Linux version working and support for ChromeOS.

Brackets is a fascinating project on several levels. What stands out is how far Adobe has moved from being the Flash company. A few years back Adobe came up with a system for having Flash applications run on the desktop and on mobile devices: Adobe AIR. It also invested in Eclipse and came up with the Flash Builder IDE.

Now here is Adobe with an open source project for a desktop application built from HTML, JavaScript, and a third-party open source browser engine; and in place of mobile AIR it has PhoneGap.

It is a big change, most of which has become publicly known in less than a year, signalled by the repositioning of Flash and AIR versus HTML in September 2011, and the abandonment of Flash for Mobile in November 2011.

As for Brackets itself, it is well worth a look though probably not a tool you want to use for real work just yet. In a few months though, that may well change.

Postscript: Brackets reminds me of another Adobe, or rather Macromedia, HTML code editor. That was HomeSite, an excellent text-based tool that Adobe discontinued in 2009; active development ceased years before that.

Updating the world’s most widely deployed SQL database engine: welcome to SQLite 4

A new version of SQLite is in preparation. If you are not a developer, you might not have heard of SQLite, but you have almost certainly used it. It is built into Mac OS X and numerous web browsers, used by many applications which run on Adobe’s Flash runtime, and is the obvious choice if you want a small, fast and reliable database engine to embed into an application. It is open source and as free as you can get:

Anyone is free to copy, modify, publish, use, compile, sell, or distribute the original SQLite code, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

SQLite3 is the current version; but now there is an update to version 4:

SQLite4 is an alternative, not a replacement, for SQLite3. SQLite3 is not going away. SQLite3 and SQLite4 will be supported in parallel. The SQLite3 legacy will not be abandoned. SQLite3 will continue to be maintained and improved. But designers of new systems will now have the option to select SQLite4 instead of SQLite3 if desired.

The reason for the new version is that some issues in version 3 could not be fixed without breaking compatibility.

So what is new? On a quick read, these seem to be the highlights:

  • A global configuration object (sqlite4_env) which eliminates all use of global and static variables.
  • A new key/value storage engine which has a “greatly simplified” interface and which is pluggable, so you can use a different one if required. The default storage engine is described as a “log-structured merge database”. A B-Tree engine may also be offered later.
  • Primary keys are now real primary keys, as opposed to unique constraints. This speeds up primary key searches.
  • Decimal maths. “All numeric values are represented internally as an 18-digit decimal number with a 3-digit base-10 exponent.” This is advantageous for currency calculations and for cross-platform consistency.
  • Foreign key constraints and recursive triggers on by default
  • Covering support in indexes (when required), to increase the number of queries that can be resolved by querying the indexes alone, at the expense of greater duplication of data

When will SQLite 4 be ready? Code is available but Author D Richard Hipp says:

Everything is still pretty makefile-touchy. Remember, this is like pre-alpha code. It works, but just barely. And things are changing rapidly.

Porting an application from SQLite3 to SQLite4 should be straightforward, according to the author. “An hour or two search-and-replace.”

image