Tim Anderson’s ITWriting

It is not unexpected, but still sad to see loss-making Borland acquired by Micro Focus for a knock-down price of $75m. Borland’s release says little beyond the financial details. Micro Focus, which is also acquiring Compuware’s ASQ (Automated Software Quality) tools (such as QADirector, DevPartner and Optimal Trace, I presume) says:

Acquiring Borland and the Compuware Testing and ASQ Business will give Micro Focus a leading market position in the highly complementary Application Testing / ASQ market. This market is estimated to be worth c.US$2 billion a year and is logically adjacent to Micro Focus’ core application management and modernization business.  The move into the ASQ market is consistent with Micro Focus’ stated strategy of extending in logically adjacent segments to expand its addressable market.

Why sad? Well, if you were around in the eighties and nineties you will remember a bold company which came up with a series of excellent products: Turbo Pascal, Borland C/C++, Quattro Pro, Paradox, and of course the incomparable Windows development tool Delphi. The visual development model in Delphi was successfully transitioned to Java in the JBuilder product, which in its early versions used a Delphi-compiled IDE.

These developer-focused products live on, of course, mostly in the hands of Embarcadero. The Borland that has been acquired is what was left when, in my developer-centric opinion, the best parts had already left.

What went wrong at Borland? It is mostly the victim of changes in the industry, made worse today by the economic downturn. It was a tools company, and the tools market was hit by the double blow of excellent open-source competition on one side (Eclipse, GCC) and vendor-subsidised tools on the other (Visual Studio).

Still, there were some spectacular own goals along the way. The 1991 acquisition of Ashton-Tate, at the time the market leader in PC database managers, was one, mainly because dBASE IV was not very good and did nothing to help Borland transition to Windows; in any case, Borland already had a better product in the form of Paradox.

Talking of Paradox, Paradox for Windows was another disaster. Wonderful product, but mostly incompatible with its DOS predecessor, and probably a tad too complex as well. It also had to compete with Microsoft Access, which was both cheaper and part of the impregnable Microsoft Office suite.

The company made up for it with Delphi; but even that under-performed relative to its quality. Enterprises felt safer with Microsoft’s Visual Basic. JBuilder did well at first; but its market share diminished rapidly in the face of competition from Eclipse and NetBeans. In retrospect, Borland should have made its core Java IDE free much earlier, to build a community round it, though competing with free is never easy.

Since it was so hard making money out of compilers and IDEs, Borland changed tack in order to target Enterprise ALM (Application Lifecycle Management). It could have worked, but it wasn’t actually a great fit with the independent developers who formed a large part of its customer base, and who tended to ignore large, complex and expensive supplementary tools in favour of just getting on with coding.

The nadir was 1998 when Borland changed its name to Inprise, to reflect its Enterprise focus. “Many thought Borland had gone out of business”, says Wikipedia. It was changed back to Borland in 2001.

Another mis-step was the way Borland (then Inprise) handled InterBase, its client-server database. In 2000, with a burst of community enthusiasm, the product was made open source. A couple of years later, it changed its mind and continued to develop InterBase as a proprietary product; but by then FireBird had been born, based on the open source code.

Thought for the day: Borland paid more for TogetherSoft in 2002 (around $185m, including $82.5m cash), than Micro Focus is paying now for Borland.

I’ve been amusing myself creating a simple online database application using Silverlight. I had this mostly working a while back, but needed to finish off some pieces in order to get it fully functional.

This is created using Silverlight 2.0 and demonstrates the following:

• A bound DataGrid (as you can see, work is still needed to get the dates formatted sensibly).
• Integration with ASP.NET authentication. You have to log in to see the data, and you have to log in with admin rights to be able to update it.
• Create,Retrieve,Update,Delete using ASP.NET web services.
• Image upload using Silverlight and an ASP.NET handler.
• Filter a DataGrid (idea taken from here).
• Written in Visual Studio 2008, and hosted on this site, which runs Debian Linux, hence Mono and MySQL. Would you have known if I had not told you?

You can try it here. I’ll post the code eventually, but it will be a couple of months as it links in with another article.

MVP Ken Cox notes in a comment to Jesse Liberty’s blog:

Hundreds of us are scouring the Internet for a realistic (but manageable and not over-engineered) sample of manipulating data (CRUD operations) in a Silverlight 2 application. There are promising pieces of the puzzle scattered all over the place. Unfortunately, after investing time in a sample, we discover it lacks a key element – like actually saving changed data back to the database.

I can safely say that mine is not over-engineered, and that yes, it does write data.

SQLite developer D Richard Hipp has posted to his mailing list to announce a third release in the space of a few days, to fix bugs discovered in version 3.6.10:

Some concern has been expressed that we are releasing too frequently. (Three releases in one week is a lot!) The concern is that this creates the impression of volatility and unreliability. We have been told that we should delay releases in order to create the impression of stability. But the SQLite developers feel that truth is more important than perception, not the other way around. We think it is important to make the highest quality and most stable version of SQLite available to users at all times. This week has seen two important bugs being discovered shortly after a major release, and so we have issued two emergency patch releases after the regularly scheduled major release. This makes us look bad. This puts "egg on our face." We do not like that. But, three releases also ensures that the best quality SQLite code base is available available to you at all times.

He goes on to say that an extended beta period would be unlikely to reduce the risk of bugs found on release, because most bugs in SQLite are found by internal testing rather than by external users. He also argues against withholding releases until they “testing is finished”:

The fallacy there is that we never finish testing. We are constantly writing new test cases for SQLite and thinking of new ways to stress and potentially break the code. This is a continuous, never-ending, and on-going process. All existing tests pass before each release. But we will always be writing new tests the day after a release, regardless of how long we delay that release. And sometimes those new tests will uncover new problems.

Anyone who has ever developed an application will know that sinking feeling when problems are discovered in code that has been distributed. Thoroughly implemented unit testing, as in SQLite, improves quality greatly. When bugs are found though, full disclosure and prompt fixes are the best possible response, so I agree with Hipp’s general approach here.

Microsoft has announced that SQL Server 2008 is released to manufacturing – ie. the bits are done, even if you can’t buy it yet. MSDN subscribers can download it now.

This is the product that was “launched” back in February; it’s been a long delay but I get the impression that the SQL team likes to wait until its release really is ready.

SQL Server 2008 is more like a suite of products than a single product now. It has a large range of editions from Compact to Enterprise, and product areas like Analysis Services and Reporting Services are distinct from the core engine.

The pieces that interest me most are the spatial data types, sparse columns, FILESTREAM data type, and the various object-relational layers including LINQ, Entity Framework, ADO.NET Data Services, and the ongoing work with SQL Server Data Services (which is far from done yet).

DBAs will likely have a very different view of what is important, as will Business Intelligence specialists.

SQL Server has prospered by being cheaper than than the likes of Oracle and DB2, and by integrating smoothly with Windows and Active Directory. I wonder if it will feel pressure from even more cost-effective open source offerings like MySQL, as they become more Enterprise-ready?

Maybe not where you think. Now, I admit I am three years late with this bug strange feature of Visual Studio but it wasted some of my time today so it is still worth reporting.

I’ve been writing about creating database applications in Visual Studio. Specifically, I was looking at what happens if you download Visual Basic Express and take the quickest, easiest route to knocking together a database application.

The default local database engine these days is SQL Server Compact 3.5:

When you create applications, the preferred local database is SQL Server Compact 3.5.

says MSDN.

OK, so you add a new database to your project and accept various defaults. The wizard then asks you whether you would like to “copy the file to your project and modify the connection”?

Sounds reasonable, if you can figure out what it means. Default is Yes, so OK do it.

Mistake. Don’t do that. Not, at least, without reading and understanding this document. But I digress. Next up, you are asked another question:

Storing connection strings in your application configuration file eases maintenance and deployment … do you want to save the connection string to the application configuration file?

It’s another option that sounds good. OK, do it.

Now you set up a little table or two, add some data-bound controls, and off you go. Run the app, enter some data, save it. Run the app again … and all your data has disappeared. Why?

Well, it has to be either that the updates are silently failing; or that the database file is getting overwritten. It’s the latter. It turns out that VB is treating your database like any other resource, and copying to bin/debug when you run the app. This is the copy you are connecting to, you update it, but next time you build and run it gets overwritten.

None of this is obvious, because when you look at the connection string in the application settings (which VB hides by default, sigh), it shows the database file in the root of your project folder. Click Test Connection, all is fine. The only warning sign is that the connection string looks like this:

Data Source=|DataDirectory|\test.sdf

So where is |DataDirectory| set? That’s not obvious either. Read here for the answer. It’s an application property that is not visible anywhere, that gets set to different values depending on how you deploy the app. I can see why someone thought this was a smart idea; but the implementation is horrible. It gives you the illusion of having one database file, when in fact you have multiple copies (source, debug, release etc) overwriting one another, and during testing you are never editing the correct one.

Once you have worked this out you can fix it, of course. But here’s another problem. You are the single user of a database. You insert a record and save it, using all the generated data-bound stuff that Visual Studio provides. Works fine. Then you edit the record you just inserted, and save again. Boom. Concurrency exception. Why?

It is all do to with a limitation of SQL Server Compact 3.5. It can’t handle multiple SQL statements. This means that a feature of the ADO.NET TableAdapter, called Refresh the Data Table in the configuration wizard, is not available. This option kicks in when you have an identity column that auto-increments, which is by the easiest way to create a primary key. In this scenario, the actual value of the identity column is not known until after you make the insert, because it is generated by the database engine. Normally, the TableAdapter would retrieve it with a Select statement immediately after the Insert statement. However, with SQL Server Compact 3.5 that does not work.

The result is that saving a record works fine, but next time around the row has an incorrect primary key in the DataSet. No wonder you get a concurrency exception.

You can work around this in code, of course. But what surprises me is just how hard Microsoft has made all this for the kind of newbie programmer who might pick up VB Express. In fact, easy database programming in VB has marched backwards since Visual Basic 3 back in 1993.

By the way, I also dislike the way VB adds so much database gunk to your main form, again by default. What if you add another form to your app? What if you want to delete the first form? It all gets messy fast.

Look at Ruby on Rails. It has simple database handling that works. OK, you are going to have to modify that code eventually; and I accept that database apps have an inherent complexity that no amount of wizards, O/R layers or even "Convention over Configuration” can remove. I still think that simple, single table, single user apps should be, well, simple. Not in VB, unfortunately.

So says Roger Jennings in his post Is the ADO.NET team abandoning LINQ to SQL? His main points in favour of ADO.NET Data Services (formerly Astoria) Entity Framework:

• It is the focus of more energetic development
• It already has richer features
• It supports multiple database engines, not just SQL Server

As Andres Aguiar, software architect at Infragistics, notes in a comment, this has a lot to do with internal politics at Microsoft:

The Data Programmability Team never owned LinQ to SQL, it was owned by the C# team. That’s why we have two O/R mappers, both teams wanted to ship theirs. The C# team looks to be thinking about functional programming now. The Data Programmability will always be thinking about data. That’s why the EF [Entity Framework] is the safe choice.

Although LINQ to SQL is now (apparently) owned by the SQL Server team, it still doesn’t seem plausible or sensible that both will get equal attention. We also learn from Matt Warren that LINQ to SQL was deliberately tied to SQL Server only:

LINQ to SQL was actually designed to be host to more types of back-ends than just SQL server. It had a provider model targeted for RTM, but was disabled before the release. Don’t ask me why. Be satisfied to know that is was not a technical reason.

Note that this wasn’t necessarily a plot in favour of SQL Server world dominance; keeping the entire stack as a Microsoft stack no doubt makes support easier. That said, to me this is the big weakness of LINQ to SQL.

I was impressed by Astoria when I first saw it at the European Tech Ed in 2007. I am not surprised it is gaining ground.

Fascinating interview with Quentin Clark, who led the cancelled WinFS project at Microsoft. Jon Udell is the interviewer.

Clark talks about how technology from WinFS is now emerging as the Entity Framework in ADO.NET (part of .NET 3.5 SP1) and the FileStream column type in SQL Server 2008 – a connection I’d already made at the Barcelona TechEd last year. He also mentions the new HierarchyID column type that enables fast querying of paths, the concept of rows which contain other rows. He adds that a future version of SQL Server will support the Win32 API so that it can support a file system:

In the next release we anticipate putting those two things together, the filesystem piece and the hierarchical ID piece, into a supported namespace. So you’ll be able to type //machinename/sharename, up pops an Explorer window, drag and drop a file into it, go back to the database, type SELECT *, and suddenly a record appears.

Put that together with the work Microsoft is doing on synchronization, and you get offline capability too – something more robust than offline files in Vista. Clark says SharePoint will also benefit from SQL Server’s file system features.

Note that Live Mesh does some of this too. I guess SQL Server is there in the Live Mesh back end, but it strikes me Microsoft is at risk of developing too many ways to do the same thing.

The piece of WinFS that shows no sign of returning is the shared data platform, which was meant to enable applications to share data:

… all that stuff is gone. The schemas, and a layer that we internally referred to as base, which was about the enforcement of the schemas, all that stuff we’ve put on the shelf. Because we didn’t need it.