Evans Data has published its 2009 Software Development Platforms survey, to which around 1200 developers contributed, scoring their chosen development tools in eighteen different categories.
The tools covered are Eclipse, Embarcadero’s Delphi, IBM’s Rational Suite, IntelliJ, Microsoft’s Visual Studio, NetBeans, Oracle JDeveloper and Sun Studio.
I was sorry not to see more products covered.
Flex Builder Flash Builder, Zend Studio, Aptana and JBuilder would all have been interesting, for example. Each developer only scored the product they actually use (a good thing), so the sample is not as big as it first appears.
I’m also mistrustful of the survey results, particularly when you look at it in detail. For example, one of the categories is “Support for frameworks”. Visual Studio came top, while Eclipse was last. But hang on: in Visual Studio (for example) are we talking MFC, or .NET Framework? The development experience for each is totally different. And were developers primarily judging on the framework tools, or the framework itself? It is hard to attach much meaning to the scores in this category.
Another flaw: the versions of the products is not specified. That means a weakness may have been fixed in a later version, but the survey does not tell you.
A third flaw: some tools are weak in several categories, dragging down their overall score, but that does not matter to developers who do not use them for that purpose. It is hard to compare like with like.
Still, while I’m wary of the survey overall, I though it brought out some interesting points. One is that developers were asked what features matter most to them. So:
The three things developers care about most (highest priority first):
1. Basic tools (editor/compiler/debugger)
3. Tool integration
The things developers care about least (lowest priority first):
1. Support for remote development
2. Support for parallel programming (sorry Intel!)
3. App Modeling tools
What about the winners and losers in the survey? I almost forgot. IBM’s Rational Suite came top, followed by Microsoft’s Visual Studio. Eclipse came last, though it still got a decent score, well below its rival NetBeans.
The low ranking for Eclipse (which is nevertheless wildly popular) deserves some comment, particularly as the top tool, IBM Rational Suite, is built on Eclipse. I spoke to Eclipse executive director Mike Milinkovich while researching this Register piece recently. One of the points we discussed was the tension (if there is one, which he disputes) between tools vendors sharing resources to build the best possible platform, and holding resources back to retain commercial differentiation. I’ll write this up in more detail shortly; but it shows why certain areas in Eclipse may not receive the attention they deserve – localization was a specific example.
Another problem with Eclipse is that it is all a bit messy, confusing and hard to manage, particularly in a team where you want every member to have an identical setup. It is still worth it though, for the riches it provides for free.