Category Archives: software development

.net, development, professional, software development, visual studio

Continuous Integration vs Continuous Delivery vs Continuous Deployment: what is the difference?

I am reading the excellent book Continuous Delivery by Jez Humble and David Farley. But what is Continuous Delivery and how does it differ from the other “continuous” development methodologies?

It helps to understand that all these methodologies spring from the Agile software development movement, and the expression Continuous Delivery is a quote from the Agile Manifesto:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Now, the starting assumption is that most software projects integrate a number of smaller projects, whether from third-parties or from team members. Since these pieces are developed to some extent independently there is a risk that changes made to one piece will require modifications to another piece; hence according to Humble and Farley:

Most software developed by large teams spends a significant proportion of its development time in an unusable state.

The business of getting all the parts to work together is called integration, and if this involves serious work you need to have an integration phase where this is the sole objective. This is a bad idea for all sorts of reasons, slowing development and preventing proper testing other than at the end of these integration phases.

The solution is called Continuous Integration (CI). You have a frequent automated build that assembles all the pieces from all the teams into a working application. If the build fails, or if automated tests run against the build fail, then this is a bug that should be fixed immediately, not later in some separate phase of development.

Tools for CI include Cruise Control, Hudson, TeamCity and others; .NET developers can also configure Visual Studio Team Foundation Server for CI.

The problem with CI alone is that the development environment is not the same as the production environment. What if the CI build works and tests pass, but once deployed the application breaks or performs badly? Perhaps the development environment runs a multi-tier application with all the tiers on a single box, but when deployed onto actual multiple machines or VMs, something goes wrong. Permission problems are another common source of errors.

Continuous Delivery means that you not only build the software, but also deploy it frequently. This usually means provisioning servers, which you can automate using a tool like Puppet for Unix-like servers, or with Virtual Lab Management in a Visual Studio environment. Automation is pretty much essential for this to work. The more closely the test environment matches the production environment, the better.

Generally though, Continuous Delivery means deployment to a test environment. What about taking the next step, and deploying continuously to production? That is the methodology called Continuous Deployment. It sounds risky; but if you have a very extensive and thorough set of automated tests, then the risks are mitigated, especially as the extent of the changes in any one deployment is reduced.

Other suggestions for reducing risk include deploying to a small subset of users first, called “canary testing”; and making rollback easy.

That said, to judge by the Humble/Farley book the distinction between Continuous Delivery and Continuous Deployment is just a little blurred. The authors acknowledge that continuous deployment into production is not always a good idea. They also imply that Continuous Deployment might mean only that your application is always ready and easy to deploy into production, not that you necessarily deploy it constantly:

Your implementation should make it possible to deploy any version of your application that has made it past the automated tests into any of your environments at the push of a button, given the correct credentials.

Compliance and security are also factors that may rightly make it impossible to automate deployment to production completely.

Client-installed applications present some special difficulties which Humble and Farley discuss.

In summary then:

Continuous integration: your application always builds and passes its tests, including all the pieces from different sub-teams.

Continuous delivery: your application always builds and deploys to a test environment and passes its tests.

Continuous deployment: your application is always ready to deploy to production through a largely automated process.

Update: I received an email from Martin Fowler about this post. He refers to Jez Humble’s post on Continuous Delivery vs Continous Deployment and adds:

– I would use your definition of Continuous Deployment for Continuous Delivery

– I would change the definition of Continuous Deployment to say something like "every good build is released to production"

However, I clarified with him that if you building for a test environment but are confident that any build that passes would be OK to deploy to production, then you are still doing Continuous Delivery. In the end, while I am sure you should use Fowler and Humble’s definitions rather than mine, it seems to me a fine distinction and that if you are doing Continuous Delivery properly then the transition to Continuous Deployment is largely a matter of policy.

adobe, internet, professional, software development, web authoring

Adobe Muse: so what is wrong with Dreamweaver?

Adobe has released a preview of Muse, a new web site design tool.

My first reaction was one of be-musement. What is wrong with Dreamweaver, the excellent web design tool included in Creative Suite? Bearing in mind that there is also a simplified Dreamweaver aimed at less technical business users, called Contribute.

Here are some distinctive features of Muse:

1. It is aimed at non-coders. The catch phrase is “Design and publish HTML websites without writing code”. Muse actually hides the code. I installed Muse on a Mac, and one of the first things I looked for was View Source. I cannot find any such feature. You have to preview the page in the browser, and view the source there. That is in contrast to Dreamweaver, where the split view shows you simultaneous HTML and visual designers, and you can edit freely in either.

2. It is an Adobe AIR application. I discovered this in a bad way. It would not install for me on Windows:

image

A curious error. Luckily I am also working on a Mac right now, and there it worked fine.

image

3. It will be sold by subscription only. The FAQ answer is worth quoting in full, as it describes one of the key advantages of cloud computing:

Muse will be sold only by subscription because it will allow the Muse team to improve the product more quickly and be more responsive to your needs. Traditionally Adobe builds up a collection of new features over 12, 18 or 24 months, then makes those changes available as a major upgrade. It is anticipated that new updates of Muse will be released much more frequently, probably quarterly. New features will be made available when they’re ready, not held to be part of an annual or biannual major upgrade. This will enable us to stay on top of browser and device compatibility issues and web design trends, as well as enabling us to respond to feature requests and market changes in a much more timely fashion.

I am reminded of Project Rome, a cancelled project which was also intended to be subscription only. Rome was for desktop publishing, Muse is for web design; otherwise there are plenty of parallels.

4. Muse promotes Adobe hosting via Business Catalyst, and if you select Publish this is the sole option:

image

Of course you can also Export as HTML. Still, it looks as if Muse is intended as part of a wider initiative which will include hosting and web analytics.

5. Muse is not a Flash authoring tool. Check out the Features page. The word Flash does not appear. Nor did any hidden Flash content appear when I exported a page as HTML. My guess: there is a quiet Flash crisis at Adobe, and the company is hastening to make its tools less Flash-centric, in favour of something more cloud and HTML 5 based. I do not mean that Flash is now unimportant. It is still critical to Adobe, and after all Muse itself runs on Flash. However it is being repositioned.

A few comments. Unfortunately I’ve not yet spoken to Adobe about Muse, but the obvious question is reflected in my heading: what is wrong with Dreamweaver? To answer my own question, I can see that Dreamweaver is a demanding tool, and that Muse, while still aimed at professionals, should be easier to learn.

On the other hand, I recall many early web design tools that tried to hide the mechanics of web pages, some more successful than others, and that in the end Dreamweaver triumphed partly thanks to its easy access to the code. Some still miss HomeSite, an even more code-centric tool. What has changed now?

Needless to say, Dreamweaver is not going away, but there is clearly overlap between the two tools.

Of course non-coders do need to be involved in web site authoring, but the trend has been towards smart content management tools, such as WordPress or Drupal, which let designers and coders develop themes while making content authoring easy for contributors. Muse is taking a different line.

Watch this space though. Even on the briefest of looks, this is an impressive AIR application, and it will be interesting to see how it fits into Adobe’s evolving business strategy.

Update: Elliot Jay Stocks blogs about the code generated by Muse, which he says is poor, and his opinion that it is too much print-oriented:

warning signs are present in this public beta that suggest Muse is very much a step in the wrong direction.

development, gaming, google, open source, professional, qt, software development

Google Native Client: browser apps unleashed, or misconceived and likely to fail?

6 Comments

Last week Google integrated Native Client into the beta of Chrome 14. Native client lets you compile C/C++ code to run in the browser. It depends on a new plug-in API called Pepper. These are open source projects sponsored by Google and implemented in the Chrome browser, and therefore also likely to turn up in Chrome OS which is an operating system in which all apps run in the browser.

Native Client is cool. For example, NaCLBox lets you run old DOS games in the browser by porting DOSBox to Native Client.

image

Another project is Qt for Google Native Client, a project currently in development. Qt is an excellent and popular GUI and application framework which would speed development of Native Client apps as well as enabling many existing applications to be ported.

It is also worth mentioning that Native Client provides another way to run .NET code in the browser, via Mono with NaCl support.

Why Native Client? Google’s vision, or at least the part of it that focuses on Chrome OS rather than Android, is that everything runs on the Internet and in the browser, making the local operating system unimportant and easily replaced. Native Client removes any performance compromises in managed languages such as JavaScript, ActionScript or Java, as well as easing migration for businesses with existing C/C++ code.

Writing native code for the browser is nothing new. Both Microsoft’s ActiveX and the NPAPI plug-in API used by non-Microsoft browsers let you extend the browser with native code. However Native Client is seamless for the user; you do not have to install any additional plug-in. The main limitation is that Native Client applets do not have access to the local operating system, for security reasons.

It is also worth noting that Native Client apps are not altogether cross-platform. They must be recompiled for different CPU instruction sets, with the current implementation supporting x86 and ARM though you have to compile two binaries. Google says it will support LLVM output to enable cross-platform binaries though this will impact performance.

But is Native Client secure? That is an open question. Google was aware of the security challenge from the beginning of the project. Unlike the plug-in mechanisms which rely mainly on trust in developer competence and signed code to verify the origin of the plug-in or ActiveX control, Native Client inspects the actual code for unsafe instructions before allowing it to run. There is also an “outer sandbox” which intercepts system calls.

However, adding any new way for code to run makes the browser less secure. Google ran a Native Client Security Contest to help identify vulnerabilities, and the contestants did not have any problem finding security flaws. Of course all of these discovered flaws will have been fixed, but there may be others and likely will be.

And is Native Client necessary? The latest JIT-compiled JavaScript engines are fast enough to enable most types of application to run at a satisfactory speed. This is not just about performance though; it is about reusing existing skills, libraries and applications. There is no doubt that Native Client is nice to have; whether its benefits outweigh the risks is harder to judge.

The last question, which may prove the most significant, is political. Google has forged ahead on its own with Native Client, saying as vendors always do that it hopes it will become a web standard. In the early days of the project, it looked like a Native Client plug-in might enable the feature in other browsers, but abandoning NPAPI for Pepper makes this difficult. Will other browser vendors support Native Client?

Here is a comment from Google’s Ian NI-Lewis that I find remarkable:

As you probably know, the rule in Web standards is "implementation wins." So we’re concentrating on getting a good quality implementation out the door. We’re doing that in Chrome. That doesn’t mean that NaCl is intended to be "Chrome only," just that we have to start somewhere.

So Native Client is non-standard, and therefore less interesting than HTML 5 until either Google has a Microsoft-Office-like de facto monopoly of web browsers, or it persuades Mozilla, Microsoft and Apple to support it.

That said, you can think of Chrome as an installable runtime in the same way as the Java Virtual Machine or Adobe Flash, just a potentially more intrusive one. Here is our app, you have to install the free Chrome browser to use it. If this happens to any great extent, I can foresee other browser makers hastening to support it.

.net, c++, intel, microsoft, open source, professional, software development

C++ 11 is approved by ISO: a big day for native code development

4 Comments

Herb Sutter reports that C++ 0x, which will be called C++ 11, has been unanimously approved by the ISO C++ committee. The “11” in the name refers to the year of approval, 2011. The current standard is C++ 98, though amended as C++ 03, so it has taken 8 or 13 years to update it depending on how you count it.

This means that compiler makers can get on with implementing the full C++ 11 standard. Most current compilers implement some of the features already. This Apache wiki shows the current status. A quick glance suggests that the open source GCC is ahead of the pack, followed by Intel C++ and then perhaps Microsoft Visual C++.

C++ 11 is pretty much compatible with C++ 03 so existing code should still work. However there are many new features, enough for Bjarne Stroustrup to say in his feature summary:

Surprisingly, C++0x feels like a new language: The pieces just fit together better than they used to and I find a higher-level style of programming more natural than before and as efficient as ever. If you timidly approach C++ as just a better C or as an object-oriented language, you are going to miss the point. The abstractions are simply more flexible and affordable than before. Rely on the old mantra: If you think of it as a separate idea or object, represent it directly in the program; model real-world objects, and abstractions directly in code. It’s easier now.

Concurrent programming is better supported in C++ 11, important for getting the best performance from modern hardware.

It is curious how the programming landscape has changed in recent year. A few years back, you might have foreseen a day when most programming would be .NET, Java or JavaScript: all varieties of managed code. While those languages do still dominate, native code has come more to the fore, thanks to factors like Apple’s focus on Objective C, and signs of internal conflict at Microsoft over the best language for coding Windows applications.

That said, C++ 11 remains a demanding language to learn and use. As Stroustrup notes, since C++ 11 is a superset of C++ 98 it is technically harder to learn all of it, though new libraries and abstractions should help beginners. The reasons for using or not using C++ are not going to change significantly with this new standard.

apple, development, open source, professional, software development

Building PasswordSafe for the Mac: Lion development hassles

10 Comments

I am doing some work on a Mac at the moment. On Windows I store passwords in PasswordSafe, an open source utility that works well, so I wondered if I could access my PasswordSafe database from the Mac.

image

I could have run the Windows version in Parallels, which I have just installed, but I figured a Mac version would be more convenient. I didn’t see a Mac build among the downloads, but PasswordSafe is cross-platform, so I downloaded the source to do a quick compile.

I was glad to find README.MAC.DEVELOPERS.txt in the PasswordSafe source and set to work. The first task is to download wxWidgets, a cross-platform GUI library, so I went off to download that. Ran the osx-build-wx script as instructed. Result: error message stating C compiler cannot create executables.

The problem seems to be that PasswordSafe expects GCC 4.0 but the latest Xcode installs GCC 4.2. The solution suggested here is to remove Xcode 4, install Xcode 3, and then reinstate Xcode 4. There are related issues concerning PPC fat binaries and older versions of the Mac SDK.

That solution seemed risky and ardous to me, and I remembered that I still had an old Mac Mini from which I was forced to upgrade in order to install Lion, the latest OS X. I hooked it up, removed Xcode 4, installed Xcode 3, and set to work again.

I get the impression not many people build PasswordSafe for the Mac. The first issue I discovered was that the steps in in the README.MAC.DEVELOPERS.txt don’t mention that after running osx-build-wx you also have to run make in order to build static libraries. That was easy though. The next thing is to load the supplied PasswordSafe project into Xcode and build.

I did that but got an error – the linker could not resolve SizeRestrictedPanel. The fix was to add SizeRestrictedPanel.cpp and SizeRestrictedPanel.h to the project. PasswordSafe then built and seems to work fine, on Lion as well as earlier versions of OS X, though there are a few cosmetic issues. You can see from the image that the caption for the New Database button is slightly awry.

If anyone wants my build, it is here. There is also a Java version, and some people have success with that on the Mac.

.net, apple, mobile, professional, software development

Mozilla to take on the cross-platform app challenge

2 Comments

Mozilla is facing an uncertain future. Its problem: basing a business (even a non-profit one) on being the alternative to Microsoft’s Internet Explorer is no longer sensible, given that Apple and Google are now doing this too, and even Microsoft is now investing in HTML 5. I discussed these issues in more detail here.

So what is Mozilla to do? Mozilla Chair Mitchell Baker has posted about a possible new approach, based on being the alternative to Apple for apps. She lists some of the problems with the current “app experience”. Apps are device-specific, require permission at many levels, and a few App Store owners (mainly Apple but also Google) control the business model and customer relationships.

Mozilla is proposing what I presume is a new app platform, which will be cross-platform and cross-device. Instead of discovering apps in a single app store, she envisages multiple providers and the ability to find apps in the same way we find web content.

In other words, if the old Mozilla was about freedom from Microsoft and allowing web technology to progress, the new Mozilla might be about freedom from Apple and allowing app technology to progress.

It is a bold vision and one that in principle would be welcome. That said, Mozilla cannot change the control Apple has over its platform, and its insistence that apps are installed only through its own App Store. Maybe she has in mind a cross-platform toolkit, or browser-based apps, or some combination.

Another snag is that whereas there was widespread dissatisfaction with Microsoft’s Internet Explorer back in 2004 when Firefox was launched, this is not the case with Apple and its app platform today. Apple’s App Store system undoubtedly has a dark side, but the user experience is good and developers are making money, some of them at least. Apple’s control over app installation and the constraints imposed on what apps can do are also good for security.

Nevertheless, having looked at a number of cross-platform mobile toolkits, from PhoneGap to Appcelerator Titanium to Adobe AIR, I can see both the significance of this kind of development and that there is plenty of scope for improvement.

microsoft, software development

Charlie Kindel leaving Microsoft

3 Comments

Charlie Kindel, Windows Phone 7 developer champion at Microsoft, is leaving to start a new company, though he is vague about exactly what it will be:

Charlie will be staying in the Seattle area building a new tech company. The new company will be in stealth mode initially but involves advertising, mobile, cloud computing, and youth athletics.

says the press release.

He goes back a long way at Microsoft, 21 years to be precise. Among the accomplishments he claims:

  • Built ActiveX and DCOM
  • Shipped Internet Explorer 3.0
  • Drove the development of the home networking features in Windows XP
  • Founded eHome and shipped the first version of Windows Media Center
  • Drove the invention of Windows Smart Displays and Windows Media Center Extenders
  • Was the driving force behind Windows Home Server.

I remember IE 3.0 coming out. Surprising though it may seem today, it was an impressive achievement, though history has not been kind either to ActiveX or to DCOM.

.net, microsoft, software development, visual studio

Real-world Microsoft Team Foundation Server: Not very good, says ThoughtWorks

12 Comments

I spoke to Sam Newman, who is European Continuous Delivery Practice Lead at ThoughtWorks, a software development company. Needless to say, we talked extensively about Continuous Delivery and I will be reporting on this separately; but I was also interested in his comments on Microsoft’s Team Foundation Server (TFS).  He told me that ThoughtWorks teams often end up working with it at their .NET clients, but it is problematic. In one case, he said, 6% of productive time was absorbed dealing with TFS.

What was the problem, performance, bugs, features lacking?

When we’ve looked at the problems we’ve had, a lot of it unfortunately comes down to the version control system. It’s not very good. It’s slow, you can’t do rollbacks, sometimes things go missing, you get locks. When we talked about 6% of time, they were things like waiting for a solution to expand in Visual Studio. A lot of those issues are in the version control system.

A frustration is that you cannot use TFS with any version control system other than its own.

Every other build server in the world, from Anthill to Go to Cruise Control to Hudson, you can put in at least 10 version control systems. In TFS they are all coupled. So you can’t take the version control and point it at Subversion. That might resolve a lot of the issues.

Why is TFS so widely used? It is because it comes in the box, says Newman.

I can’t think of a single client that wanted a tool, went out into the marketplace, and selected TFS because it is the right tool for them. Most clients use TFS because it comes with their Enterprise MSDN licence.

I have tried TFS myself and found it pretty good; but then I am just testing it on small projects as a solo developer, so it is hard for me to replicate the experience of a real-world team. You would have thought that performance issues, such as waiting ages for a solution to expand in Visual Studio, could be solved by tracing the reason for the delay; but apparently this is not easy.

This is anecdotal evidence, and of course there may be plenty of TFS installations out there that work very well; I would be interested in hearing of counter examples. I am also not sure to what extent the problems apply to all versions of TFS, or whether there is improvement in TFS 2010.

Newman recognizes that anecdotal evidence is not much use: he says ThoughtWorks is trying to collect some solid data that can be used both to discuss with clients making version control and build system choices, and with Microsoft.

Performance is a feature, and makes a large contribution to user satisfaction. The first release of Outlook 2007 was extraordinarily slow in some setups, and I remember the pain of clicking on a folder and then waiting tapping my fingers while it thought about expanding. It sounds like some TFS users are having a similar experience but in Visual Studio.

delphi, development, embarcadero, iphone, professional, software development

Delphi for Windows, Mac and iOS: screenshots and video of cross-platform development

14 Comments

Embarcardero is drip-feeding information about its forthcoming RAD Studio XE2 in an annoying manner; nevertheless the product does look interesting and promises cross-platform native code apps for Windows 64-bit, Windows 32-bit, Mac OS X and Apple iOS. I have grabbed some screens from a video recently posted by Embarcadero’s Andreano Lanusse; the video is also embedded below.

Here is Delphi XE2 showing a FireMonkey application in the designer. FireMonkey is a new cross-platform GUI framework.

image

Note the list of target platforms on the right. If you squint you can see 64-bit Windows, OSX, and 32-bit Windows.

image

How do you compile for the Mac? It is clear from the demo that Lanusse is running in a VMWare virtual machine on a Mac. He also has a Remote Profile option set to target the host Mac:

image

He then refers to a “Platform assistant” which you can see running in a terminal window on the Mac.  He is then able to compile and run from the Windows IDE:

image

Finally, he targets iOS, though this is a separate project, not just another target. The process exports the project to Xcode, Apple’s Mac and iOS IDE:

image

Next, we see the app running on the iPad simulator:

image

The ability to target the Mac is nice to have, but I suspect it is iOS that will attract more interest, given the importance of Apple’s mobile platform.

Here’s the complete video where you can perhaps puzzle out a few more details.

Update: there is also some Q&A in the comments here.

Graphics rendering is Direct2D or Direct3D on Windows, OpenGL on Mac. FireMonkey renders all components through the graphics API, it does not support use native OS components, though Embarcadero’s Michael Swindell says:

FireMonkey client area controls are rendered by OpenGL on Mac, but appear and work just like Cocoa controls – or however you want them to. There are many different Cocoa UI styles in OSX apps, and Firemonkey can render any of them – including iTunes, or Prokit which is an Apple UI style for Pro apps like Final Cut, not available to devs via Cocoa. Windows are Cocoa Windows and the client areas and all user controls are rendered by OpenGL in HD(2D) or 3D. Menus are std and rendered by Cocoa in the menu bar, and common dialogs are rendered by Cocoa. If the “true OSX” look isn’t for you, you’re welcome to use any included Style, download a custom style, or create your own custom style.

Swindell also addresses the matter of Linux and Android:

We do plan Linux and Android. But no eta yet until we get Win/OSX/iOS out. We would also like to provide language bindings for other languages.

Finally, a bit more about that Platform Assistant:

Developer requires a PC and a Mac (or Mac with VM running Windows). You will develop on Windows, and use the platform assistant (PA running on your Mac) to compile natively to your Mac and the PA handles debugging communication between the Mac and your IDE running on Windows. Delphi (or C++Builder) and Firemonkey create compiled stand alone OSX executables that you can sell/distribute to your users. They are native Mac apps. They “copy install” and run like any other Mac app, or you can use a Mac installer if you like.

adobe, development, mobile, open source, professional, software development

PhoneGap is at version 1.0

I’ve just spotted that PhoneGap has reached version 1.0. The release was announced at PhoneGap day in Portland, on Friday 29th July.

I have spent some time trying out various cross-platform mobile development tools. PhoneGap is among the most interesting and popular, and is also open source and free to use. If you believe that using the browser engine as an application runtime is the most sensible route to cross-platform mobile applications, then PhoneGap is the leading contender. It wraps your application to look like a native app, and also provides ways to call the native API when necessary.

PhoneGap received a boost when Adobe built it into Dreamweaver 5.5. I tried it out and was impressed with the design environment, but I am not sure how serious Adobe is about PhoneGap since there is no documentation on how to package your PhoneGap app for release, and my post has comments from puzzled users. My solution was to export the project to Eclipse and the standard PhoneGap tools, which misses part of the value of having it integrated into Dreamweaver.

Adobe installs PhoneGap into the Dreamweaver directory, so another issue is how to take advantage of the latest version if you are using Adobe’s tools. Overall I would suggest that using the PhoneGap SDK and Eclipse is a better option, though there is no problem with bringing in Dreamweaver for parts of the design.

I interviewed Nitobi president André Charland about PhoneGap earlier this year.