ThoughtWorks bemoans excessive software complexity, advocates small, focused services

ThoughtWorks has released its latest Technology Radar, an opinionated analysis of software development trends.

Things the folk at ThoughtWorks like include automated build and deployment, essential for Continuous delivery; NOSQL database managers especially Neo4j; mobile-first development; the AppCode IDE for Apple’s Objective-C; the Graphite realtime graphing tool for creating dashboards; Clojure and Scala for programming.

I meet some of the ThoughtWorks team at developer conferences from time to time, and generally find them smart and though-provoking to talk to. They must be the despair of the big enterprise software vendors, with a liking for open source and an aversion to heavyweight high-maintenance systems.

This remark particularly caught my eye:

Simple architectures—Simple continues to gain traction, including both techniques for building and composing applications, as well as infrastructure-based techniques to enable simple deployment, failover and recovery. This theme is a recurring one for us, but we have not yet seen the usage shifts we believe are necessary.

I asked consultant James Lewis and practice lead Sam Newman to expand on that. Why do we continue to choose complexity over simplicity?

“A lot of people like to stay inside their big box, and don’t understand the complexities that then creates,” said Newman. “There’s a lack of critical thought given to how services talk to each other. A lot of them are driven by whatever the vendor says you do. Java makes RMI very easy. [Microsoft] .NET makes binding to WSDL [SOAP] schemas very easy. All these tools make bad things very easy to do.

“When you start talking to organisations about smaller services that are focused on doing one thing well, they have the horror associated with now having more than one box to manage and operate. So it’s hard to talk about moving from one big monolithic complicated box that is hard to change, to lots of little boxes, without also having conversations with those clients about how they get better at managing multiple services.

“Netflix has 300 services. Each service runs on at least six machines. They are very good at deploying those services. Yet they are not an overly complicated domain compared to some of our clients.”

“Amazon were talking about this in 2004,” adds Lewis, “the idea that you join up development of these small, simple applications with the operational control, so the same people who build them are also the people who run them. Now that we’re seeing both private and public clouds, and the ability to spin up machines becoming more and more prevalent, its starting to become more attractive.”

This is a consistent theme from ThoughtWorks. Break up complex solutions into many small services, think about how they talk to each other (with REST and HTTP favoured), and think about the infrastructure and how to automate it as well as the software itself.

“In many organisations these conversations are happening,” Newman told me. “I go to clients now, and they talk about the Enterprise Service Bus as being something they’d love to get rid of if they only knew how. Five to ten years ago, to even mention the Enterprise Service Bus as being a problem, they’d look at you with daggers in their eyes.”

“It’s almost like we’re now able to fulfil the promise of service orientation,” says Lewis. “It needed these additional practices, around things like automated deployment, automated rollback, and an understanding that people and process are tied intrinsically with it.”

Another issue, claims Lewis, is that software architects simply get out of touch with best practice.

“Most architects who build big systems are quite a long way from their codebases. They sit in rooms talking to other architects. They might have last written a line of code five or ten years ago. What they do is to design systems as they would have done ten years ago. People do get divorced from the latest trends and perpetuate less effective ways of doing things.”

Finally, here’s something for the Microsoft platform people who read this site. ThoughtWorks is not altogether averse to Microsoft and mentions the Azure cloud platform as something which is becoming interesting. But Windows Phone:

Despite a promising start to Windows Phone, a well thought-out user interface, and probably the best development experience of any mobile platform, we have seen several stumbles in the execution of the platform strategy by Microsoft and its partners. This makes us less optimistic about the future of the platform than we were in the last radar.

Translation: nice mobile platform, but nobody’s buying it. Then again, on Monday next week Windows Phone 8 will be properly unveiled. Still hope?