Tag Archives: testing

Xamarin acquires LessPainful, announces Test Cloud for mobile apps

Xamarin, a company which provides tools for cross-platform development in C#, has announced its acquisition of LessPainful and the creation of cloud-based testing for mobile apps based on LessPainful’s technology and the Calabash scripting language it created.

The Test Cloud will perform automated user-interface tests on real devices, hosted by Xamarin, will provide detailed reports in the event of test failures, and will support Continuous Integration so that bugs are caught as early as possible.


“After you’ve conquered the cross-platform mobile development problem, testing is the next large pain point” says Xamarin CEO Nat Friedman. “You can’t just get by with manual testing. There’s a need for the same level of tools and processes in mobile testing that you have in desktop and web testing.”

“Quality is actually more important on mobile than in other places. Mobile sessions are very short. People are really intolerant of low quality on mobile. The release cycles are shorter too. People are revving more frequently, and testing is a bigger challenge.”

Another issue with mobile testing is the number of devices out there, especially if you throw cross-platform into the mix. “You have on Android all these manufacturers who customise the OS in different ways, you have multiple different versions that are in use, and you have multiple different form factors and device capabilities. The testing permutation matrix is huge.”

“Automated UI testing is the only kind of testing that can ensure that the app does what it is supposed to do.”

Friedman says that the Xamarin UI tests are more robust than competing UI test frameworks because they do not depend on UI image recognition. “The right answer is object-based, you identify user interface elements on the screen by object IDs”.”

How does testing on real devices work? If you have 50 developers testing on 27 devices in Xamarin’s cloud, will there be racks and racks of devices to support them?  “That’s what it looks like at our end, racks and racks of devices,” confirmed Friedman. “The service is going to be built based on device/hour usage. We’ll be able to scale up to match what people need.

“We talk to developers who spend $8,000 a month just to get new devices. That’s not counting the labour and everything else they need to do, to set up their own testing infrastructure. It’s a giant pain point.”

Xamarin’s Test Cloud will offer plug-ins for Jenkins, TeamCity, and Microsoft’s Team Foundation Server, to support Continuous Integration.

The scripting language for the Test Cloud is either Calabash , or C# scripting which is under development by Xamarin.

The Test Cloud is not just for applications developed using Xamarin’s C# framework, but also supports other frameworks including those written in native iOS Objective C. However, only iOS and Android are supported.

Availability is set for the third quarter of 2013.

Xamarin’s Evolve conference is currently under way in Austin, Texas, with around 600 developers in attendance. Friedman says the company is growing fast. 1000 developers a day download the tools, there are over 15,000 paid developers, and the company now has 65 employees.

More information on the Xamarin Test Cloud is here.

What’s coming in Microsoft Visual Studio

Microsoft is beginning to talk in detail about the next version of Visual Studio, though currently mostly in the area of ALM (Application Lifecycle Management) tools.

Continuous testing and support for diverse test frameworks

The new Visual Studio will support unit tests that run in the background. Visual Studio VP Jason Zander adds that:

With Visual Studio vNext we are enabling you to use your favorite unit testing framework integrated deeply into the IDE. We will support MS Test, xunit, and nunit with vNext. You will also be able to target both .NET and native C++ code. Adding test frameworks is an extensibility point as well so if you don’t see your favorite one listed here, you can easily add it.

Storyboarding in PowerPoint
This is not exactly a Visual Studio feature; but the new version will include a PowerPoint plug-in and templates that lets you mock up a user interface. Why bother, when Microsoft already has Sketchflow in Expression Blend, and tools in Visio for laying out a GUI? Apparently many users are more comfortable in Office.

Integration with System Center

Visual Studio 2010 already includes a virtual lab management feature that lets you test applications on virtual machines managed by System Center Virtual Machine Manager. But what about deployed applications? A new connector for System Center Operations Manager brings similar integration, so that bugs can be reported directly to Team Foundation Server complete with stack trace enabled by IntelliTrace, a historical debugging feature.

Context switching

The thinking here is that when developers are interrupted they lose the flow of their work. Context switching lets you shelve code changes, open windows and other activity tied to the current task. You can then do other work; when you later resume the task Visual Studio recovers its state.

New Team Explorer

Team Explorer is the connector and window in Visual Studio that forms the client for Team Foundation Server. This has been revamped for the new version, and now uses “full asynchronous communication” to improve load time and responsiveness. There are new views for common categories of information, including work items, pending changes, builds, reports and bugs.

New Agile collaboration tools

There are new tools in the Web Access client for Team Foundation Server for feedback and collaboration on projects using Agile methodology. Backlogs shows features to be implemented in a sprint, a unit of project iteration. The Task Board shows the backlog in a new visual view.

Connector for Project Server

A new connector for Project Server enables project-style views of project progress, such as Gantt charts.

Feedback tools

A new feedback mechanism aimed at stakeholders lets users enter feedback into Team Foundation Server. Tools include a web recorder that lets users comment on actions in a web application with linked recordings.

Code Clone Detection

This is a code quality feature that analyses a project looking for common code that should be refactored into a shared block.

Code Review

Code Review lets team members comment on code, similar in some ways to a commented document in Word.

Hosted Team Foundation Server on Azure

“Any team up and running within 30 seconds” is Microsoft’s claim for a new hosted option for Team Foundation Server. An exaggeration no doubt; but since a full-featured TFS takes some effort and infrastructure to implement, the hosted option will be welcome.

Visual Studio tends to be synchronized to some extent with new versions of Windows, so I would guess we will learn more about Visual Studio vNext at the Professional Developers Conference (though it may be called something else) in Anaheim on September 13-16 this year.

You can read more about Visual Studio vNext on Jason Zander’s blog and in a white paper [pdf].