Now that I have a lovely ITWriting.com App I thought I should check out whether it is ready to fly.
I therefore ran the App Certification Kit that installs with Visual Studio 2012.
The tool asks you to select an installed app and then exercises it. I saw my app open, though I did not see it get beyond the first screen.
Eventually – bad news:
However, there is only one thing wrong with it:
Yes, the version installed is the debug build. I can fix this simply by rebuilding in release configuration.
What does the Kit test? Here is the list:
- Crashes and hangs test
- App manifest compliance test
- Windows security features test
- Supported API test
- Performance test
- App manifest resources test
- Debug configuration test
- File encoding
- Direct3D feature level support
- App Capabilities test
- Windows Runtime metadata validation
That sounds most impressive and makes a great list for you to show to your customer.
I am sceptical though. If the app was not exercised beyond the opening screen, might it not be a bit buggy after all?
I inserted the following line of code into the the Click event handler for reading a blog:
int iCrash = 1 / string.Empty.Length;
I then rebuilt the app in release mode and ran the App Certification Test. Great news!
Thanks though to my umm, bug-unfix, the app crashes whenever I click to read a blog.
I mention this not to poke fun at the App Certification Kit, but to observe that it does not do a good job of automatically detecting crashes and hangs.
The implication is that the human testers are the ones who will do this before an app is admitted to the store. I think they would find my obvious bug; but how much time will they have to test every feature of an app?