Tag Archives: microsoft

Should you convert your Visual Basic .NET project to C#? Why and why not…

When Microsoft first started talking about Roslyn, the .NET compiler platform, one of the features described was the ability to take some Visual Basic code and “paste as C#”, or vice versa.

Some years later, I wondered how easy it is to convert a VB project to C# using Roslyn. The SharpDevelop team has a nice tool for this, CodeConverter, which promises to “Convert code from C# to VB.NET and vice versa using Roslyn”. You can also find this on the Visual Studio marketplace. I installed it to try out.

image

Why would you do this though? There are several reasons, the foremost of which is cross-platform support. The Xamarin framework can use VB to some extent, but it is primarily a C# framework. .NET Core was developed first for C#. Microsoft has stated that “with regard to the cloud and mobile, development beyond Visual Studio on Windows and for non-Windows platforms, and bleeding edge technologies we are leading with C#.”

Note though that Visual Basic is still under active development and history suggests that your Windows VB.NET project will continue running almost forever (in IT terms that is). Even Visual Basic 6.0 applications still run, though you might find it convenient to keep an old version of Windows running for the IDE.

Still, if converting a project is just a right-click in Visual Studio, you might as well do it, right?

image

I tried it, on a moderately-size VB DLL project. Based on my experience, I advise caution – though acknowledging that the converter does an amazing job, and is free and open source. There were thousands of errors which will take several days of effort to fix, and the generated code is not as elegant as code written for C#. In fact, I was surprised at how many things went wrong. Here are some of the issues:

The tool makes use of the Microsoft.VisualBasic namespace to simplify the conversion. This namespace provides handy VB features like DateDiff, which calculates the difference between two dates. The generated project failed to set a reference to this assembly, generating lots of errors about unknown objects called Information, Strings and so on. This is quick to fix. Less good is that statements using this assembly tend to be more convoluted, making maintenance harder. You can often simplify the code and remove the reference; but of course you might introduce a bug with careless typing. It is probably a good idea to remove this dependency, but it is not a problem if you want the quickest possible port.

Moving from a case-insensitive language to a case-sensitive language is a problem. Visual Studio does a good job of making your VB code mostly consistent with regard to case, but that is not a fix. The converter was unable to fix case-sensitivity issues, and introduced some of its own (Imports System.Text became using System.text and threw an error). There were problems with inheritance, and even subtle bugs. Consider the following, admittedly ugly and contrived, code:

image

Here, the VB coder has used different case for a parameter and for referencing the parameter in the body of the method. Unfortunately another variable with the different case is also accessible. The VB code and the converted C# code both compile but return different results. Incidentally, the VB editor will work very hard to prevent you writing this code! However it does illustrate the kind of thing that can go wrong and similar issues can arise in less contrived cases.

C# is more strict than VB which causes errors in conversion. In most cases this is not a bad thing, but can cause headaches. For example, VB will let you pass object members ByRef but C# will not. In fact, VB will let you pass anything ByRef, even literal values, which is a puzzle! So this compiles and runs:

image

Another example is that in VB you can use an existing variable as the iteration variable, but in C# foreach you cannot.

Collections often go wrong. In VB you use an Item property to access the members of a collection like a DataReader. In C# this is omitted, but the converter does not pick this up.

Overloading sometimes goes wrong. The converter does not always successfully convert overloaded methods. Sometimes parameters get stripped away and a spurious new modifier is added.

Bitwise operators are not correctly converted.

VB allows indexed properties and properties with parameters. C# does not. The converter simply strips out the parameters so you need to fix this by hand. See https://stackoverflow.com/questions/2806894/why-c-sharp-doesnt-implement-indexed-properties if the language choices interest you.

There is more, but the above gives some idea about why this kind of conversion may not be straightforward.

It is probably true that the higher the standard of coding in the original project, the more straightforward the conversion is likely to be, the caveat being that more advanced language features are perhaps more likely to go wrong.

Worth it?

So is it worth it? It depends on the rationale. If you do not need cross-platform, it is doubtful. The VB code will continue to work fine, and you can always add C# projects to a VB solution if you want to write most new code in C#.

If you do need to move outside Windows though, conversion is worthwhile, and automated conversion will save you a ton of manual work even if you have to fix up some errors.

There are two things to bear in mind though.

First, have lots of unit tests. Strange things can happen when you port from one language to another. Porting a project well covered by tests is much safer.

Second, be prepared for lots of refactoring after the conversion. Aim to get rid of the Microsoft.VisualBasic dependency, and use the stricter standards of C# as an opportunity to improve the code.

Microsoft’s Dynamics CRM 2016/365: part brilliant, part perplexing, part downright sloppy

I have just completed a test installation of Microsoft’s Dynamics CRM on-premises; it is now called Dynamics 365 but the name change is cosmetic, and in fact you begin by installing Dynamics CRM 2016 and it becomes Dynamics 365 after applying a downloaded update.

Microsoft’s Dynamics product has several characteristics:

1. It is fantastically useful if you need the features it offers

2. It is fantastically expensive for reasons I have never understood (other than, “because they can”)

3. It is tiresome to install and maintain

I wondered if the third characteristic had improved since I last did a Dynamics CRM installation, but I feel it has not much changed. Actually the installation went pretty much as planned, though it remains fiddly, but I wasted considerable time setting up email synchronization with Exchange (also on-premises). This is a newish feature called Server-Side Synchronization, which replaces the old Email Router (which still exists but is deprecated). I have little love for the Email Router, which when anything goes wrong, fills the event log with huge numbers of identical errors such that you have to disable it before you can discover what is really going wrong.

Email is an important feature as automated emails are essential to most CRM systems. The way the Server-Side Synchronization works is that you configure it, but CRM mailboxes are disabled until you complete a “Test and Enable” step that sends and receives test emails. I kept getting failures. I tried every permutation I could think of:

  • Credentials set per-user
  • Credentials set in the server profile (uses Exchange Impersonation to operate on behalf of each user)
  • Windows authentication (only works with Impersonation)
  • Basic authentication enabled on Exchange Web Services (EWS)

All failed, the most common error being “Http server returned 401 Unauthorized exception.” The troubleshooting steps here say to check that the email address of the user matches that of the mailbox; of course it did.

An annoyance is that on my system the Test and Enable step does not always work (in other words, it is not even tried). If I click Test and Enable in the Mailbox configuration window, I get this dialog:

image

However if I click OK, nothing happens and the dialog stays. If I click Cancel nothing happens and the dialog stays. If I click X the dialog closes but the test is not carried out.

Fortunately, you can also access Test and Enable from the Mailbox list (select a mailbox and it appears in the ribbon). A slightly different dialog appears and it works.

I was about to give up. I set Windows authentication in the server profile, which is probably the best option for most on-premises setups, and tried the test one more time. It worked. I do not know what changed. As this tech note (which is about server-side synchronization using Exchange Online) remarks:

If you get it right, you will hear Microsoft Angels singing

But what’s this about sloppy? There is plenty of evidence. Things like the non-functioning dialog mentioned above. Things like the date which shows for a mailbox that has not been tested:

image

Or leaving aside the email configuration, things like the way you can upload Word templates for use in processes, but cannot easily download them (you can use a tool like the third-party XRMToolbox).

And the script error dialog which has not changed for a decade.

Or the warning you get when viewing a report in Microsoft Edge, that the browser is not supported:

image

so you click the link and it says Edge is supported.

Or even the fact that whenever you log on you get this pesky dialog:

image

So you click Don’t show this again, but it always reappears.

It seems as if Microsoft does not care much about the fit and finish of Dynamics CRM.

So why do people persevere – in fact, the Dynamics business is growing for Microsoft, largely because of Dynamics 365 online and its integration with Office 365. The cloud is one reason, which removes at least some of the admin burden. The other thing though is that it does bring together a set of features that make it invaluable to many businesses. You can use it not only for sales and marketing, but for service case management, quotes, orders and invoices.

It is highly customizable, which is a mixed blessing as your CRM installation becomes increasingly non-standard, but does mean that most things can be done with sufficient effort.

In the end, it is all about automation, and can work like magic with the right carefully designed custom processes.

With all those things to commend it, it would pay Microsoft to work at making the user interface less annoying and the administration less prone to perplexing errors.

All the way from 1997: Compaq PC Companion C140 still works, but as badly as it did on launch

I am having a clear-out which is bringing back memories and unearthing some intriguing items. One is this Compaq C140 PC Companion, running Windows CE, which launched in December 1997.

image

The beauty of this device is that it takes two AA batteries. I stuck in some new ones and found that it started up fine, not bad after more than 20 years. Most more recent devices have a non-replaceable rechargeable battery which usually fails long before the rest of the electronics, rendering the entire device useless (at least without surgery).

The C140 runs Windows CE 1.0 and has a monochrome touch screen designed to be used mainly with a stylus. 4MB RAM, 4MB storage, and comes with versions of Word, Excel, Calendar, Contacts and Tasks. There is also a calculator and a world clock. It is expandable with PCMCIA cards (though not many have drivers). The idea is that you link it to your PC with the supplied serial cable and synch with Outlook, hence PC Companion.

The odd thing is, looking at this device I still find it superficially compelling. A pocketable device running Word and Excel, with a full QWERTY keyboard, stylus holder so you do not lose it, what’s not to like?

A lot, unfortunately. The biggest problem is the screen. There is a backlight and a contrast dial, but it is faint and hard to read in most lights and you constantly fiddle with the contrast.

The next issue is the keyboard. It is too cramped to type comfortably. And the format, though it looks reassuringly like a small laptop, is actually awkward to use. It works on a desk, which seems to miss the point, but handheld it is useless. You need three hands, one for the device, one for the stylus, and a third for typing. The design is just wrong and has not been thought through.

I have searched for years for small portable devices with fast text input. I suppose a smartphone with a Swype keyboard or similar comes closest but I am still more productive with a laptop and in practice the thing that has made most improvement for me is that laptops have become lighter and with longer battery life.

Spare a thought though for Microsoft (and its partners) with its long history of trying to make mobile work. You can argue back and forth about whether it was right to abandon Windows Phone, but whatever your views, it is a shame that decades of effort ended that way.

Gartner on Mobile App Development Platforms: Kony, Mendix, Microsoft, Oracle and Outsystems the winners

Gartner has published a paper and Magic Quadrant on Mobile App Development Platforms (MDAPs), which you can read for free thanks to Progress, pleased to be named as a “Visionary”, and probably from other sources.

According to Gartner, an MDAP has three key characteristics:

  • Cross-platform front-end development tools
  • Back-end services that can be used by diverse clients, not just the vendor’s proprietary tools.
  • Flexibility to support public and internal deployments

Five vendors ranked in the sought-after “Leaders” category. These are:

  • Kony, which offers Kony Visualizer for building clients, Kony Fabric for back-end services, and Kony Nitro Engine, a kind of cross-platform runtime based on Apache Cordova .
  • Mendix, which has visual development and modeling tools and multi-cloud, containerised deployment of back-end services
  • Microsoft, which has Xamarin cross-platform development, Azure cloud services, and PowerApps for low-code development
  • Oracle, which has Oracle Mobile Cloud Enterprise including JavaScript Extension Toolkit and deployment via Apache Cordova
  • Outsystems, a low-code platform which has the Silk UI Framework and a visual modeling language, and hybrid deployment via Apache Cordova

Of course there are plenty of other vendors covered in the report. Further, because this is about end-to-end platforms, some strong cross-platform development tools do not feature at all.

A few observations. One is the prominence of Apache Cordova in these platforms. Personally I have lost enthusiasm for Cordova, now that there are several other options (such as Xamarin or Flutter) for building native code apps, which I feel deliver a better user experience, other things being equal (which they never are).

With regard to Microsoft, Gartner notes the disconnect between PowerApps and Xamarin, different approaches to application development which have little in common other than that both can be used with Azure back-end services.

image
Microsoft PowerApps

I found the report helpful for its insight into which MDAP vendors are successfully pitching their platform to enterprise customers. What it lacks is much sense of which platforms offer the best developer experience, or the best technical capability when it comes to solving those unexpected problems that inevitably crop up in the middle of your development effort and take a disproportionate amount of time and effort to solve.

Microsoft’s strong financials, and some notes on Azure vs AWS and the risks of losing in mobile

Microsoft delivered another strong set of figures in its latest financial results, for the period April-June 2018. Total revenue of $30.085 million was up 17% year on year, and all three of the company’s sectors (Office, Azure and consumer) showed strong growth.

What’s notable? Largely this is more of the same. A few things to note. Linked in revenue increased 37% year on year – an acquisition that seems to be making sense for the company. Dynamics 365 revenue grew by 65%. The Dynamics story is all about cloud synergy. As an on-premises product Dynamics CRM (the part of the suite I know best) was relatively undistinguished but as a cloud product the seamless integration between Office 365 and Dynamics 365 (and Azure Active Directory) makes it compelling.

Windows 10 is doing OK, possibly as more businesses heave themselves off Windows 7 and buy new PCs with OEM licenses as they do.

Even areas in which Microsoft is far from dominant did well. Gaming was up 39%, Surface 25% and Search advertising up 17%.

The biggest growth in the quarter, according to the breakdown here, was in Azure. up 89%. This growth is not without pain; the Register reports capacity issues in the UK South region, for example, with users getting the message “Unfortunately, due to high demand for virtual machines in this region, we are not able to approve your quota request at this time.” You can still create VMs, but not necessarily in the region you want.

Will Microsoft outpace AWS? My take on this has not changed. AWS does very little wrong and remains the pre-eminent cloud for IAAS and many services by some distance. What AWS does not have is Office 365, or armies of Microsoft partners helping enterprise customers to shunt more and more of their IT infrastructure into Azure. Microsoft makes more money from licensing: Windows Server, SQL Server, Office 365 and Dynamics seats, and so on. AWS does more business at a lower margin. These are big differences. I see it as unlikely that Azure will overtake AWS in the provision of essential cloud services like VMs, containers, cloud storage and so on. AWS also has a better reliability track record. However, the success of Azure means that enterprise customers no longer need to go to AWS to get the benefits of cloud. Perhaps the more interesting question is the extent to which AWS (or Google) can persuade enterprise customers to shift away from Microsoft’s high-margin applications.

Longer term, there is significant risk for the company in its retreat from mobile. We are now seeing Google work hard in the laptop market with Chromebooks alongside Android mobile. Coming sometime is Google Fuchsia which may be a single operating system for both. It is worth recalling that Microsoft built its success on winning users for its PC operating system; and that IBM lost its IT dominance by ceding this to Microsoft.

Here is the breakdown by segment, such as it is:  

Quarter ending June 30th 2018 vs quarter ending June 30th 2017, $millions

Segment Revenue Change Operating income Change
Productivity and Business Processes 9668 +1140 3466 +575
Intelligent Cloud 9606 +1784 3901 +990
More Personal Computing 10811 +1576 3012 +826

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

Microsoft announces free version of Teams, ahead of Inspire partner conference

Microsoft’s partner conference, Inspire, kicks off in Las Vegas next week; and as part of the event the company has announced big news concerning Teams: a free version.

image

What is Teams? It is a collaboration tool for Office 365, or at least it was, since the new free version can be used with any email address and without Office 365. Here is what you get:

  • Chat
  • Audio and video calling
  • 10GB online storage, plus 2GB for each additional team member (SharePoint/OneDrive)
  • Word, Excel and PowerPoint online
  • Ability to install unlimited additional applications

Teams is a strategic product for Microsoft – see here for the reason. A free version is way for the company to promote Office 365, and you will see an upgrade link in the user interface.

There are also new features coming to Teams. One seems minor, but will be popular. It deals with the problem of video conferencing from home, and not being sure what may happen behind you. You may remember this:

image

So now Teams video conferencing will let you blur the background. Here is Raanah Amjadi, Marketing Manager, Microsoft Teams, demonstrating the feature:

image

In addition, Teams is getting a new Live Events feature. This is where you broadcast a presentation or meeting to others in your company. Automatic speech-to-text will do close captions (so you can watch with the sound done, if you trust it enough), and this then enables text search of the event with index points into the video. Bing Translate is also included in Teams so you can have multi-lingual conversations.

image

Microsoft Workplace Analytics is getting enhancements including “My Analytics” which will give you AI-powered “nudges” in Outlook online. I am not sure I trust this to be much real-world use; but the example shown was intriguing: alert you if you try to schedule a meeting with someone out of their working hours.

Whiteboard, a collaboration canvas, is now generally available for Windows 10 and mobile.

image

Free Teams is available immediately here.

Account options when setting up Windows 10, and Microsoft’s enforced insecurity questions

How do you sign into Windows 10? There are now four options. I ran through a Windows 10 setup using build 1803 (which was released in April this year) and noted how this has evolved. Your first decision: is this a personal or organisational PC?

image

If you choose Setup for an organisation, you will be prompted to sign into Office 365, also known as Azure AD. The traditional Domain join, for on-premises Active Directory, has been shunted to a less visible option (the red encircling is mine). In larger organisations, this tends to be automated anyway.

image

But this one is personal. It is a similar story. You are prompted to sign in with a Microsoft account, but there is another option, called an Offline account (again, the red circle is mine).

image

This “Offline account” was in Windows 7 and earlier the only option for personal accounts. I still recommend having an administrative “offline account” set up so you can always be sure of being able to log into your PC, even without internet. Think about some of the scenarios. Someone might hack your Microsoft account, change your password, and now you cannot even log onto your PC. Unless you have an offline account.

I’ve been awkward and selected Offline account. Windows, or rather Microsoft, does not like it. Note the mind games in the screenshot below. Although I’ve made a positive selection for Offline account, the default and highlighted option now is to change my mind. I do not like this.

image

Now I can set up my offline account. A screen prompts for a username, then for a password, all the time nagging that I should create an online account instead.

image

I type and confirm the password; but now I get this:

image

Yes, I have to create “security questions”, with no option to skip. If you try to skip, you get a “This field is required” message. Worse still, they are from a pre-selected list:

image

I really hate this. These are not security questions; they are insecurity questions. Their purpose is to let me (or someone else) reset the password, forming a kind of back door into the PC. The information in the questions is semi-secret; not impossible for someone determined to discover. So Microsoft is insisting that I make my account less secure.

Of course you do not have to give honest answers. You can call your first pet yasdfWsd9gAg!!hea. But most people will be honest.

Does it matter, given that a PC account offers rather illusory security anyway? Unless you encrypt the hard drive, someone who steals the PC can reset the password by booting into Linux, or take out the disk and read it from another PC. All true; but note that Microsoft makes it rather easy to encrypt your PC with Bitlocker, in which case the security is not so illusory.

Just for completeness, here is what comes next, an ad for Cortana:

image

Hey Cortana! How do I delete my security answers?

I do get why Microsoft is doing this. An online account is better in that settings can roam, you can use the Store, and you can reset the password from one PC to restore access to another. The insecurity questions could be a life-saver for someone who forgot their password and need to get back into their PC.

But such things should be optional. There is nothing odd about wanting an offline account.

Surface Go: Microsoft has another go at a budget tablet

Microsoft has announced Surface Go, a cheaper, smaller model to sit at the budge end of its Surface range of tablets and laptops.

The new model starts at $399, will be available for pre-order today in selected territories, and ships on August 2nd.

In the UK, the Surface Go is £379 inc VAT for 4GB RAM and 64GB storage, or £509.99 inc VAT for 8GB RAM and 128GB storage.

image

I go back a long way with Surface, having been at the launch of the first device, Surface RT, back in 2012. The device was a flop, but I liked it. The design was genuinely innovative and sought to make sense of a Windows in transition from desktop-only to a viable touch/tablet device. It failed primarily because of the poor range of available apps, lack of user acceptance for Windows 8, and somewhat underpowered hardware. There were also keyboard issues: the fabric-based Touch keyboard was difficult to use because it gave no tactile feedback, and the Type keyboard less elegant and still somewhat awkward.

Surface Pro came next, and while it was more useful thanks to full Windows 8 and an Intel Core i5 CPU, it was disappointing, with battery life issues and a tendency to stay on in your bag, overheating and wasting battery. There were other niggling issues.

The big disappointment of Surface for me is that even with full, Apple-like control over hardware and software, the devices have not been trouble-free.

Another issue today is that Windows 10 is not designed for touch in the same way as Windows 8. Therefore you rarely see Windows tablets used as tablets; they are almost always used as laptops, even if they are 2-in-1 devices. The original kickstand design is therefore rather pointless. If I got another Surface it would be a Surface Laptop or Surface Book.

Of course they are not all bad. It is premium hardware and some of the devices are delightful to use and perform well. They are expensive though, and I suggest careful comparison with what you can get for the same money from partners like HP, Lenovo and others.

What about this one? Key specs:

  • 10″ screen, kickstand design
  • 1800 x 1200 (217 PPI) resolution
  • 8.3mm thick
  • USB-C 3.1 port, MicroSD, headphone jack socket
  • Intel® Pentium® Gold Processor 4415Y
  • Windows Hello camera supporting face-recognition log in
  • Up to nine hours battery life
  • Intel® HD Graphics 615
  • Display supports Surface Pen with 4096 levels of pressure sensitivity
  • Signature Type Cover with trackpad supporting 5-point gestures
  • Windows Hello face authentication camera (front-facing)
  • 5.0 MP front-facing camera with 1080p Skype HD video
  • 8.0 MP rear-facing autofocus camera with 1080p HD video
  • Single microphone
  • 2W stereo speakers with Dolby® Audio™ Premium

It sounds a great deal for £379 or $399 but you will pay more, for three reasons:

  • The base spec is minimal in terms of RAM and SSD storage and you will want the higher model
  • The Type Cover is essential and will cost – a Pro Type Cover is $159.99 and this may be a bit less
  • The Surface Pen is £99.99 or $99.99

This means your $399 will soon be $550 or more.

It could still be a good deal if it turns out to be a nice device. The Hello camera is a plus point, but where I would particularly recommend a Surface is if you want Pen support. Microsoft is good at this. Unfortunately I do not get on well with pen input, but some people do, and for artists and designers it is a real advantage.

On Microsoft Teams in Office 365, and why we prefer walled gardens to the Internet jungle

Gartner has recently delivered a report called Why Microsoft Teams will soon be just as common as Outlook, which gave me pause for reflection.

The initial success of Office 365 was almost all to do with email. Hosted Exchange at a reasonable cost is a an obvious win for businesses who were formerly on on-premises Exchange or Small Business Server. Microsoft worked to make the migration relatively seamless, and with strong Active Directory support it can be done with users hardly noticing. Exchange of course is more than just email, also handling calendars and tasks, and Outlook and Exchange are indispensable tools for many businesses.

The other pieces of Office 365, such as SharePoint, OneDrive and Skype for Business (formerly Lync) took longer to gain traction, in part because of flaws in the products. Exchange has always been an excellent email server, but in cloud document storage and collaboration Microsoft’s solution was less good than alternatives like DropBox and Box, and ties to desktop Office are a mixed blessing, welcome because Office is familiar and capable, but also causing friction thanks to the need for old-style software installations.

Microsoft needed to up its game in areas beyond email, and to its credit it has done so. SharePoint and OneDrive are much improved. In addition, the company has introduced a range of additional applications, including StaffHub for managing staff schedules, Planner for project planning and task assignment, and PowerApps for creating custom applications without writing code.

We have also seen a boost to the cloud-based Dynamics suite thanks to synergy between this and Office 365.

Having lots of features is one thing, winning adoption is another. Microsoft lacked a unifying piece that would integrate these various elements into a form that users could easily embrace. Teams is that piece. Introduced in March 2017, I initially thought there was nothing much to it: just a new user interface for existing features like SharePoint sites and Office 365/Exchange groups, with yet another business messaging service alongside Skype for Business and Yammer.

Software is about usability as much or more than features though, and Teams caught on. Users quickly demanded deeper integration between Teams and other parts of Office 365. It soon became obvious that from the user’s perspective there was too much overlap between Teams and Skype for Business, and in September 2017 Microsoft announced that Teams would replace Skype for Business, though this merging of two different tools is not yet complete.

image

To see why Teams has such potential you need only click Add a tab in the Windows client. Your screen fills with stuff you can add to a Team, from document links to Planner to third-party tools like Trello and Evernote.

image

This is only going to grow. Users will open Teams at the beginning of the day and live there, which is exactly the point Garner is making in its attention-grabbing title.

A good thing? Well, collaboration is good, and so is making better use of what you are paying for with an Office 365 subscription, so it has merit.

The part that troubles me is that we are losing diversity as well as granting Microsoft a firmer hold on its customers.

It all started with email, remember. But email is a disaster, replete with unwanted marketing, malware links, and some number of communications that have some possible value but which life is too short to investigate. In the consumer world, people prefer the safer world of Facebook Messenger or WhatsApp, where messages are more likely to be wanted. Email is also ancient, hard to extend with new features, and generally insecure.

Business-oriented messaging software like Slack and now Teams have moved in, to give users a safer and more usable way of communicating with colleagues. Consumers prefer Facebook’s walled garden to the internet jungle, and business users are no different.

It is a trade-off though. Email, for all its faults, is open and has multiple providers. Teams is not.

This will not stop Teams from succeeding, even though there are plenty of user requests and considerable dissatisfaction with the current release. Performance can be poor, the clients for Mac and mobile not as good as for Windows, and there is no Linux client at all.

Third-parties with applications or services that make sense in the Teams environment should hasten to get their stuff available there.

Inside Azure Cosmos DB: Microsoft’s preferred database manager for its own high-scale applications

At Microsoft’s Build event in May this year I interviewed Dharma Shukla, Technical Fellow for the Azure Data group, about Cosmos DB. I enjoyed the interview but have not made use of the material until now, so even though Build was some time back I wanted to share some of his remarks.

Cosmos DB is Microsoft’s cloud-hosted NoSQL database. It began life as DocumentDB, and was re-launched as Cosmos DB at Build 2017. There are several things I did not appreciate at the time. One was how much use Microsoft itself makes of Cosmos DB, including for Azure Active Directory, the identity provider behind Office 365. Another was how low Cosmos DB sits in the overall Azure cloud system. It is a foundational piece, as Shukla explains below.

image

There were several Cosmos DB announcements at Build. What’s new?

“Multi-master is one of the capabilities that we announced yesterday. It allows developers to scale writes all around the world. Until yesterday Cosmos DB allowed you to scale writes in a single region but reads all around the world. Now we allow developers to scale reads and writes homogeneously all round the world. This is a huge deal for apps like IoT, connected cars, sensors, wearables. The amount of writes are far more than the amount of reads.

“The second thing is that now you get single-digit millisecond write latencies at the 99 percentile not just in one region.

“And the third piece is that what falls out of this high availability. The window of failover, the time it takes to failover from one region when a disaster happens, to the other, has shrunk significantly.

“It’s the only system I know of that has married the high consistency models that we have exposed with multi-master capability as well. It had to reach a certain level of maturity, testing it with first-party Microsoft applications at scale and then with a select set of external customers. That’s why it took us a long time.

“We also announced the ability to have your Cosmos Db database in your own VNet (virtual network). It’s a huge deal for enterprises where they want to make sure that no data leaks out of that VNet. To do it for a global distributed database is specially hard because you have to close all the transitive networking dependencies.”

image
Technical Fellow Dharma Shukla

Does Cosmos DB work on Azure Stack?

“We are in the process of going to Azure Stack. Azure Stack is one of the top customer asks. A lot of customers want a hybrid Cosmos DB on Azure Stack as well as in Azure and then have Active – Active. One of the design considerations for multi master is for edge devices. Right now Azure has about 50 regions. Azure’s going to expand to let’s say 200 regions. So a customer’s single Cosmos DB table spanning all these regions is one level of scalability. But the architecture is such that if you directly attach lots of Azure Stack devices, or you have sensors and edge devices, they can also pretend to be replicas. They can also pretend to be an Azure region. So you can attach billions of endpoints to your table. Some of those endpoints could be Azure regions, some of them could be instances of Azure Stack, or IoT hub, or edge devices. This kind of scalability is core to the system.”

Have customers asked for any additional APIs into Cosmos DB?

“There is a list of APIs, HBase, richer SQL, there are a number of such API requests. The good news is that the system has been built in a way that adding new APIs is relatively easy addition. So depending on the demand we continue to add APIs.”

Can you tell me anything about how you’ve implemented Cosmos DB? I know you use Service Fabric. Do you use other Azure services?

“We have dedicated clusters of compute machines. Cosmos DB is a Ring 0 service. So it’s there any time Azure opens a new region, Cosmos DB clusters have provision by default. Just like compute, storage, Cosmos DB is also one of the Ring 0 services which is the bottommost. Azure Active Directory for example depends on Cosmos DB. So Cosmos DB cannot take a dependency on Active Directory.

“The dependency that we have is our own clusters and machines, on which we put Service Fabric. For deployment of Cosmos DB code itself, we use Service Fabric. For some of the load balancing aspects we use Service Fabric. The partition management, global distribution, replication, is our own. So Cosmos DB is layered on top of Service Fabric, it is a Service Fabric application. But then it takes over. Once the Cosmos DB bits are laid out on the machine then its replication and partition management and distribution pieces take over. So that is the layering.

“Other than that there is no dependency on Azure. And that is why one of the salient aspects of this is that you can take the system and host it easily in places like Azure Stack. The dependencies are very small.

“We don’t use Azure Storage because of that dependency. So we store the data locally and then replicate it. And all of that data is also encrypted at rest.”

So when you say it is not currently in Azure Stack, it’s there underneath, but you haven’t surfaced it?

“It is in a defunct mode. We have to do a lot of work to light it up. When we light up it on such on-prem or private cloud devices, we want to enable this active to active pathway. So you are replicating your data and that is getting synchronized with the cloud and Azure Stack is one of the sockets.”

Microsoft itself is using Cosmos DB. How far back does this go? Azure AD is quite old now. Was it always on Cosmos DB / DocumentDB?

“Over the years Office 365, Xbox, Skype, Bing, and more and more of Azure services, have started moving. Now it has almost become ubiquitous. Because it’s at the bottom of the stack, taking a dependency on it is very easy.

“Azure Active Directory consists of a set of microservices. So they progressively have moved to Cosmos DB. Same situation with Dynamics, and our slew of such applications. Skype is by and large on Cosmos DB now. There are still some fragments of the past.  Xbox and the Microsoft Store and others are running on it.”

Do you think your customers are good at making the right choices over which database technology to use? I do pick up some uncertainty about this.

“We are working on making sure that we provide that clarity. Postgres and MySQL and MariaDB and SQL Server, Azure SQL and elastic pools, managed instances, there is a whole slew of relational offerings. Then we have Cosmos DB and then lots of analytical offerings as well.

“If you are a relational app, and if you are using a relational database, and you are migrating from on-prem to Azure, then we recommend the relational family. It comes with this fundamental scale caveat which is that up to 4TB. Most of those customers are settled because they have designed the app around those sorts of scalability limitations.

“A subset of those customers, and a whole bunch of brand new customers, are willing to re-write the app. They know that that they want to come to cloud for scale. So then we pitch Cosmos DB.

“Then there are customers who want to do massive scale offline analytical processing. So there is, Databricks, Spark, HD Insight, and that set of services.

“We realise there are grey lines between these offerings. We’re tightening up the guidance, it’s valid feedback.”

Any numbers to flesh out the idea that this is a fast-growing service for Microsoft?

“I can tell you that the number of new clusters we provision every week is far more than the total number of clusters we had in the first month. The growth is staggering.”