Tag Archives: azure

Point-in-time restore: a handy built-in feature in Azure SQL

I am working on a project that is hosted in Azure and I made a mistake, running a SQL script that was dependent on another SQL script that I had forgotten to run. It messed up the foreign keys and I would have to restore a backup … but my most recent backup was from the day before. Annoying.

But wait. Looking the Azure portal I saw this:

image

This is a plain Azure SQL instance with no extras, but look, you can restore the database from 6 minutes ago.

I did it; it restored to a second database. I deleted the bad one, renamed the restored one, ran my scripts in the right order, and all was well.

I recommend you do not run scripts in the wrong order … but if you do, or make some other error, this is a handy feature of Azure SQL which I was not aware of before.

Developing software for playing bridge

I am a duplicate bridge player in my spare time and enjoyed playing in my local club once or twice a week. That was before COVID-19 and then, in March this year, lockdown. Bridge clubs were no longer able to meet. There are more important things in the world; but bridge is both a lot of fun and a welcome distraction from weightier matters, and my thoughts soon turned to what we could do to continue playing in these new circumstances.

The answer was to play online; but while there are plenty of ways to play bridge online, the existing systems were not designed with the idea of being a way for bridge clubs to meet in a new context. If anything, the reverse is true: online bridge site were designed for people who could not easily get to a club or wanted to play at any time with whoever else happened to be available. Clubs like my own, by contrast, wanted to replicate their face-to-face meetings with an online equivalent. A further complication back in March was that the biggest online bridge site, called Bridgebase, was immediately overloaded and declared that it was unwilling to allow new people to qualify as directors, people allowed to run online bridge sessions.

My immediate instinct was to build a new site for playing bridge. I was not quite starting from scratch. Back in the early days of Windows 8, I started work on a bridge game for Microsoft’s new and as it turned out ill-fated platform. I had got some way with it; I had created a bridge engine that understood about cards and hands and tricks and shuffling and scoring and all the various elements that go into playing bridge. It was written in C# and what is now UWP XAML. It is designed of course for a solo player. Here is the bidding screen:

image

and the play screen:

image

This is how it looks on Windows 10; it looked a bit better on Windows 8 though it would not win any prizes for design. My software could play bridge though; the reason I never finished it was that I never cracked getting the AI working. But for human to human play that did not matter. A weekend or two coding, I thought, and I could have a website up and running so our club could play bridge online. I made an immediate start, registering the domain name YourBridgeClubOnline.co.uk.

Well, three months later and here we are.

image

image

It is, I have to say, still under development. But it works and we have been able to play bridge again, as a club.

What took you so long? Ha! Much of my old bridge engine code remains untouched and has proved useful; it all runs fine on .NET Core. Even the (useless) AI has been handy, as I can test the mechanics of play without involving others. But I had, of course, wildly underestimated the problem of converting a game for solo play on Windows, to a multi-player web application. There is much to think about:

The UI. I am not a designer (I am sure you can tell) but spent ages puzzling over how to get a workable user interface in the browser for everything from tablets to desktops. Not smartphones yet but it is coming. I decided early on to take a view on compatibility. No Internet Explorer. JavaScript fetch API is required. When time is against you, it is easier to say, just use another browser, than to waste too much time supporting old browsers.

Messaging – both the API kind, and the chat kind. I am using C#, ASP.NET Core and SignalR. In general it works well. SignalR uses WebSockets as first preference, but falls back to Server Sent Events or long polling where necessary. In my first experiments I did my own polling and switching to SignalR was a great relief.

Registration and login. I am using the stuff that comes in the box, ASP.NET Core Identity. It has saved me a ton of work. It’s a bit annoying and not too well documented. I don’t really like using GUIDs for the primary key, for example, and I believe there is way to avoid it, but it isn’t top priority when you are going for Minimum Viable Product.

JavaScript. I’ve written tons of it and I don’t even like the language. I have a new respect for it though. The thing is, it is very fast and there is nothing you cannot do. The worst thing is the friction of doing some debugging in the browser, and some in Visual Studio. I am thinking of switching to VS Code for development since it works nicely with ASP.NET Core and is better for JavaScript than Visual Studio.

Scoring. My Windows software could score a hand of bridge. But duplicate is different; you have to compare the scores with others who played the same hands and work out the percentages, then export the results to standard formats for display on club websites and submission to the English Bridge Union. It was more work than I had expected and I am not done yet; the system only understands Pairs at the moment, not Teams (a different way of scoring).

Directing. Someone has to manage an online bridge session, settle any arguments, and fix errors like cards played by accident. It all needs coding and there was nothing like it in the Windows version.

Movements. Imagine you have 28 people playing bridge (or 14 pairs). They need to all play the same hands, but never play the same hand twice, and it has to be so arranged that each pair plays against other pairs in a defined sequence so it is balanced and fair. We call this the movement. Online, you have a bit more flexibility because you don’t need to share physical cards: everyone can play the same hand at the same time if you like. It is still quite fiddly though, and I did not do any of this in the old Windows version. I saved some time by writing an import function to enable re-use of movements made for EBUScore, a widely used scoring and bridge session management application. There is more to do though.

Claims. This is where, half way through the hand, a player says, “There’s no point in playing on, I’m obviously going to win all the remaining tricks.” A trick is a sequence of four cards played one from each hand, which is won by one of the pairs. This statement is called a claim, and has to be agreed by the other players. Getting this working was more difficult than I had expected – because built into my bridge engine was the idea that you could score by counting the tricks each side had won. But claimed tricks are never played. With hindsight, I should have allowed for this from the beginning.

Database. Every detail of play has to be stored on the server. I am using Dapper and SQL Server currently, though it is possible that PostgreSQL would work just as well. I started with Entity Framework Core, still there as it is used by ASP.NET Core Identity, but I am happier with Dapper.

Things that worked well

Three months is longer than I had thought it would take to get to a playable system, but I suppose as a spare time project it is not too bad. It would not be possible without the likes of ASP.NET Core and Dapper and SignalR doing so much for you. C# is a delight for coding. I am also using an Azure App Service for all this test and development and that has worked well. I am deploying to a Linux container of course; but the nice thing about App Service is that it will scale to a considerable extent without the hassle of Kubernetes. If the project succeeds and needs to scale up, there is an Azure SignalR service ready and waiting. I was nevertheless interested to see that AWS now offers .NET Core on Elastic Beanstalk, complete with some nice Visual Studio integration. Trying it there would be an interesting experiment, though I’m not sure AWS is so savvy about SignalR.

Open Source?

Could this have been done quicker by making it open source and seeking collaborators early on? Will it become open source? I need help for sure, though I also feel the code needs some cleaning up before it is fit to share more widely. You will recall though that I had started out thinking that it would be a small matter to convert my solo bridge game to an online multiplayer web application. I figured it would be better to get something working and then ask for help. But I am open to offers! Note: this is not a commercial project.

Rewarding

Most of the software projects I have been involved in have been business applications. Bridge is a lot more fun. I do see software development as a creative act. I recall starting work on the bridge game back in 2011 (I think); starting a new blank project in Visual Studio and thinking, hmm, I had better write a class to represent a pack of cards. From that beginning I ended up with an application that could play bridge, after a fashion, and now one that multiple people can play concurrently. It is rewarding and I will not regret the time spent on it, irrespective of how much actual use it gets.

Annoying Azure capacity problems in UK West region

I have a test setup of Windows Virtual Desktop (WVD) and was experimenting with adding an additional VM. At least, I tried to. My WVD virtual network is in the UK West region. And when I try to create a VM I get the message: Your subscription doesn’t support virtual machine creation in UK West. Choose a different location.

image

This was annoying because my WVD virtual network is in UK West, so no, another region would not do. If you click Learn More you get this page which says that if you get this message and still want to deploy a VM in the region, you have to raise a support case.

I am guessing but I presume this is a capacity problem and that Microsoft is discouraging VM creation in the region. The problem for the customer is that such things are opaque; there is nowhere you can see which Azure regions are running close to capacity.

Microsoft earnings: strong quarter, but Xbox revenue dives

Microsoft has announced its quarterly financial statements, reporting revenue of $33.7 billion, up 12% on the same period last year.

The company stated that Azure revenue is up 64% year on year. Azure has overtaken the other two segments and is now the biggest, by a small amount. In addition, Azure gross margin has improved by 6% year on year.

Office 365 revenue is up 31% year on year.

Gaming was a black spot, declining 10% year on year – though Xbox Live monthly active users is at a record 65 million. The main problem is a 48% decline in the volume of Xbox consoles sold.

Quarter ending June 30th 2019 vs quarter ending June 30th 2018, $millions

Segment Revenue Change Operating income Change
Productivity and Business Processes 11047 +1379 4344 +878
Intelligent Cloud 11391 +1785 4502 +601
More Personal Computing 11279 +468 3559 +547

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

A post that can save you money: scheduling Azure Virtual Machines for start/stop

I have written recently about Windows Virtual Desktop, the ability to set up a virtual desktop environment on Azure at a relatively low cost, provided your users have Microsoft 365 accounts. My test setup is minimal but I have been watching the cost which is currently working out at £5.39 per day. This excludes the cost of Microsoft 365; it is purely for the infrastructure including VPN gateway, storage and VM. Bandwidth is a variable cost but almost negligible on my usage. Of that cost, the VM is around 75%. So if I could shut down the VM when not in use the savings are substantial.

It turns out this is pretty easy on Azure though it requires some plumbing. VMs do have a built-in option to shutdown on a schedule, but not to start up. To get start/stop, you need an Automation Account.

image

With the automation account created, select it, hit Start/Stop VM, then click “Learn more about and enable the solution”.  You get this dialog.

image

Here we learn that to save money, we have to spend it, on three new services: Automation, Log Analytics, and Monitor. It is not too bad though as there is a free tier for these services that may be all I need. Hit Create.

image

In this window you have to configure three sections. Nothing challenging, but note that in Configuration you set the Target Resource Group Names. No pick list here, you have to type in the names. Or use a wildcard, which is unlikely to be a good idea since by default it will start and stop ALL your VMs. The schedule is not very smart, just a daily on and off, but see below. Once done, click Create to add the solution.

All done, but what about weekends, for example. This is easily fixed if you create your own schedules. Just go into your automation account and click Schedules under Shared Resources. The wizard-created schedules are listed, and you can modify them or create new ones. It looks as if you might need 5 schedules, one per weekday, recur every week, to make your VMs not run at weekends. There is no Monday-Friday option.

More documentation here. Note that automation can also run PowerShell scripts which will be even more flexible.

Scheduling cloud resources to shut down when not in use must be one of the most effective ways to reduce IT spend.

Update: here is the outcome of my efforts:

image

The Management resource group has the runbook that performs the start/stop action. The cost of this is small. Overall cost has gone down by about £2.00 or about 40% in my case. I appreciate this is a very small test deployment, but it would support maybe 4 or 5 users without any problem and my experience shows that you can indeed make a large saving by scheduling VMs to stop when not in use.

Microsoft is making lots of money. Anything else notable in its first quarter financials?

Microsoft has released its statements for the first quarter in its financial year, ending 30th September. Here is the segment breakdown. Everything has moved in the right direction.

Quarter ending September 30th 2018 vs quarter ending September 30th 2017, $millions

Segment Revenue Change Operating income Change
Productivity and Business Processes 9771 +1533 3881 +875
Intelligent Cloud 8567 +1645 2931 +794
More Personal Computing 10746 +1368 3143 +578

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

Any points of interest? In his earnings call statement, CEO Satya Nadella talked Teams, the Office 365 conferencing and collaboration solution:

“Teams is now the hub for teamwork for 329,000 organizations, including 87 of the Fortune 100. And, we are adding automated translation
support for meetings, shift scheduling for firstline workers, and new industry-specific offerings including healthcare and small business.”

He also mentioned Power Apps and Flow, interesting to me because they are the most successful so far of the company’s efforts to come up with a low-code development platform:

“Power BI, Power Apps and Flow are driving momentum with customers and have made us a leader in no-code app building and business analytics in the cloud.”

He also mentioned the pending GitHub acquisition, which he says is “an opportunity to bring our tools and services to new audiences while enabling GitHub to grow and retain its developer-first ethos.”

Note that despite the cloud growth, Windows remains the biggest single segment in terms of revenue.

Determining how much of Microsoft’s business is “cloud” is tricky. The figures in the productivity segment lump together Office 365 and on-premises products, while Office 365 itself is in part a subscription to desktop Office, so not pure cloud. Equally, the “intelligent cloud” segment includes on-premises server licenses. No doubt this fuzzing of what is and is not cloud in the figures is deliberate.

Microsoft’s 82 Ignite announcements: what really matters

Microsoft’s PR team has helpfully summarised many of the announcements at the Ignite event, kicking off today in Orlando. I count 82, but you might make it fewer or many more, depending on what you call an announcement. And that is not including the business apps announcements made at the end of last week, most notably the arrival of the HoloLens-based Remote Assist in Dynamics 365.

image

Not all announcements are equal. Some, like the release of Windows Server 2019, are significant but not really news; we knew it was coming around now, and the preview has been around for ages. Others, like larger Azure managed disk sizes (8, 16 and 32TB) are cool if that is what you need, but hardly surprising; the specification of available cloud infrastructure is continually being enhanced.

Note that this post is based on what Microsoft chose to reveal to press ahead of the event, and there is more to come.

It is worth observing though that of these 82 announcements, only 3 or 4 are not cloud related:

  • SQL Server 2019 public preview
  • [Windows Server 2019 release] – I am bracketing this because many of the new features in Server 2019 are Azure-related, and it is listed under the heading Azure Infrastructure
  • Chemical Simulation Library for Microsoft Quantum
  • Surface Hub 2 release promised later this year

Microsoft’s journey from being an on-premises company, to being a service provider, is not yet complete, but it is absolutely the focus of almost everything new.

I will never forget an attendee at a previous Microsoft event a few years back telling me, “this cloud stuff is not relevant to us. We have our own datacenter.” I cannot help wondering how much Office 365 and/or Azure that person’s company is consuming now. Of course on-premises servers and applications remain important to Microsoft’s business, but it is hard to swim against the tide.

Ploughing through 82 announcements would be dull for me to write and you to read, so here are some things that caught my eye, aside from those already mentioned.

1. Azure confidential computing in public preview. A new series of VMs using Intel’s SGX technology lets you process data in a hardware-enforced trusted execution environment.

2. Cortana Skills Kit for Enterprise. Currently invite-only, this is intended to make it easier to write business bots “to improve workforce productivity” – or perhaps, an effort to reduce the burden on support staff. I recall examples of using conversational bots for common employee queries like “how much holiday allowance do I have remining, and which days can I take off?”. As to what is really new here, I have yet to discover.

3. A Python SDK for Azure Machine Learning. Important given the popularity of Python in this space.

4. Unified search in Microsoft 365. Is anyone using Delve? Maybe not, which is why Microsoft is bringing a search box to every cloud application, which is meant to use Microsoft Graph, AI and Bing to search across all company data and bring you personalized results. Great if it works.

5. Azure Digital Twins. With public preview promised on October 15, this lets you build “comprehensive digital models of any physical environment”. Once you have the model, there are all sorts of possibilities for optimization and safe experimentation.

6. Azure IoT Hub to support the Android Things platform via the Java SDK. Another example of Microsoft saying, use what you want, we can support it.

7. Azure Data Box Edge appliance. The assumption behind Edge computing is both simple and compelling: it pays to process data locally so you can send only summary or interesting data to the cloud. This appliance is intended to simplify both local processing and data transfer to Azure.

8. Azure Functions 2.0 hits general availability. Supports .NET Core, Python.

9. Helm repositories in Azure Container Registry, now in public preview.

10 Windows Autopilot support extended to existing devices. This auto-configuration feature previously only worked with new devices. Requires Windows 10 October Update, or automated upgrade to this.

Office and Office 365

In the Office 365 space there are some announcements:

1. LinkedIn integration with Office 365. Co-author documents and send emails to LinkedIn contacts, and surface LinkedIn information in meeting invites.

2. Office Ideas. Suggestions as you work to improve the design of your document, or suggest trends and charts in Excel. Sounds good but I am sceptical.

3. OneDrive for Mac gets Files on Demand. A smarter way to use cloud storage, downloading only files that you need but showing all available documents in Mac Finder.

4. New staff scheduling tools in Teams. Coming in October. ”With new schedule management tools, managers can now create and share schedules,employees can easily swap shifts, request time off, and see who else is working.” Maybe not a big deal in itself, but Teams is huge as I previously noted. Apparently the largest Team is over 100,000 strong now and there are 50+ out there with 10,000 or more members.

Windows Virtual Desktop

This could be nothing, or it could be huge. I am working on the basis of a one-paragraph statement that promises “virtualized Windows and Office on Azure … the only cloud-based service that delivers a multi-user Windows 10 experience, is optimized for Office 365 Pro Plus … with Windows Virtual Desktop, customers can deploy and scale Windows and Office on Azure in minutes, with built-in security and compliance.”

Preview by the end of 2018 is targeted.

Virtual Windows desktops are already available on Azure, via partnership with Citrix or VMWare Horizon, but Microsoft has held back from what is technically feasible in order to protect its Windows and Office licensing income. By the time you have paid for licenses for Windows Server, Remote Access per user, Office per user, and whatever third-party technology you are using, it gets expensive.

This is mainly about licensing rather than technology, since supporting multiple users running Office applications is now a light load for a modern server.

If Microsoft truly gets behind a pure first-party solution for hosted desktops on Azure at a reasonable cost, the take up would be considerable since it is a handy solution for many scenarios. This would not please its partners though, nor the many hosting companies which offer this.

On the other hand, Microsoft may want to compete more vigorously with Amazon Web Services and its Workspaces offering. Workspaces is still Windows, but of course integrates nicely with AWS solutions for storage, directory, email and so on, so there is a strategic aspect here.

Update: A little more on Microsoft Virtual Desktop here.

More details soon.

Microsoft’s strong financials, and some notes on Azure vs AWS and the risks of losing in mobile

Microsoft delivered another strong set of figures in its latest financial results, for the period April-June 2018. Total revenue of $30.085 million was up 17% year on year, and all three of the company’s sectors (Office, Azure and consumer) showed strong growth.

What’s notable? Largely this is more of the same. A few things to note. Linked in revenue increased 37% year on year – an acquisition that seems to be making sense for the company. Dynamics 365 revenue grew by 65%. The Dynamics story is all about cloud synergy. As an on-premises product Dynamics CRM (the part of the suite I know best) was relatively undistinguished but as a cloud product the seamless integration between Office 365 and Dynamics 365 (and Azure Active Directory) makes it compelling.

Windows 10 is doing OK, possibly as more businesses heave themselves off Windows 7 and buy new PCs with OEM licenses as they do.

Even areas in which Microsoft is far from dominant did well. Gaming was up 39%, Surface 25% and Search advertising up 17%.

The biggest growth in the quarter, according to the breakdown here, was in Azure. up 89%. This growth is not without pain; the Register reports capacity issues in the UK South region, for example, with users getting the message “Unfortunately, due to high demand for virtual machines in this region, we are not able to approve your quota request at this time.” You can still create VMs, but not necessarily in the region you want.

Will Microsoft outpace AWS? My take on this has not changed. AWS does very little wrong and remains the pre-eminent cloud for IAAS and many services by some distance. What AWS does not have is Office 365, or armies of Microsoft partners helping enterprise customers to shunt more and more of their IT infrastructure into Azure. Microsoft makes more money from licensing: Windows Server, SQL Server, Office 365 and Dynamics seats, and so on. AWS does more business at a lower margin. These are big differences. I see it as unlikely that Azure will overtake AWS in the provision of essential cloud services like VMs, containers, cloud storage and so on. AWS also has a better reliability track record. However, the success of Azure means that enterprise customers no longer need to go to AWS to get the benefits of cloud. Perhaps the more interesting question is the extent to which AWS (or Google) can persuade enterprise customers to shift away from Microsoft’s high-margin applications.

Longer term, there is significant risk for the company in its retreat from mobile. We are now seeing Google work hard in the laptop market with Chromebooks alongside Android mobile. Coming sometime is Google Fuchsia which may be a single operating system for both. It is worth recalling that Microsoft built its success on winning users for its PC operating system; and that IBM lost its IT dominance by ceding this to Microsoft.

Here is the breakdown by segment, such as it is:  

Quarter ending June 30th 2018 vs quarter ending June 30th 2017, $millions

Segment Revenue Change Operating income Change
Productivity and Business Processes 9668 +1140 3466 +575
Intelligent Cloud 9606 +1784 3901 +990
More Personal Computing 10811 +1576 3012 +826

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

Inside Azure Cosmos DB: Microsoft’s preferred database manager for its own high-scale applications

At Microsoft’s Build event in May this year I interviewed Dharma Shukla, Technical Fellow for the Azure Data group, about Cosmos DB. I enjoyed the interview but have not made use of the material until now, so even though Build was some time back I wanted to share some of his remarks.

Cosmos DB is Microsoft’s cloud-hosted NoSQL database. It began life as DocumentDB, and was re-launched as Cosmos DB at Build 2017. There are several things I did not appreciate at the time. One was how much use Microsoft itself makes of Cosmos DB, including for Azure Active Directory, the identity provider behind Office 365. Another was how low Cosmos DB sits in the overall Azure cloud system. It is a foundational piece, as Shukla explains below.

image

There were several Cosmos DB announcements at Build. What’s new?

“Multi-master is one of the capabilities that we announced yesterday. It allows developers to scale writes all around the world. Until yesterday Cosmos DB allowed you to scale writes in a single region but reads all around the world. Now we allow developers to scale reads and writes homogeneously all round the world. This is a huge deal for apps like IoT, connected cars, sensors, wearables. The amount of writes are far more than the amount of reads.

“The second thing is that now you get single-digit millisecond write latencies at the 99 percentile not just in one region.

“And the third piece is that what falls out of this high availability. The window of failover, the time it takes to failover from one region when a disaster happens, to the other, has shrunk significantly.

“It’s the only system I know of that has married the high consistency models that we have exposed with multi-master capability as well. It had to reach a certain level of maturity, testing it with first-party Microsoft applications at scale and then with a select set of external customers. That’s why it took us a long time.

“We also announced the ability to have your Cosmos Db database in your own VNet (virtual network). It’s a huge deal for enterprises where they want to make sure that no data leaks out of that VNet. To do it for a global distributed database is specially hard because you have to close all the transitive networking dependencies.”

image
Technical Fellow Dharma Shukla

Does Cosmos DB work on Azure Stack?

“We are in the process of going to Azure Stack. Azure Stack is one of the top customer asks. A lot of customers want a hybrid Cosmos DB on Azure Stack as well as in Azure and then have Active – Active. One of the design considerations for multi master is for edge devices. Right now Azure has about 50 regions. Azure’s going to expand to let’s say 200 regions. So a customer’s single Cosmos DB table spanning all these regions is one level of scalability. But the architecture is such that if you directly attach lots of Azure Stack devices, or you have sensors and edge devices, they can also pretend to be replicas. They can also pretend to be an Azure region. So you can attach billions of endpoints to your table. Some of those endpoints could be Azure regions, some of them could be instances of Azure Stack, or IoT hub, or edge devices. This kind of scalability is core to the system.”

Have customers asked for any additional APIs into Cosmos DB?

“There is a list of APIs, HBase, richer SQL, there are a number of such API requests. The good news is that the system has been built in a way that adding new APIs is relatively easy addition. So depending on the demand we continue to add APIs.”

Can you tell me anything about how you’ve implemented Cosmos DB? I know you use Service Fabric. Do you use other Azure services?

“We have dedicated clusters of compute machines. Cosmos DB is a Ring 0 service. So it’s there any time Azure opens a new region, Cosmos DB clusters have provision by default. Just like compute, storage, Cosmos DB is also one of the Ring 0 services which is the bottommost. Azure Active Directory for example depends on Cosmos DB. So Cosmos DB cannot take a dependency on Active Directory.

“The dependency that we have is our own clusters and machines, on which we put Service Fabric. For deployment of Cosmos DB code itself, we use Service Fabric. For some of the load balancing aspects we use Service Fabric. The partition management, global distribution, replication, is our own. So Cosmos DB is layered on top of Service Fabric, it is a Service Fabric application. But then it takes over. Once the Cosmos DB bits are laid out on the machine then its replication and partition management and distribution pieces take over. So that is the layering.

“Other than that there is no dependency on Azure. And that is why one of the salient aspects of this is that you can take the system and host it easily in places like Azure Stack. The dependencies are very small.

“We don’t use Azure Storage because of that dependency. So we store the data locally and then replicate it. And all of that data is also encrypted at rest.”

So when you say it is not currently in Azure Stack, it’s there underneath, but you haven’t surfaced it?

“It is in a defunct mode. We have to do a lot of work to light it up. When we light up it on such on-prem or private cloud devices, we want to enable this active to active pathway. So you are replicating your data and that is getting synchronized with the cloud and Azure Stack is one of the sockets.”

Microsoft itself is using Cosmos DB. How far back does this go? Azure AD is quite old now. Was it always on Cosmos DB / DocumentDB?

“Over the years Office 365, Xbox, Skype, Bing, and more and more of Azure services, have started moving. Now it has almost become ubiquitous. Because it’s at the bottom of the stack, taking a dependency on it is very easy.

“Azure Active Directory consists of a set of microservices. So they progressively have moved to Cosmos DB. Same situation with Dynamics, and our slew of such applications. Skype is by and large on Cosmos DB now. There are still some fragments of the past.  Xbox and the Microsoft Store and others are running on it.”

Do you think your customers are good at making the right choices over which database technology to use? I do pick up some uncertainty about this.

“We are working on making sure that we provide that clarity. Postgres and MySQL and MariaDB and SQL Server, Azure SQL and elastic pools, managed instances, there is a whole slew of relational offerings. Then we have Cosmos DB and then lots of analytical offerings as well.

“If you are a relational app, and if you are using a relational database, and you are migrating from on-prem to Azure, then we recommend the relational family. It comes with this fundamental scale caveat which is that up to 4TB. Most of those customers are settled because they have designed the app around those sorts of scalability limitations.

“A subset of those customers, and a whole bunch of brand new customers, are willing to re-write the app. They know that that they want to come to cloud for scale. So then we pitch Cosmos DB.

“Then there are customers who want to do massive scale offline analytical processing. So there is, Databricks, Spark, HD Insight, and that set of services.

“We realise there are grey lines between these offerings. We’re tightening up the guidance, it’s valid feedback.”

Any numbers to flesh out the idea that this is a fast-growing service for Microsoft?

“I can tell you that the number of new clusters we provision every week is far more than the total number of clusters we had in the first month. The growth is staggering.”

Cosmos DB or SQL Server? Do you need Kubernetes? VM or App Service? A guide to Azure worth checking out

One of the best features of Microsoft Build, possibly the best, is the exhibition. Microsoft sets up stands for each of its product teams, and the staff there generally include the people who actually build that product, making this a great way to interact with them and get authoritative answers to questions.

I interviewed several executives at Build and asked a couple of times, how can your customers work out which Azure service is the best fit for what they need? It is not a trivial question, now that there are so many different services which overlapping functionality.

It is critically important. You can waste a large amount of money and cause unnecessary frustration by selecting the wrong services.

None of these executives mentioned that Microsoft has a rather good guide for exactly this question. It is called the Azure Architecture Center and I discovered it on the show floor.

image

The stand was called Azure Clinic and I told the guy his costume reminded me of Dr GUI. He was too young to remember this MSDN character of old but another guy on the stand overheard and said it brought back bad memories!

You can find the Azure Architecture Center here. It does not make any assumptions about the depth of knowledge you have, which seems right to me since it is aimed at developers who are not sure exactly what they need. There is a ton of useful material, like this decision tree for the compute services (click to enlarge):

image 

Recommended.