Category Archives: cloud computing

Microsoft is making lots of money. Anything else notable in its first quarter financials?

Microsoft has released its statements for the first quarter in its financial year, ending 30th September. Here is the segment breakdown. Everything has moved in the right direction.

Quarter ending September 30th 2018 vs quarter ending September 30th 2017, $millions

Segment Revenue Change Operating income Change
Productivity and Business Processes 9771 +1533 3881 +875
Intelligent Cloud 8567 +1645 2931 +794
More Personal Computing 10746 +1368 3143 +578

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

Any points of interest? In his earnings call statement, CEO Satya Nadella talked Teams, the Office 365 conferencing and collaboration solution:

“Teams is now the hub for teamwork for 329,000 organizations, including 87 of the Fortune 100. And, we are adding automated translation
support for meetings, shift scheduling for firstline workers, and new industry-specific offerings including healthcare and small business.”

He also mentioned Power Apps and Flow, interesting to me because they are the most successful so far of the company’s efforts to come up with a low-code development platform:

“Power BI, Power Apps and Flow are driving momentum with customers and have made us a leader in no-code app building and business analytics in the cloud.”

He also mentioned the pending GitHub acquisition, which he says is “an opportunity to bring our tools and services to new audiences while enabling GitHub to grow and retain its developer-first ethos.”

Note that despite the cloud growth, Windows remains the biggest single segment in terms of revenue.

Determining how much of Microsoft’s business is “cloud” is tricky. The figures in the productivity segment lump together Office 365 and on-premises products, while Office 365 itself is in part a subscription to desktop Office, so not pure cloud. Equally, the “intelligent cloud” segment includes on-premises server licenses. No doubt this fuzzing of what is and is not cloud in the figures is deliberate.

Microsoft Azure Stack: a matter of compliance

At the Ignite conference last week in Orlando, Microsoft’s hardware partners were showing off their latest Azure Stack boxes.

In conversation, one mentioned to me that Azure Stack was selling better in Europe than in the USA. Why? Because stricter compliance regulations (perhaps alongside the fact that the major cloud platforms are all based in North America) makes Azure Stack more attractive in Europe.

image
Lenovo’s Azure Stack

Azure Stack is not just “Azure for your datacentre”. It is a distinctive way to purchase IT infrastructure, where you buy the hardware but pay for the software with a usage-based model.

Azure / Azure Stack VMs are resilient so you cannot compare the value directly with simply running up a VM on your own server. Azure Stack is a premium option. The benefits are real. Microsoft mostly looks after the software, you can use the excellent Azure management tools, and you get deep integration with Azure in the cloud. Further, you can diminish the cost by scaling back at times of low demand; especially easy if you use abstracted services such as App Service, rather than raw VMs.

How big is the premium? I would be interested to hear from anyone who has done a detailed comparison, but my guess is that running your own servers with Windows Server Datacenter licenses (allowing unlimited VMs once all the cores are licensed) is substantially less expensive.

You can see therefore that there is a good fit for organizations that want to be all-in on the cloud, but need to run some servers on-premises for compliance reasons.

Google announces Cloud Build: CI/CD for the Google Cloud Platform

Google Cloud Next is under way in San Francisco, and yesterday saw the announcement of Cloud Build, Continuous Integration and Continuous Deployment for the Google Cloud Platform.

image

Cloud Build runs a series of automated build steps and then optionally pushes built images to Googles container registry. It is a natural fit with Kubernetes but can be used with both containerised and direct deployments.

You can create your own build steps or use a prebuilt one. The prebuilt steps are:

  • bazel: runs the bazel tool
  • curl: runs the curl tool
  • docker: runs the docker tool
  • dotnet: run the dotnet tool
  • gcloud: runs the gcloud tool
  • git: runs the git tool
  • go: runs the go tool
  • gradle: runs the gradle tool
  • gsutil: runs the gsutil tool
  • kubectl: runs the kubectl tool
  • mvn: runs the maven tool
  • npm: runs the npm tool
  • wget: runs the wget tool
  • yarn: runs the yarn tool

Note that dotnet is in there so you can use this immediately with .NET Core.

There is also an option to  build locally. For example, you could build locally and only after a successful local build, invoke Cloud Build.

Cloud Build integrates with GitHub:

With this new integration, you can easily set up CI through Cloud Build and automate builds and tests as part of your GitHub workflow.

I doubt Google celebrated when Microsoft acquired GitHub but it is good to see GitHub continuing to support diverse platforms.

Overall this is an important feature as Google races to extend its cloud platform to match what is on offer from its key competitors, AWS and Microsoft Azure.

AWS embraces hybrid cloud? Meet Snowball Edge

Amazon has announced Snowball Edge, an on-premises appliance that supports Amazon EC2 (Elastic Compute Cloud), AWS Lambda (“serverless” computing) and S3 (Simple Storage Service), all running locally.

image

Sounds like Microsoft’s Azure Stack? A bit, but the AWS appliance is tiny by comparison and therefore more limited in scope. Nevertheless, it is a big turnaround for the company, which has previously insisted that everything belongs in the cloud. One of the Snowball Edge case studies is the same general area as one used by Microsoft for Azure Stack: ships.

The specifications are shy about revealing what is inside, but there is 100TB storage (82TB usable), 10GB, 20GB and 40GB network connections (GBase-T, SFP+ and QSFP+), size is 259x671x386mm (pretty small), and power consumption 400 watts.

Jeff Barr’s official blog post adds that there is an “Intel Xeon D processor running at 1.8 GHz, and supports any combination of instances that consume up to 24 vCPUs and 32 GiB of memory.”

You can cluster Snowball Edge appliances though so substantial systems are possible.

Operating systems currently supported are Ubuntu Server and CentOS7.

Amazon’s approach is to extend its cloud to the edge rather than vice versa. You prepare your AMIs (Amazon Machine Instances) in the cloud before the appliance is shipped. The very fast networking support shows that the intent is to maintain the best possible connectivity, even though the nature of the requirement is that internet connectivity in some scenarios will be poor.

A point to note is that whereas the documentation emphasises use cases where there are technical advantages to on-premises (or edge) computing, Barr quotes instead a customer who wanted easier management. A side effect of the cloud computing revolution is that provisioning and managing cloud infrastructure is easier than with systems (like Microsoft’s System Center) designed for on-premises infrastructure. Otherwise they would not be viable. Having tasted what is possible in the cloud, customers want the same for on-premises.

Microsoft announces free version of Teams, ahead of Inspire partner conference

Microsoft’s partner conference, Inspire, kicks off in Las Vegas next week; and as part of the event the company has announced big news concerning Teams: a free version.

image

What is Teams? It is a collaboration tool for Office 365, or at least it was, since the new free version can be used with any email address and without Office 365. Here is what you get:

  • Chat
  • Audio and video calling
  • 10GB online storage, plus 2GB for each additional team member (SharePoint/OneDrive)
  • Word, Excel and PowerPoint online
  • Ability to install unlimited additional applications

Teams is a strategic product for Microsoft – see here for the reason. A free version is way for the company to promote Office 365, and you will see an upgrade link in the user interface.

There are also new features coming to Teams. One seems minor, but will be popular. It deals with the problem of video conferencing from home, and not being sure what may happen behind you. You may remember this:

image

So now Teams video conferencing will let you blur the background. Here is Raanah Amjadi, Marketing Manager, Microsoft Teams, demonstrating the feature:

image

In addition, Teams is getting a new Live Events feature. This is where you broadcast a presentation or meeting to others in your company. Automatic speech-to-text will do close captions (so you can watch with the sound done, if you trust it enough), and this then enables text search of the event with index points into the video. Bing Translate is also included in Teams so you can have multi-lingual conversations.

image

Microsoft Workplace Analytics is getting enhancements including “My Analytics” which will give you AI-powered “nudges” in Outlook online. I am not sure I trust this to be much real-world use; but the example shown was intriguing: alert you if you try to schedule a meeting with someone out of their working hours.

Whiteboard, a collaboration canvas, is now generally available for Windows 10 and mobile.

image

Free Teams is available immediately here.

Ubuntu goes minimal (but still much bigger than Alpine Linux), cosies up to Google Cloud Platform

Ubuntu has announced “Minimal Ubuntu”, a cut-down server image designed for containerised deployments. The Docker image for Minimal Ubuntu 18.04 is 29MB:

Editors, documentation, locales and other user-oriented features of Ubuntu Server have been removed. What remains are only the vital components of the boot sequence.  Images still contain ssh, apt and snapd so you can connect and install any package you’re missing. The unminimize tool lets you ‘rehydrate’ your image into a familiar Ubuntu server package set, suitable for command line interaction.

says Canonical.

29MB is pretty small; but not as small as Alpine Linux images, commonly used by Docker, which are nearer 5MB. Of course these image sizes soon increase when you add the applications you need.

I pulled Ubuntu 18.04 from Docker Hub and the image size is 31.26MB so this hardly seems a breakthrough.

Canonical quotes Paul Nash, Group Product Manager for Google Cloud Platform, in its press release. The image is being made available initially for Amazon EC2, Google Compute Engine, LXD, and KVM/OpenStack. The kernel has been optimized for each deployment, so the downloadable image is optimized for KVM and slightly different than the AWS or GCP versions.

Pusher: a nice solution for sending messages and notifications to web and mobile apps

Pusher is a London company which runs cloud services for publish/subscribe in web and mobile applications. The idea is to deliver real-time updates, a concept that has many use cases. Examples include price updates in finance apps, status updates to track a delivery, news updates, or anything where users want to monitor progress or keep in touch with fast-moving developments.

The service passed my “get up and running quickly” test. I created a free account (limited to 100 connections and 200k messages per day) and a new channel:

image 

I’m guessing it runs on AWS, looking at the datacentre locations:

image

I chose a JavaScript client and ASP.NET MVC for the back end. On my PC I pasted the JavaScript into a web page running locally on Apache (in Windows Subsystem for Linux). I also created a new ASP.NET MVC project and added the sample code with some trivial modifications. I was able to send a message to the web page; it triggers an annoying alert but of course you could easily amend this to update the UI in more user-friendly ways.

image

Of course you could roll your own solution for this but what you get with Pusher is all the plumbing pre-done for many different clients and automatic scalability.

Pusher also has a service called Beams (formerly Push Notifications) which lets you send notifications to Android and IOS apps.

Pusher or roll your own? As with many cloud services, you are putting a high level of trust in Pusher (security and reliability) if you use the service, and you will need a paid subscription:

image

You are saving considerable development time though, and as Google and Apple update their SDKs or change the rules, Pusher will presumably adapt accordingly.

Can Azure easily do this, I wondered? I headed over to Azure Notification Hubs. I noticed that the amount of admin you have to do to support each device is greater. Second, Microsoft promised to support “push to web” in March 2016:

image

… but has not done so nor even bothered to update those asking:

image

It is odd that Microsoft, with all its drive behind Azure, is still in the habit of leaving customers in the dark in certain areas.

Inside Azure Cosmos DB: Microsoft’s preferred database manager for its own high-scale applications

At Microsoft’s Build event in May this year I interviewed Dharma Shukla, Technical Fellow for the Azure Data group, about Cosmos DB. I enjoyed the interview but have not made use of the material until now, so even though Build was some time back I wanted to share some of his remarks.

Cosmos DB is Microsoft’s cloud-hosted NoSQL database. It began life as DocumentDB, and was re-launched as Cosmos DB at Build 2017. There are several things I did not appreciate at the time. One was how much use Microsoft itself makes of Cosmos DB, including for Azure Active Directory, the identity provider behind Office 365. Another was how low Cosmos DB sits in the overall Azure cloud system. It is a foundational piece, as Shukla explains below.

image

There were several Cosmos DB announcements at Build. What’s new?

“Multi-master is one of the capabilities that we announced yesterday. It allows developers to scale writes all around the world. Until yesterday Cosmos DB allowed you to scale writes in a single region but reads all around the world. Now we allow developers to scale reads and writes homogeneously all round the world. This is a huge deal for apps like IoT, connected cars, sensors, wearables. The amount of writes are far more than the amount of reads.

“The second thing is that now you get single-digit millisecond write latencies at the 99 percentile not just in one region.

“And the third piece is that what falls out of this high availability. The window of failover, the time it takes to failover from one region when a disaster happens, to the other, has shrunk significantly.

“It’s the only system I know of that has married the high consistency models that we have exposed with multi-master capability as well. It had to reach a certain level of maturity, testing it with first-party Microsoft applications at scale and then with a select set of external customers. That’s why it took us a long time.

“We also announced the ability to have your Cosmos Db database in your own VNet (virtual network). It’s a huge deal for enterprises where they want to make sure that no data leaks out of that VNet. To do it for a global distributed database is specially hard because you have to close all the transitive networking dependencies.”

image
Technical Fellow Dharma Shukla

Does Cosmos DB work on Azure Stack?

“We are in the process of going to Azure Stack. Azure Stack is one of the top customer asks. A lot of customers want a hybrid Cosmos DB on Azure Stack as well as in Azure and then have Active – Active. One of the design considerations for multi master is for edge devices. Right now Azure has about 50 regions. Azure’s going to expand to let’s say 200 regions. So a customer’s single Cosmos DB table spanning all these regions is one level of scalability. But the architecture is such that if you directly attach lots of Azure Stack devices, or you have sensors and edge devices, they can also pretend to be replicas. They can also pretend to be an Azure region. So you can attach billions of endpoints to your table. Some of those endpoints could be Azure regions, some of them could be instances of Azure Stack, or IoT hub, or edge devices. This kind of scalability is core to the system.”

Have customers asked for any additional APIs into Cosmos DB?

“There is a list of APIs, HBase, richer SQL, there are a number of such API requests. The good news is that the system has been built in a way that adding new APIs is relatively easy addition. So depending on the demand we continue to add APIs.”

Can you tell me anything about how you’ve implemented Cosmos DB? I know you use Service Fabric. Do you use other Azure services?

“We have dedicated clusters of compute machines. Cosmos DB is a Ring 0 service. So it’s there any time Azure opens a new region, Cosmos DB clusters have provision by default. Just like compute, storage, Cosmos DB is also one of the Ring 0 services which is the bottommost. Azure Active Directory for example depends on Cosmos DB. So Cosmos DB cannot take a dependency on Active Directory.

“The dependency that we have is our own clusters and machines, on which we put Service Fabric. For deployment of Cosmos DB code itself, we use Service Fabric. For some of the load balancing aspects we use Service Fabric. The partition management, global distribution, replication, is our own. So Cosmos DB is layered on top of Service Fabric, it is a Service Fabric application. But then it takes over. Once the Cosmos DB bits are laid out on the machine then its replication and partition management and distribution pieces take over. So that is the layering.

“Other than that there is no dependency on Azure. And that is why one of the salient aspects of this is that you can take the system and host it easily in places like Azure Stack. The dependencies are very small.

“We don’t use Azure Storage because of that dependency. So we store the data locally and then replicate it. And all of that data is also encrypted at rest.”

So when you say it is not currently in Azure Stack, it’s there underneath, but you haven’t surfaced it?

“It is in a defunct mode. We have to do a lot of work to light it up. When we light up it on such on-prem or private cloud devices, we want to enable this active to active pathway. So you are replicating your data and that is getting synchronized with the cloud and Azure Stack is one of the sockets.”

Microsoft itself is using Cosmos DB. How far back does this go? Azure AD is quite old now. Was it always on Cosmos DB / DocumentDB?

“Over the years Office 365, Xbox, Skype, Bing, and more and more of Azure services, have started moving. Now it has almost become ubiquitous. Because it’s at the bottom of the stack, taking a dependency on it is very easy.

“Azure Active Directory consists of a set of microservices. So they progressively have moved to Cosmos DB. Same situation with Dynamics, and our slew of such applications. Skype is by and large on Cosmos DB now. There are still some fragments of the past.  Xbox and the Microsoft Store and others are running on it.”

Do you think your customers are good at making the right choices over which database technology to use? I do pick up some uncertainty about this.

“We are working on making sure that we provide that clarity. Postgres and MySQL and MariaDB and SQL Server, Azure SQL and elastic pools, managed instances, there is a whole slew of relational offerings. Then we have Cosmos DB and then lots of analytical offerings as well.

“If you are a relational app, and if you are using a relational database, and you are migrating from on-prem to Azure, then we recommend the relational family. It comes with this fundamental scale caveat which is that up to 4TB. Most of those customers are settled because they have designed the app around those sorts of scalability limitations.

“A subset of those customers, and a whole bunch of brand new customers, are willing to re-write the app. They know that that they want to come to cloud for scale. So then we pitch Cosmos DB.

“Then there are customers who want to do massive scale offline analytical processing. So there is, Databricks, Spark, HD Insight, and that set of services.

“We realise there are grey lines between these offerings. We’re tightening up the guidance, it’s valid feedback.”

Any numbers to flesh out the idea that this is a fast-growing service for Microsoft?

“I can tell you that the number of new clusters we provision every week is far more than the total number of clusters we had in the first month. The growth is staggering.”

Microsoft and GitHub, and will GitHub get worse?

Microsoft has announced an agreement to acquire GitHub for $7.5 billion (in Microsoft stock). Nat Friedman, formerly CEO of Xamarin, will become GitHub’s CEO, and GitHub will continue to run somewhat independently. A few comments.

image

Background: GitHub is a cloud-based source code repository based on Git, a distributed version control system created by Linus Torvalds. It is free to use for public, open source projects but charges a fee (from 7$ to $21 per user per month) for private repositories.

First, why? This one is easy. Microsoft is a big customer of GitHub. Microsoft used to have its own hosting service for open source software called CodePlex but abandoned it in favour of GitHub, formally closing CodePlex in March 2017:

Over the years, we’ve seen a lot of amazing options come and go but at this point, GitHub is the de facto place for open source sharing and most open source projects have migrated there. We migrated too.

said Brian Harry.

Microsoft also uses GitHub for its documentation, and this has turned out to be a big improvement on its old documentation sites.

Note also that Microsoft has many important open source projects of its own, including much of its developer platform (.NET Core, ASP.NET Core and Entity Framework Core). Many of its projects are overseen by the .NET Foundation. Other notable open source, Github-hosted projects include Visual Studio Code, a programmer’s editor that has won many friends, and TypeScript, a typed superset of JavaScript that compiles to standard JavaScript code.

When big companies become highly dependent on the services of another company they may become anxious about it. What if the other company were taken over by a competitor? What if it were to run into trouble, or to change in ways that cause problems? Acquisition is an easy solution.

In the case of GitHub, there was reason to be anxious since it appears not to be profitable – unsurprising given the large number of free accounts.

Second, Microsoft is always pitching to developers, trying to attract them to its platform and especially Azure services. It has a difficult task because it is the Windows company and the Windows platform overall is in decline, versus Linux on servers and Android/iOS on mobile. Therefore it is striving to become a cross-platform company, and with considerable success. I discuss this at some length in this piece. Note that there is a huge amount of Linux on Azure, including “more than 40%” of the virtual machines. More than 50%? Maybe.

If Microsoft can keep GitHub working as well as before, or even improve it, it will do a lot to win the confidence of developers who are currently outside the Microsoft platform ecosystem.

image

Will GitHub get worse?

The tricky question: under Microsoft, will GitHub get worse? The company’s track record with acquisitions is spotty, ranging from utter disasters (Nokia, Danger) to doubtful (Skype), to moderately successful so far (LinkedIn, Xamarin).

Under the current leadership, I doubt anything bad will happen to GitHub. I’d guess it will migrate some infrastructure to Azure (GitHub runs mainly from its own datacentres as I understand it) but there is no need to re-engineer the platform to run on Windows.

Some businesses will be uncomfortable hosting their valuable source code with Microsoft. That is understandable, in the same way that I hear of retailers reluctant to use Amazon Web Services (since it is a platform owned by a competitor), but it is a low risk. Others have long-standing mistrust of Microsoft and will want to migrate away from GitHub because of this.

Personally I think it is right to be wary of any giant global corporation, and dislike the huge and weakly regulated influence they have on our lives. I doubt that Microsoft is any worse than its peers in terms of trustworthiness but of course this is open to debate.

Another point: with this acquisition, free GitHub hosting for open source projects will be likely to continue. The press release says:

GitHub will retain its developer-first ethos and will operate independently to provide an open platform for all developers in all industries. Developers will continue to be able to use the programming languages, tools and operating systems of their choice for their projects — and will still be able to deploy their code to any operating system, any cloud and any device.

It is of course in Microsoft’s interests to make this work and the success of Visual Studio Code and TypeScript (which also come from the developer side of the company) shows that it can make cross-platform projects work. So I am optimistic that GitHub will be OK.

Update: I’ve noticed Sam Newman and Martin Fowler taking this view, a good sign from a people I respect and who are by no means from the usual Microsoft crowd.

image

Official announcements

Press release: https://news.microsoft.com/?p=406917

Chris Wanstrath’s Blog Post: https://blog.github.com/2018-06-04-github-microsoft/

Satya Nadella’s Blog Post: https://blogs.microsoft.com/?p=52553832

Cosmos DB or SQL Server? Do you need Kubernetes? VM or App Service? A guide to Azure worth checking out

One of the best features of Microsoft Build, possibly the best, is the exhibition. Microsoft sets up stands for each of its product teams, and the staff there generally include the people who actually build that product, making this a great way to interact with them and get authoritative answers to questions.

I interviewed several executives at Build and asked a couple of times, how can your customers work out which Azure service is the best fit for what they need? It is not a trivial question, now that there are so many different services which overlapping functionality.

It is critically important. You can waste a large amount of money and cause unnecessary frustration by selecting the wrong services.

None of these executives mentioned that Microsoft has a rather good guide for exactly this question. It is called the Azure Architecture Center and I discovered it on the show floor.

image

The stand was called Azure Clinic and I told the guy his costume reminded me of Dr GUI. He was too young to remember this MSDN character of old but another guy on the stand overheard and said it brought back bad memories!

You can find the Azure Architecture Center here. It does not make any assumptions about the depth of knowledge you have, which seems right to me since it is aimed at developers who are not sure exactly what they need. There is a ton of useful material, like this decision tree for the compute services (click to enlarge):

image 

Recommended.