Wrestling with SharePoint and Office 365: code to bulk move documents

I have mixed feelings about SharePoint, Microsoft’s flexible but infuriating collaboration platform. It makes difficult things easy and easy things difficult, or something like that. Today’s story is an example, and may also be of interest if you are wondering how to write code that manipulates documents in SharePoint as found in Office 365.

The problem started when some contacts of mine who use Office 365 could not open a folder in Windows Explorer. They received a permission error along with the famous invitation to “Contact your network administrator to request access.”


The folder in question is actually a SharePoint folder which accesses Office 365 through WebDAV. I took a look, and found that, whatever the problem was, it had nothing to do with permissions. I also observed that there was no problem accessing the folder through the web browser; but like many users, these people prefer to use Explorer.

Next, I started moving files out of the troublesome folder into another one. I began to suspect that some rogue document was causing the error. This suspicion proved correct, but it was not easy to track down. The problem: the SharePoint web user interface does not provide any bulk copy or move option. If you want to move a bunch of documents, the recommended way is to use Windows Explorer, the exact feature that in this instance was not working.

Moving documents one by one through a laborious Web UI is no fun, so I then had the bright idea of writing some code to move the documents. This means taking a dive into the labyrinthine SharePoint API.

I was surprised how hard this is. Here is how I got started:

First, I downloaded the SharePoint SDK and run the setup. I chose to install only the Foundation help and samples.

Next, I created a new Windows Forms project in Visual Studio 2010. Note you must set the project to target the full .NET Framework 4.0, not just the Client profile

After that, I had to copy two DLLs from my own SharePoint 2010 server. These are in:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI

I am surprised they are not included with the SDK.

After that, I looked for some code samples for the SharePoint Client Object Model. You can find this described here or consult the reference here. It is a capable API, but you soon realise why there is plenty of work for SharePoint specialists. ClientContext, CAML queries, FolderServerRelativeUrl: there is a lot to get your head around.

The first problem I had though was authentication. Office 365 uses claims-based authentication, whereas all the SharePoint API examples seem to assume you are on an intranet and already authenticated for your SharePoint server. Coding for claims-based authentication is a headache.

I tried code from here to authenticate against the Office 365 claims-based federation server, but with no success. It seems to be based on beta code and does not work now. I then read the official document on the subject here and downloaded the sample code. Here is what worked for me:

Add the ClaimsAuth project from the sample to my Windows Forms solution.

Modify this line in ClaimsWebAuth.cs:


to this


the reason being that ClaimsAuth is designed for a console application.

Then I could run some basic Client Object Model code like this:

ClientContext cc = ClaimClientContext.GetAuthenticatedContext(url);

            if (cc != null)
                lbTest.Text = "Title: " + cc.Web.Title;

All this does is to connect to the Office 365 SharePoint site at url and display its title – but if you can do that, you have got past the first hurdle.

Next I had to figure out how to move all the documents in one folder to another. Again, I found this tricky. I was able to list all the items in a library, which is the top-level folder for a collection of documents, but how do you list all the items in a subfolder? Something to do with CAML, it seems, also known as Collaborative Application Markup Language. Does anyone out there love CAML? I thought not. CAML queries are like SQL queries chopped up into XML elements.

Another characteristic of the Client Object Model is that you constantly have to call the Load and ExecuteQuery methods of your ClientContext object, otherwise you will get a PropertyOrFieldNotInitializedException. There is a good reason for this, as it reduces the amount of data passing over the wire, but it can also be perplexing.

Here is the code I ended up with, where “docs” is the top-level folder or library name,

if (cc != null)
                var site = cc.Web;

                var lib = site.Lists.GetByTitle("docs");
                CamlQuery camlQuery = new CamlQuery();
                camlQuery.FolderServerRelativeUrl = @"/docs/path/to/sourcefolder";

                string camlQueryXml = "<View>" +
                    "<Query>" +
                      "<OrderBy>" +
                        "<FieldRef Name=’FileLeafRef’ Ascending=’True’ />" +
                      "</OrderBy>" +
                    "</Query>" +
                camlQuery.ViewXml = camlQueryXml;

                ListItemCollection lis = lib.GetItems(camlQuery);


                ListItem FolderCF = null;
                foreach (ListItem li in lis)

                    string sTitle = li.FieldValues["FileLeafRef"].ToString();

                  // you can inspect the title to see if you want to move the file,

                  //eg only those beginning with a letter in the first half of the alphabet

                        File thefile = li.File;


                        if (!li.File.ServerObjectIsNull.Value)
                            string dest = @"/docs/path/to/destfolder/" + thefile.Name;
                            thefile.MoveTo(dest, MoveOperations.Overwrite);



Pretty simple? Maybe it is to a SharePoint guru; all I can say is that I did not find it intuitive.

Note that this is not intended as production code so if you borrow it please add exception handling etc. It was a quick hack to solve a problem.

The good news is that once I was able to move documents from folder to folder programmatically, I was able to troubleshoot the original problem. Mysteriously, there was one document which, if it was in a folder, caused the access denied error when opened with WebDAV. Once I isolated the document, I discovered that if I renamed it, the problem went away. Curiously, there are no special characters in the name, just letters and spaces, so this is something of a mystery.

Still, it was a useful exercise, especially since moving a batch of documents using the Client Object Model seems quicker than using Explorer and WebDAV.

Office in Windows RT: not licensed for business use?

Journalist Jon Honeyball remarked on Twitter that the version of Microsoft Office in Windows RT, and therefore in the first Microsoft Surface Tablet, is Office Home and Student 2013.

I was sceptical, but it is there on the spec sheet [pdf]:


We already knew that Outlook is missing; but now it seems possible that Office in Windows RT is licensed only for non-commercial use. Here is the statement about Office 2010 Home and Student:

I own or work for a small business; can I use Office Home and Student 2010 at my work?

No. Office Home and Student 2010 is licensed only for non-commercial use for members of your household.

Such a restriction would blow so large a hole in the positioning of Windows RT as the ideal BYOD (Bring Your Own Device) for business that I am inclined to believe it will be changed for Office in Windows RT.

Then again, Office is a huge business for Microsoft and it is easy to hear the internal debate over this. “You cannot just give it away”.

Another possibility is that Microsoft will come up with some licensing deal which permits use of Office in Windows RT at work, for a suitably Enterprisey fee.

Update: Note that Microsoft has already announced a few things here about Windows RT licensing:

Windows RT Virtual Desktop Access (VDA) Rights: When used as a companion of a Windows Software Assurance licensed PC, Windows RT will automatically receive extended VDA rights. These rights will provide access to a full VDI image running in the datacenter which will make Windows RT a great complementary tablet option for business customers.

Companion Device License: For customers who want to provide full flexibility for how employees access their corporate desktop across devices, we are introducing a new Companion Device License for Windows SA customers. For users of Windows Software Assurance licensed PCs this optional add-on will provide rights to access a corporate desktop either through VDI or Windows To Go on up to four personally owned devices.

This means that if you have a PC licensed with Windows Software Assurance, you can access a virtual desktop from Windows RT without further charge.

Generally, I believe Microsoft also allows you to use Remote Desktop into a physical client without an additional license, provided it is single-user. In other words, only one user at a time can use a physical Windows 7 installation, whether sitting at the machine or remotely.

None of these provisions covers Office on the client though. They are concerned only with remote desktop access of various kinds.

Microsoft to make its own tablet called Surface, puts Windows RT centre stage

Microsoft has announced its own tablet, called Surface, for “work and play”, said CEO Steve Ballmer at an event in Los Angeles yesterday.


The first of what will be a family of devices has a 10.6” Corning Gorilla Glass screen, is just 9.3mm thick, and has a magnesium “VaporMg” case with a built-in stand/magnetic cover which doubles as a multitouch keyboard.

Surface comes in two forms. One runs Windows RT with an NVidia processor, which means it is the ARM version of Windows 8. There is a desktop UI alongside Metro, but the desktop is there only to run Microsoft Office (which is bundled), Explorer, and whatever other utilities Microsoft chooses to include. It is not possible to install new desktop applications. Users can only install Metro-style apps from the Windows Store.

The other runs Windows 8 Professional. Note that this x86 version is heavier (903g vs 676g), thicker (13.5mm vs 9.3mm) and more power-hungry (42 W-h vs 31.5 W-h). However, it does benefit from USB 3.0 rather than USB 2.0.

Why has Microsoft done this, and risked alienating the hardware partners on which it depends for the success of Windows?

I posted on this subject a few days ago. Yes, Microsoft’s hardware partners have driven the success of Windows, but they have also been part of the problem as Apple has captured a gradually increasing proportion of the personal computer market. Problems include foistware(unwanted software) bundled with PCs and rushed designs that have too many annoyances.

With Windows Phone 7, it was not until Nokia entered the market a year after the launch that we saw hardware and design quality that does justice to the operating system. Distracted by Android, partners like HTC and Samsung brought out drab, unimaginative phones that contributed to a poor start for Microsoft’s smartphone OS.

Now with Windows 8, the danger is that the same may happen again. We have seen few Windows RT designs, and evidence that vendors are having difficulty in reimagining Windows.

Until today, that is. The announcement ensures that Windows RT will win plenty of attention at launch, alongside the x86 editions, and that Microsoft has a measure of control over its own destiny, in how Windows 8 is realised in hardware.

Microsoft says that the Windows RT Surface will launch at the same time as Windows 8, but that the Intel edition will follow a few months later.

How much for a Surface? The press release says:

Suggested retail pricing will be announced closer to availability and is expected to be competitive with a comparable ARM tablet or Intel Ultrabook-class PC. OEMs will have cost and feature parity on Windows 8 and Windows RT.


Close up with Asus PadFone: is a converged device in your future?

Asus held an event in London to show off the devices it revealed at Computex in Taipei recently, though sadly there was no Windows RT device to be seen.

Among the Zenbook Ultrabooks and Transformer Primes there was something innovative though, which was a near-final sample of the PadFone, which combines smartphone, tablet and Android laptop into one package.

The thinking is simple: why have an expensive smartphone as well as an expensive tablet, each perhaps with its own SIM card and contract, when the smartphone can power both? In the PadFone, the phone docks into the tablet, and the tablet clips into a keyboard case. As a final flourish, there is an optional headset stylus, a stylus with a Bluetooth headset built-in so you can answer the phone easily when it is docked.

Here are the three main pieces:


The tablet, note, is useless until you dock the phone. You do this by opening a flap on the back and dropping it in.


The tablet then works just like any other Android tablet, though it is heavier than average, and has a bulbous section on the underside.

Attach to the keyboard case, and you have a laptop.


The tablet has a 10.1”, 1280 x 800 screen with Gorilla Glass, a speaker and headphone jack, and a front-facing camera.

The phone has 1GB RAM, 16GB flash storage plus Micro-SD support, Qualcomm 8260A Snapdragon S4 Dual-core processor with Andreno 225 GPU, rear camera and its own front-facing camera, and runs Android ICS.

The keyboard adds USB ports and a card reader.

Each device has its own battery so a full setup has three batteries, or  four if you count one in the stylus headset. However you can have scenarios where the tablet is out of power but the phone is not, for example, which would be annoying.

I spent some time with the PadFone, scribbling on the excellent note-taking app which comes with it, and assembling and disassembling the unit to get a feel for how it works. There is plenty to like. The phone itself looks great and seems fast and capable. Docking and removing it is straightforward, particularly since the flap acts as a lever to eject the phone gently. Asus assured me that it has been tested for thousands of insertions. The tablet worked well too, though it is heavier than most and the protrusion which holds the smartphone is inelegant.

A winner then? I am not sure. It is interesting and innovative, but the mechanics need some refinement. Most people have a case to protect their smartphones, but for the PadFone you will either need to remove the phone from its case when you dock it, or else treat the tablet as the case, in which case it will not slip so easily into a jacket pocket or handbag.

The stylus headset is not just a gimmick; you will need this, or another Bluetooth headset, to make sense of using the phone when it is docked.

Some variations on this theme occur to me. After another generation of miniaturisation, perhaps you could design a phone so slim that it fits into the case more like an old PCMCIA card used to slot into a laptop, without an ugly protruding flap? Another idea would be to make all the communication between phone and tablet wireless, building just enough smarts into the tablet that it works as a kind of remote desktop into your phone.

The Asus folk present told me that the PadFone is first-generation and we can expect the concept to evolve. Another goal is to make a splash in the smartphone market, using the PadFone as differentiation from all the other Android devices out there.

Apparently the PadFone will normally be sold on contract, and while it will be bundled with the tablet, whose name is the PadFone Station, the keyboard and stylus headset will be optional extras.

BlackBerry 10: QNX multitasking goodness

I attended the London BlackBerry Jam, one of around 500 developers (the event was sold out) who showed up to learn about developing for Blackberry 10 and in the hope of snagging a prototype of RIM’s next smartphone, the BlackBerry 10 Dev Alpha. The event is part of a tour of 26 cities worldwide. I also spoke to Vivek Bhardwaj, Head of Software Portfolio EMEA for RIM. Does BlackBerry 10 have what it takes to  compete against Apple iPhone and Google Android?

A few quick observations. The event was split into native and web tracks, with the native track focusing on  C/C++ development and the Cascades UI framework, and the web track covering WebWorks, the HTML 5 developer tools which you can use to target mobiles as far back as BlackBerry 5, or to create apps that share code between BlackBerry and other mobile platforms.

There was also a tour of various mobile JavaScript libraries. One that caught my attention was Zepto.js, which implements most of the jQuery API in just 8.4k of compressed code, around 25% of the size of jQuery. The trade-off, aside from some missing features, is lack of support for Internet Explorer, though support for IE 10 is under consideration. Thought-provoking: the price of legacy platforms?

It turned out that the device Bhardwaj was holding, though pretty much the same hardware as the Dev Alpha giveaway, was more revealing as a glimpse of the next-generation BlackBerry. The Dev Alpha, which has an impressive 1280 x 768 screen, comes with no applications other than the web browser and its user interface is pretty much that of the PlayBook. Bhardwaj’s demo device on the other hand is a complete early version of BlackBerry 10 though there were parts that he would not demonstrate. I went to compose an email and he said, “we are not ready to show that yet.”


So what is distinctive about BlackBerry 10? One of its key features is multitasking, thanks to the processor scheduling capabilities of QNX, the embedded operating system which underlies both PlayBook and BlackBerry 10. According to Bhardwaj, this enables up to 8 apps to run at once. “Applications all run simultaneously. We don’t need to pause them,” he said. “It’s much more about flow.”

Although apps do run full screen, you can take advantage of the multitasking by “peeking” at a background app. This means holding your finger towards the corner of an app and dragging it left to see a little of what is underneath. In the pic below this is another image.


Talking to developers at the event I picked up considerable enthusiasm for BlackBerry 10, though compatibility is a headache. If your customer asks you to support all versions of BlackBerry back to the 5.0 series, you are stuck with WebWorks and the pre-WebKit browser in 5.0. If you can convince your customer to forget 5.0, then you can develop for WebKit. If you want to use Cascades then you are restricted to PlayBook and BlackBerry 10.

BlackBerry 10 also supports Adobe AIR, for Flash-based apps, and an Android runtime for repackaged Android apps.

The prototype BlackBerry 10 phone looks good, but as a brand new platform is it sufficiently exciting to revive RIM’s fortunes? “I don’t believe that in a trillion-dollar plus industry there can only be two players. I think there is more than enough space for four or five platforms. It’s very short-sighted if we think there can only be three ecosystems,” said Bhardwaj.  

Three reasons why Microsoft should make its own Windows RT (ARM) Tablet

Rumours are flying that Microsoft will announce an own-brand Windows RT tablet on Monday.

No comment on the truth of these, but it would be a smart move.

Here are three reasons.

First, the OEM foistware problem. This has got a little better in recent years, but not enough to compete with Apple and its clean machines. The problem is so bad that Microsoft set up its own retail stores to sell  cleaned-up Windows PCs:

Many new PCs come filled with lots of trialware and sample software that slows your computer down—removing all that is a pain, so we do it for you! Every PC the Microsoft Store sells is put on a software diet and performance is tuned to run the best it can.

Microsoft addressed this in Windows Phone by imposing conditions on the extent to which OEMs can customise the user interface or embed their own software. It cannot do this  though with Windows 8 on x86. Manufacturing its own model is one of the few ways Microsoft can get Windows PCs that work as designed into the hands of consumers.

Second, the design problem. Few Windows PCs (if any) are as well designed as Macs or iPads. Manufacturers are geared towards low prices and frequent model changes rather than intensive work on every detail of the design.

Third, Microsoft wants to make a splash with Windows RT, the ARM version, and there is evidence that it is having difficulty communicating its benefits or convincing its OEM partners to get fully behind it.

This third is the biggest issue, which might drive Microsoft to compete with its third-party partners, and requires some explanation. Many people I speak to cannot see the point of Windows RT. This version of Windows 8 will not run x86 applications, so you cannot install any of your old software. Further, there is no way to install desktop applications, so software vendors cannot port their existing applications. They must create new Metro-style apps instead. So why bother with Windows RT?

This reaction is understandable, but unfortunately for Microsoft Windows 8 on x86 has no chance of competing with Apple’s iPad.

Yesterday I attended an Asus event in London where the company was showing its new range of Android tablets and Windows ultrabooks. It was not showing its prototype Windows 8 machines, but I was able to discuss the likely Windows 8 products, All but one is x86, and they will have Wacom digitizers,  which means they will work with a stylus like an old-style Tablet PC, as well as with touch. That will push up the price.

Worse still, these x86 devices, like the Samsung Slate on which I run WIndows 8 Release Preview, will not be enjoyable to use with touch alone. Users will find themselves running applications designed for keyboard and mouse: you can get them to work, but it is frustrating. These devices are not Windows reimagined, they are the old Windows plus a few new tricks.

Too expensive, too hard to use: Windows 8 on x86 is not an iPad-beater.

Windows RT on the other hand is more promising. This does have a desktop, but it will only run Office, Windows Explorer, and whatever other desktop utilities Microsoft chooses to provide. Office aside, you will be forced to use touch-friendly Metro apps most of the time. Microsoft can tune Windows RT by removing legacy components that are no longer needed, because applications which rely on them cannot be installed. You also get the power efficiency of ARM, so a long battery life. Finally, if Microsoft has done it right Windows RT should be more secure, since the entire operating system is locked down.

Windows RT is critical to Microsoft and if it has to make its own hardware in order to market it properly, then it should do so.

Amazon web service APIs: a kind of cloud standard?

I am at the Cloud Computing World Forum in London where one of the highlights was a keynote yesterday from Amazon CTO Werner Vogels. Amazon, oddly enough, does not have a stand here; yet the company dominates the IAAS (Infrastructure as a service) market and has moved beyond that into more PAAS (Platform as a service) type services.


Vogels said that the reason for Amazon’s innovation in web services was its low margin business model. This was why, he said, none of the established enterprise software companies had been able to innovate in the same way.

He added that Amazon did not try to lock its customers in:

If this doesn’t work for you, you should be able to work away. You should be in charge, not the enterprise software company.

Sounds good; but Amazon has its own web services API and if you build on it there is an element of lock-in. Or is there? I was intrigued by a remark made by Huawei’s Head of Enterprise R&D John Roese at a recent cloud computing seminar:

We think there is an imperative in the industry to settle on standardised interfaces. There should be no more of this rubbish where people think they can differentiate based on proprietary interfaces in the cloud. A lot of suppliers are not very interested in this because they lose the stickiness of the solution, but we will not see massive cloud adoption without that portability. 

But what are these standardised interfaces? Roese said that Huawei uses Amazon APIs. For example, the Huawei Cloud Storage Engine:

The CSE boasts a high degree of openness. It supports S3-like interfaces and exposes the internal storage service enabler to 3rd party applications.

There is also Eucalyptus, open source software for private clouds that uses Amazon APIs. Is the Amazon web services API becoming a de-facto standard, and what does Amazon itself think about third parties adopting its API?

I asked Vogels, whose response was not encouraging for the likes of Huawei treating it as a standard. The question I put: is the adoption of Amazon’s API by third parties influencing his company in its maintenance and evolution of those APIs?

It is not influencing us. It is influencing them. Who is adopting our APIs? We licensed Eucalyptus. People ask us about standardisation. I’d rather focus on innovation. If others adopt our APIs, I don’t know, we rather focus on innovation.

The lack of interest in standardisation does undermine Vogels’ comments about the freedom of its customers to walk away, though lock-in is not so bad if your use of public cloud is primarily at the IAAS level (though you may be locked into the applications that run on it).

Another mitigating factor is that third parties can wrap cloud management APIs to make one cloud look like another, even if the underlying API is different. Flexiant, which offers cloud orchestration software, told me that it can do this successfully with, for example, Amazon’s API and that of Microsoft Azure. Perhaps, then, standardisation of cloud APIs matters less than it first appears.

Would you be happy to visit your doctor online? John Sculley says most of us should

I’m at the Cloud Computing World Forum in London where former Apple CEO John Sculley has been speaking about healthcare in the cloud. Sculley is involved with a US company called MDLive which lets you make a virtual appointment with a doctor rather than turning up at your local surgery, sitting in the waiting room for an hour, and then getting 7 minutes consultation.


Sculley says this puts together several strands:

  • Most visits to the doctor are unnecessary
  • In person visits are more expensive and there is always pressure to reduce costs
  • Sensor technology is in its infancy but promises much – we already have Fitbit and the like, which monitors exercise, but in future your mobile may alert you to an impending heart attack or perform other automated monitoring of your health, and upload data to an internet service.

I asked Sculley if there had been studies of accuracy of diagnosis from an in-person versus an online appointment. The online ones are actually more accurate, he claims, because they make better use of available data.

Sculley calls online doctor surgeries an example of “Domain Expertise as a Service”, the implication being that the same kind of logic will apply to other kinds of consultation, not just healthcare.

Visual Studio LightSwitch to get HTML5 support

Microsoft’s Visual Studio LightSwitch is a rapid application development tool designed to create database-oriented, browser hosted applications with little code.

LightSwitch is intriguing because it does model-driven development. You design the model, LightSwitch generates the application. Microsoft’s idea was that non-specialist developer would like the tool, though there is little evidence of that. However, it does enable professional developers to put together functional applications quickly.

Unfortunately LightSwitch was conceived when Silverlight was all the rage at Microsoft. It generates Silverlight apps, which means they can run either on the desktop or in the browser, but also cuts out every device out there that does not run Silverlight. Including of course hot new mobile devices from Apple iOS to Android and even, on the Metro side, Windows 8.

But this is model-driven development, right? Just add code generation for HTML 5 and the same tool will generate standard browser-based apps.

Microsoft has announced just that at TechEd, under way this week in Florida. Visual Studio VP Jason Zander demonstrated LightSwitch for HTML5:


which means you can run the apps on iOS:


This feature is not in the current RC of Visual Studio 2012 but will be added later.

HTML 5 support puts LightSwitch back on the map as a RAD (Rapid Application Development) tool. But is there any appetite for model-driven development, which has all-but failed in so many other implementations?

That is as yet unproven, but at least the fate of LightSwitch no longer depends on that of Silverlight.

The new Windows Azure: a better cloud platform from Microsoft

Microsoft’s Scott Guthrie has posted more details of changes in Windows Azure. I was also able to sign up for the preview of Virtual Machines and Web Sites (my web site application is pending).

In the past the Azure portal for managing your cloud services has been functional but ugly and irritating. This has been replaced by a new portal (in preview) which is a great improvement.



Even better, the portal has a REST API which developers can program directly, giving Amazon-like programmatic control of your Azure infrastructure, though I have not looked at the actual API yet. The SDK is open source and hosted on GitHub under an Apace 2 license.

Guthrie talks about scaling in Azure web sites. You can control the number of VM instances used by your web site and “Windows Azure automatically handles load balancing traffic across VM instances”. The one thing I do not see is how you would have instances brought online and taken offline in response to demand. However, given the REST API you would imagine that writing code to do this would not be too challenging. The portal includes a dashboard for monitoring performance so the API can access this information.

You can have up to 10 web sites on a free shared hosting environment, and pay only when you upgrade to a dedicated VM.

Free is free; but once you do scale up it does not look cheap to me. Here are a couple of samples from the calculator:


or for a busier site:


These kinds of figures would put me off moving this site to Azure, for example. That said, I would be interested to see a detailed cost comparison between Azure, Amazon, and other cloud hosting providers like Rackspace.

There are more features mentioned in Guthrie’s post and it does look like an improvement both in features and usability.