[Update 2018-07-31] Special thanks for my peer John Ferringer for pointing out an issue with my prior CSOM snippet. Updating CSOM code to handle large number of sites as previous sample would truncate results if large number of sites were traversed.[/Update]
In the past I’ve written posts for “PowerShell to Enumerate SharePoint 2010 or SharePoint 2013 Permissions” or “PowerShell Script To Traverse All Sites In SharePoint 2010 (or 2007) Farm” to assist with traversing through all sites within a SharePoint on-prem farm. In this post I’ll share a snippet I recently used for traversing through all site collections and subsites within a SharePoint Online tenant.
Background
If you’ve worked with the SharePoint Online Management Shell you may know that originally it was not able to retrieve Personal Sites (also known as My Sites / OneDrive for Business sites) in SharePoint Online. As far as I’m aware this was primarily a limitation of the underlying client side libraries (Microsoft.SharePoint.Client.*). Fast forward to a recent release (I don’t have the specific one but I can confirm it is in the 16.1.6621.1200 release of the Microsoft.SharePointOnline.CSOM NuGet package) and now it is supported to retrieve Personal Sites. In the management shell this is accomplished by calling the following:
Note: If you do not see the below Gist please refer to code at this location: PS-Get_All_Sites_SPO.ps1
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Connect-SPOService –Url '<tenantAdminUrl>' | |
| Get-SPOSite –Limit all –IncludePersonalSite $true |
The problem is that if you want to traverse all site collections in SharePoint Online in client side object model (CSOM) you would need to know how the SharePoint Online Management Shell implements that inclusion of Personal Sites with the rest of site collections. In order to find this out I used a disassembler (ILSPY in my case, but there are many alternatives available as well) on the underlying libraries to recreate the process in my own code.
Solution
The resulting CSOM that I came up with is below. Feel free to borrow this and use in your own code but note that it is provided as-is with no warranty.
Note: If you do not see the below Gist please refer to code at this location: CSOM_Traverse_All_Sites_SPO.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| List <SiteProperties> list = new List <SiteProperties>(); | |
| SPOSitePropertiesEnumerable ssp = null; | |
| SPOSitePropertiesEnumerableFilter sspFilter = new SPOSitePropertiesEnumerableFilter(); | |
| SharePointOnlineCredentials creds = new SharePointOnlineCredentials("myUsernameGoesHere", securePassword); | |
| using (ClientContext cc = new ClientContext("myURLGoesHere")) | |
| { | |
| cc.Credentials = creds; | |
| String nextIndex = null; | |
| Tenant tenant = new Tenant(cc); | |
| //loop through all site collections including personal sites (even though not being used) | |
| //borrowed this code from after decompiling SPO Management Shell assemblies | |
| sspFilter.IncludePersonalSite = PersonalSiteFilter.Include; | |
| sspFilter.IncludeDetail = true; | |
| do | |
| { | |
| sspFilter.StartIndex = nextIndex; | |
| ssp = tenant.GetSitePropertiesFromSharePointByFilters(sspFilter); | |
| cc.Load(ssp); | |
| cc.ExecuteQuery(); | |
| list.AddRange(ssp); | |
| nextIndex = ssp.NextStartIndexFromSharePoint; | |
| }while(nextIndex != null); | |
| foreach (SiteProperties sp in list) | |
| { | |
| //DO YOUR WORK HERE FOR EACH SITE COLLECTION, such as looping through subwebs | |
| cc.Load(cc.Web, w => w.NoCrawl, | |
| w => w.Webs, | |
| w => w.Url); | |
| cc.ExecuteQuery(); | |
| //check subweb(s) | |
| foreach (var subweb in cc.Web.Webs) | |
| { | |
| cc.Load(subweb, sw => sw.NoCrawl, | |
| sw => sw.Url); | |
| cc.ExecuteQuery(); | |
| } | |
| } | |
| } |
Conclusion
In this post I shared a snippet for traversing all site collections in SharePoint Online with C# CSOM code. In my daily job I’ve been using this in combination with Azure Functions for a number of interesting interactions with a SharePoint Online tenant. More to come on those scenarios in future weeks. For now let me know in the comments if you have any questions or issues implementing the above snippet in your own code. Happy coding!
-Frog Out
The Frog Pond of Technology 
Hi Brian – I had similar code, but I noticed that it wasn’t returning all of my site collections. My original code would max around 200 site collections, IIRC.
Here’s what we’re using now:
private static void GetListOfSiteCollections()
{
using (ClientContext adminContext = new ClientContext({https://tenant-admin.sharepoint.com}))
{
int startIndex = 0;
adminContext.Credentials = new SharePointOnlineCredentials(SharePointAdminUsername, SharePointAdminPassword);
var tenant = new Tenant(adminContext);
SPOSitePropertiesEnumerable spp = null;
while (spp == null || spp.Count > 0)
{
spp = tenant.GetSiteProperties(startIndex, true);
adminContext.Load(spp);
adminContext.ExecuteQuery();
foreach (SiteProperties sp in spp)
{
SiteProperties sprops = tenant.GetSitePropertiesByUrl(sp.Url, true);
adminContext.Load(sprops);
adminContext.ExecuteQuery();
string SiteCollectionURL = sprops.Url;
}
startIndex += spp.Count;
}
}
}
LikeLike
Kelly,
It looks like you are traversing SharePoint Online sites when my code was for on-prem. I recently had a need to traverse all sites within an SPO tenant for a customer and found that the coded needed is quite different. It is similar to what you have but with a few differences to include personal sites. Will be blogging it out this week or next. Thanks for sharing your code. Hope all is well with you.
-Brian T. Jackett
LikeLike
Kelly,
Completely forgot that I had already blogged about the SPO traversal script. See this post for more details.
https://briantjackett.com/2017/08/23/sharepoint-csom-to-traverse-all-sites-in-sharepoint-online/
LikeLike