SharePoint CSOM to Traverse All Sites in SharePoint Online

   In the past I’ve written posts for “PowerShell to Enumerate SharePoint 2010 or SharePoint 2013 Permissions” or “PowerShell Script To Traverse All Sites In SharePoint 2010 (or 2007) Farm” to assist with traversing through all sites within a SharePoint on-prem farm.  In this post I’ll share a snippet I recently used for traversing through all site collections and subsites within a SharePoint Online tenant.

Background

   If you’ve worked with the SharePoint Online Management Shell you may know that originally it was not able to retrieve Personal Sites (also known as My Sites / OneDrive for Business sites) in SharePoint Online.  As far as I’m aware this was primarily a limitation of the underlying client side libraries (Microsoft.SharePoint.Client.*).  Fast forward to a recent release (I don’t have the specific one but I can confirm it is in the 16.1.6621.1200 release of the Microsoft.SharePointOnline.CSOM NuGet package) and now it is supported to retrieve Personal Sites.  In the management shell this is accomplished by calling the following:

Connect-SPOService -Url ‘<tenantAdminUrl>’

Get-SPOSite -Limit all -IncludePersonalSite $true

   The problem is that if you want to traverse all site collections in SharePoint Online in client side object model (CSOM) you would need to know how the SharePoint Online Management Shell implements that inclusion of Personal Sites with the rest of site collections.  In order to find this out I used a disassembler (ILSPY in my case, but there are many alternatives available as well) on the underlying libraries to recreate the process in my own code.

Solution

   The resulting CSOM that I came up with is below.  The yellow code is processing all site collections within SharePoint Online while the green code is processing sites (subsites / webs / “not site collections”).  Feel free to borrow this and use in your own code but note that it is provided as-is with no warranty.

SPOSitePropertiesEnumerable ssp = null;

SPOSitePropertiesEnumerableFilter sspFilter = new SPOSitePropertiesEnumerableFilter();

SharePointOnlineCredentials creds = new SharePointOnlineCredentials(“myUsernameGoesHere”, securePassword);

using (PnPClientContext cc = new PnPClientContext(“myURLGoesHere”))

{

    cc.Credentials = creds;

   Tenant tenant = new Tenant(cc);

    //loop through all site collections including personal sites (even though not being used)

    //borrowed this code from after decompiling SPO Management Shell assemblies

    sspFilter.IncludePersonalSite = PersonalSiteFilter.Include;

    sspFilter.IncludeDetail = true;

    sspFilter.StartIndex = null;

    ssp = tenant.GetSitePropertiesFromSharePointByFilters(sspFilter);

    cc.Load(ssp);

    cc.ExecuteQuery();

    foreach (SiteProperties sp in ssp)

    {

        //DO YOUR WORK HERE FOR EACH SITE COLLECTION, such as looping through subwebs

        cc.Load(cc.Web, w => w.NoCrawl,

                                    w => w.Webs,

                                    w => w.Url);

        cc.ExecuteQuery();

        //check subweb(s)

        foreach (var subweb in cc.Web.Webs)

        {

            cc.Load(subweb, sw => sw.NoCrawl,

                                        sw => sw.Url);

            cc.ExecuteQuery();

        }

    }

}

Conclusion

   In this post I shared a snippet for traversing all site collections in SharePoint Online with C# CSOM code.  In my daily job I’ve been using this in combination with Azure Functions for a number of interesting interactions with a SharePoint Online tenant.  More to come on those scenarios in future weeks.  For now let me know in the comments if you have any questions or issues implementing the above snippet in your own code.  Happy coding!

      -Frog Out

3 thoughts on “SharePoint CSOM to Traverse All Sites in SharePoint Online

  1. Hi Brian – I had similar code, but I noticed that it wasn’t returning all of my site collections. My original code would max around 200 site collections, IIRC.

    Here’s what we’re using now:

    private static void GetListOfSiteCollections()
    {
    using (ClientContext adminContext = new ClientContext({https://tenant-admin.sharepoint.com}))
    {
    int startIndex = 0;
    adminContext.Credentials = new SharePointOnlineCredentials(SharePointAdminUsername, SharePointAdminPassword);

    var tenant = new Tenant(adminContext);
    SPOSitePropertiesEnumerable spp = null;

    while (spp == null || spp.Count > 0)
    {
    spp = tenant.GetSiteProperties(startIndex, true);
    adminContext.Load(spp);
    adminContext.ExecuteQuery();

    foreach (SiteProperties sp in spp)
    {

    SiteProperties sprops = tenant.GetSitePropertiesByUrl(sp.Url, true);

    adminContext.Load(sprops);
    adminContext.ExecuteQuery();

    string SiteCollectionURL = sprops.Url;

    }

    startIndex += spp.Count;
    }
    }
    }

    Like

    • Kelly,

      It looks like you are traversing SharePoint Online sites when my code was for on-prem. I recently had a need to traverse all sites within an SPO tenant for a customer and found that the coded needed is quite different. It is similar to what you have but with a few differences to include personal sites. Will be blogging it out this week or next. Thanks for sharing your code. Hope all is well with you.

      -Brian T. Jackett

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s