In this post I’ll talk through my personal experience deploying the new cloud hybrid search service application for SharePoint 2013 (also available in SharePoint 2016).  By no means am I an expert on this topic (especially in many of the supporting technologies such as AAD Connect, AD FS, etc.) but this is more meant to increase exposure to this new offering.  For an overview of cloud hybrid search and more information about actual implementation (which I will refer back to later) please read through Cloud Hybrid Search Service Application written by two of my Microsoft peers Neil and Manas (they are the true experts).

 

Components

   Here is a list of the high level components I used for my deployment.

Note: My Azure VM configuration is not using best practices for where or how to deploy different services.  Also my mention of GoDaddy and DigiCert are purely for example purposes and not an endorsement for either company.  I just happen to use their services and products in this scenario.

• Office 365 (O365) trial tenant (sign up for one here)
• 4 Azure VMs
  • A1 – Active Directory Domain Services (AD DS)
  • A1 – Active Directory Federation Services (AD FS)
  • A2 – Azure Active Directory Connect (AAD Connect), Web Application Proxy (WAP)
  • A4 – SQL Server 2014, SharePoint 2013 farm with Service Pack 1 and at least Aug 2015 CU
• Custom domain (purchased through GoDaddy but any domain registrar should work)
  • Note: Office 365 does have a partnership with GoDaddy so configuration may be easier due to automated updates that can be performed
  • Additionally I was able to modify public DNS records through GoDaddy to allow federated authentication through AD FS
• SSL wildcard certificate purchased from DigiCert
  • Only required if want to allow Office 365 user to open / preview a search result that resides on-prem with Office Online Server (new version of Office Web Apps Server 2013, not discussed in this post)
  • I also used this certificate for other purposes such as securing AD FS communication and implementing Remote Desktop Gateway (the latter is unrelated to this post)
• Custom result source to display O365 search results in my on-prem farm

 

   Next we’ll take a look at some of these components more in depth.

 

SharePoint Server

   The new cloud hybrid search service application is available in SharePoint Server 2013 with the August 2015 CU or later.  I have heard from my peers that there are some issues with cloud hybrid search as of the October, November, and December 2015 CUs.  As such use either the August or September 2015 CUs at the time of this writing (Dec 8, 2015) or wait until the Jan 2016 CU which should contain the fix (link).  The SharePoint Server 2016 IT Preview 1 also supports cloud hybrid search although I have not tested it out myself.

 

Cloud Search Service Application

   To provision a cloud hybrid search service application the property CloudIndex on the service application must be set to True.  This property is a read-only property and can only be set at creation time.  As such you will need to create a new search service application in order to utilize the cloud hybrid search service.

   I have not tested creating a new cloud hybrid search service application using a restored backup admin database from an existing search service application.  The thought behind this would be to retain at least a portion of your existing search service application.  If you try this and have any findings let me know in the comments below.

 

Custom Domain

   A custom domain is not a requirement for cloud hybrid search.  I used one so that I could allow end users (demo accounts) to log into Office 365 as a federated user “someUser@<fakecompany>.com” rather than the default domain “someUser@<O365TenantDomain>.onmicrosoft.com”.

 

AAD Connect

   In order to search for on-prem content that has been indexed by Office 365 the user will need to have an account that is synchronized to Azure Active Directory / Office 365.  This allows the search service in Office 365 to show content based on the Access Control List (ACL) defined on-prem.

   There are multiple options available for synchronizing accounts between on-prem and AAD but the predominate ones include DirSync, AAD Sync, and AAD Connect.  Since AAD Connect is the future looking tool of choice of these three I decided to use it.  AAD Connect automates many of the tedious tasks of configuring federated authentication by stepping through a wizard.

   That said I did run into a number of issues during configuration due to missing certificates, invalid permissions, or other steps I missed or was unaware of.  If I got part of the way through the configuration and ran into a failure that I couldn’t recover from then I had to uninstall AAD Connect (do not remove all prerequisites when prompted), wipe out the contents of “<install drive>:Program FilesMicrosoft Azure AD SyncData”, and then re-install.

 

Display Search Results On-Prem

 

***PLEASE READ AS THIS IS IMPORTANT***

    The default scenario for cloud hybrid search is to index both on-prem and O365 content which are then queried in O365.  It is possible to create or modify an on-prem result source to use the remote index from your Office 365 tenant which allows for querying and display the combined search results on-prem.  The problem though is that when you query for and click results on-prem the search analytics click data is not incorporated back to the cloud index to further influence search results.

Ex. I queried for “SharePoint” in on-prem search center and clicked the 4th result on result page.  Multiple other users also searched for “SharePoint” and clicked the same 4th result.  SharePoint search (via timer jobs and other background processes) incorporates that click data and adjusts the 4th result to now appear higher in rankings upon subsequent search queries.

   I have unsuccessfully tested a few options to manually pass the search click data up to SharePoint Online.  These include creating a ClientContext object and calling the RecordPageClick() method on SearchExecutor, modifying the display template page, and more.  I did hear from a SharePoint MVP that successfully tested out a way to push search analytics data between on-prem and O365 but it took a fair amount of customizations to accomplish.  If I find out any additional information, workaround, or updates on this topic I’ll update this post to reflect that.

 

Example

   As you can see from the below screenshots I can initiate a search query from on-prem or O365 (respectively) and get the same combined result set.

 

 

 

 

Conclusion

   Due to my prior inexperience around AD FS, Web Application Proxy, AAD Connect, and other applications it took me a few days to get everything working end-to-end.  After that small hurdle I was very excited to be seeing combined on-prem and O365 search results in both on-prem and O365.  Do note though the section above calling out the current issue with search analytics data not being sent back and forth.  Aside from that I am looking forward to testing this out with customers and reaping the many benefits such as inclusion of content in the Microsoft Graph (formerly Office Graph) / Delve and other O365 only offerings.

 

      -Frog Out