Using Microsoft Flow to Start and Stop a Set of Azure VMs

   In this blog post I’ll walk through creating a Microsoft Flow flow for starting (and another for stopping) a set of Azure Resource Manager (ARM) VMs.  Note that this is not my own original work.  I implemented this based on the work of someone else I found online but can no longer find the original owner’s reference.  If you do find this elsewhere please feel free to let me know in the comments.

Background

   While it is possible to start and stop Azure VMs from the newly released Azure mobile app, most time I need to start up a set (3-5) VMs at a time for a SharePoint farm / app dev environment / etc.  I was able to find a sample someone wrote in Microsoft Flow to trigger the start / stop from the Flow mobile app.  The flow calls Azure AD to get an access token using an Azure AD app that has permissions to start / stop VMs.  The access token is then passed into a series of REST calls to start up VMs in order (usually domain controller, database server, app server, web front end, etc.)  Finally the flow will send a mobile push notification letting me know that the VMs have started.

Word of caution

   This solution embeds the client ID and client secret (essentially user name and password) for the Azure AD app which has permissions to the Azure VM.   This could be a security risk and as such should be cautioned from doing this.  Treat this sample as a proof of capability for development purposes only.  I’m continuing to explore alternatives (ex. Managed Service Identity, Azure connector in Microsoft Flow) which would increase security for this solution.  If anyone has any suggestions please feel free to let me know in the comments.

Solution – Start Azure VMs

   I won’t go into detail on each and every step as some of these are self explanatory or a repeat of others (ex. 2nd and 3rd VM to be started.)  Before going into the flow to be created, ensure you have an Azure AD app registered with permissions on the desired VMs to be started / stopped.

Register Azure AD App

   Log into the “new” Azure portal (portal.azure.com) and go into the Azure AD screen.  First click on Properties to view the directory ID.  Make note of this for future use.

image

   Click App registrations and create a new app of type “Web app / API”.

image

   Make note of the application ID (also known as client ID).

image

   Go into the Required Permissions setting for the app.  Add a permission for the “Windows Azure Service Management API”.  Choose the permission “Access Azure Service Management as organization users” which is currently in preview.

   Create a key for the Azure AD app and write this down.  You will only get to see this key once and cannot retrieve it at a later time.  If you lose the key value you will need to create a new one.

Assign access control to resource group

   Now that the Azure AD App has been registered it will need access control to the resource group (or individual Azure VMs, more administration if this option) so that the app can start / stop the desired VMs.  I granted Virtual Machine Contributor role to the Azure AD App but more fine grained controls might be possible if security concerns are a factor.

image

Microsoft Flow sample

  1. Manually trigger a flow
  2. Get access token for Azure
  3. Parse JSON to extract access token
  4. Start VMs (in series)
  5. Push notification if successful

image

Manually trigger a flow

   This is self explanatory.  This will let you initiate the flow from Flow web portal or the Flow mobile app.

Get access token for Azure

   This step will use an HTTP POST action to the Azure AD directory where the Azure AD app is registered.  Ideally you should send a request to this URI using Postman or a similar REST endpoint testing tool to get a sample of the JSON response to be used in the following step.

image

  • Method: POST
  • Uri: https://login.microsoftonline.com/<directoryID from previous step>/oauth2/token
  • Headers
    • Content-Type: application/x-www-form-urlencoded
  • Body: resource=https://management.azure.com/&client_id=<client ID from previous step>&grant_type=client_credentials&client_secret=<client secret from previous step>

Example JSON response using Postman:

{
   “token_type”: “Bearer”,
   “expires_in”: “3599”,
   “ext_expires_in”: “0”,
   “expires_on”: “1508115492”,
   “not_before”: “1508111592”,
   “resource”: “https://management.azure.com/”,
   “access_token”: “<removed value>”
}

Parse JSON

   Either using the sample JSON response above or your own you can define the schema of the JSON to be parsed.  Specify the “Body” of the JSON response from the prior HTTP POST action.  The important element to parse out is “access_token”.

image

{

    “type”: “object”,

    “properties”: {

        …<other properties here>…,

        “access_token”: {

            “type”: “string”

        }

    }

}

Start VM REST call

   Add another HTTP POST action this time specifying the following configuration.

image

  • Method: POST
  • Uri: https://management.azure.com/subscriptions/<Azure subscription ID>/resourceGroups/<resource group name>/providers/Microsoft.Compute/virtualMachines/<Azure VM name>/start?api-version=2016-04-30-preview
  • Headers
    • Authorization: Bearer <insert the bearer token “input” from prior Parse JSON step>

   Note that I used an older version for the “api-version=” portion of query string (highlighted in green).  A newer version might also be available and compatible but I haven’t tested anything newer.

   Create as many additional HTTP POST actions that call off to additional VMs as needed.  I hand coded the Uri for each as Microsoft Flow didn’t yet support expressions and other dynamic variables when this solution was first created.  You may want to investigate those to reduce repeated syntax if possible.

Notify when VMs started

   Straight forward action with a simple notification to let me know when flow has completed.

image

Solution – Stop Azure VMs

   The steps for stopping a set of Azure VMs will be identical to the “start” flow except that stopping VMs can be done in parallel as the order is not as important.  In your own scenario the order may be important so consider that when creating your own solution.

  1. Manually trigger a flow
  2. Get Access Token for Azure
  3. Parse JSON to extract access token
  4. Stop VMs (in parallel)
  5. Push notification if successful

image

   The other important difference will be to call to “deallocate” (highlighted in red) the VM rather than “start” using the Azure Service Management API.  See example below for the HTTP POST to a VM.

Sample Execution

   As you can see from the below sample executions of both flows the start and stop of each VM can take some time (2-3 minutes) but is still an easier process of clicking one button rather than multiple clicks within the Azure Portal or mobile app.

image

image

Conclusion

   Hopefully this walkthrough will help others who are interested in automating Azure VMs to start and stop (or any other authenticated actions against Azure resources).  I’m hoping to try out additional options to remove the need to store client ID and secret within the flow.  For the time being try out this process and let me know if you have any issues.

      -Frog Out

Controlling Office 365 Admin Access with Azure AD Privileged Identity Management (PIM)

   Controlling, monitoring, and revoking access to privileged accounts can be a difficult process.  Recently my coworker Ken Kilty shared with me a new service for Azure Active Directory called Privileged Identity Management (Azure AD PIM).  After spending some time with it I wanted to share with a broader audience since I had never heard of it previously.

image

 

Overview

   Please read the What is Azure AD Privileged Identity Management first for a good overview of implementation, example scenario, and additional links to resources.  Note that Azure AD PIM requires Azure AD Premium P2 licenses.  If you would like to test this out there is a free 30 day trial of Azure AD Premium P2 for up to 100 users.

   Granting administrator access, for any application or server, to users should always be done with caution.  Sometimes what starts out as a temporary elevation of permissions turns into a permanent assignment.  Azure AD PIM answers many of the tough questions for Azure AD, Office 365, and related services such as:

  • Who has admin access to <service X>?
  • How do I grant truly temporary access to <service Y>?
  • How can I review all current admins to see if they still need admin access?

   The goal with Azure AD PIM is to allow administrators to define either permanent or “eligible” assignment of specific elevated permissions within Azure and Office 365.  Currently there are 21 roles that can be managed such as Global Administrator, Password Administrator, SharePoint Service Administrator, Exchange Administrator, and more.  See Assigning administrator roles in Azure Active Directory for a more complete listing of roles.  Users who are defined as “eligible” will be able to elevate themselves to roles they have been assigned for a set number of hours (1-72) defined by a Azure AD PIM administrator.  During this role elevation process the “eligible” user will need to verify their identity through a text / call verification or multifactor authentication (MFA) mechanism.  One of the key advantages is that this entire interaction is tracked and auditable.  Administrators can even require an incident or service ticket number prior to elevation and receive alerts when elevation requests are processed.

 

Conclusion

   I have seen privileged role access handled in many different ways at customers over the years.  Having a consistent and auditable process ensures that changes can be tracked and users who no longer need elevated permissions can be removed.  In the time I’ve tested out Azure AD Privileged Identity Management I am very happy with the overall process and review options.  One word of advice for users elevating yourself.  You will need to log out and log back in in order to update your claim token with the new elevated role claims.  Give Azure Active Directory Privileged Identity Management a try and share any feedback in the comments below.

 

      -Frog Out

Slides and Demo Scripts from SharePoint Saturday Twin Cities 2016

A big thank you out to the organizers, attendees, and sponsors from SharePoint Saturday Twin Cities.  It has been a number of years since I was last at this event and great to see it alive and well.  I had a great time getting to talk with attendees and fellow speakers as well as presenting two sessions.  Slides, code, and scripts from my sessions are below.  If you have any feedback or follow up questions please leave a comment below.

 

On a personal note I was completely flattered when Jeff Teper (“father of SharePoint” himself) and James Phillips (CVP for Business Apps at Microsoft) tweeted back to me during my prep time this morning.  Great to see the Corporate Vice President or SharePoint and OneDrive engaged and supporting community events.  A sign of good things to come from the SharePoint and PowerApps leadership.

 

image

 

PowerApps and Microsoft Flow Intro for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

 

Running Your Azure Dev / Test VMs for Cheap

PowerShell script for creating new Azure RM VM

Slides

 

-Frog Out

Slides and Demo Scripts from Dog Food Conference 2016

   A big thank you out to the organizers, attendees, and sponsors from Dog Food Conference.  I had a great time getting to talk with attendees and fellow speakers as well as presenting two sessions.  Slides, code, and scripts from my sessions are below.  If you have any feedback or follow up questions please leave a comment below.

 

PowerApps and Microsoft Flow Intro for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

 

Running Your Azure Dev / Test VMs for Cheap

PowerShell script for creating new Azure RM VM

Slides

 

      -Frog Out

Speaking at Central Ohio Azure User Group June 2016

Next Monday, June 13th I am excited to be presenting at the Central Ohio Azure user group (COAzure). This will be my first time speaking at this group.  Session details are below.  Meeting is from 6:00pm-8:00pm at the Microsoft building in Columbus near Polaris mall.  Come on out and learn more about Azure VM.

 

Session

Title: Running Your Dev/Test Virtual Machines in Azure for Cheap

Abstract: With an MSDN subscription you can run your dev / test SharePoint environment in Azure IaaS for less than the cost of a cup of coffee each day. In this session we will overview the basics of Azure IaaS (Infrastructure as a Service), the pieces you will use to be successful deploying SharePoint in Azure (including the new Azure Resource Manager templates), and how to use resources as efficiently as possible to reduce your costs and boost your farm performance. This session is targeted to SharePoint developers and administrators. Prior knowledge of Azure is helpful but not a requirement.

 

 

-Frog Out

Start Learning About PowerApps

   In this post I’ll talk about PowerApps, a new enterprise service for building enterprise applications and share resources on where to find out more information.

   Note: PowerApps is currently in private preview and is subject to change after this article is posted.  As such this article may contain out of date information by the time you read this.  Additionally I am a Microsoft employee but the views and opinions expressed in this article are my own and not reflective of Microsoft or the PowerApps product group.

 

Background

   On Nov 30th, 2015 at the European Convergence conference Microsoft unveiled a new enterprise service for building enterprise applications called PowerApps.  At a high level PowerApps allows power users and developers to build scalable applications that connect to numerous services (Office 365, SalesForce, OneDrive, Dropbox, etc.) using PowerPoint and Excel-like tools to be consumed on Windows, iOS, and Android.  These applications can be built once and then consumed on any platform.  No need to re-compile, design separate UIs per platform, etc. like you see with the current state of most mobile or web development.

 

Components

   I’ll briefly walk through some of the highlights for different components or aspects of which to be aware.

 

Tools and client player

   Currently it is possible to create and consume PowerApps apps on Windows and iOS.  I can’t speak to the final plans but it is my understanding that it is on the roadmap to be able to create and consume on all platforms including Windows (PC and mobile), iOS, Android, and web.  You will not be limited to only consuming from the platform that you created on though.

 

Design language

   PowerApps is designed to be able to author apps using Excel and PowerPoint type skills.  There is no need to code your solution.  That said if you are a developer and wish to code backend interactions or create a custom API to connect to that is available (with the Enterprise plan, more on that below).

 

Connectors

   Out of the box PowerApps ships with a dozen or so connectors for pulling or pushing data to the following sources.  By configuring a connector to these services you can perform simple CRUD (create, read, update, and delete) operations on data in these sources.

  • Dropbox
  • Dynamics CRM Online
  • Google Drive
  • Microsoft Translator
  • Office 365 Outlook
  • Office 365 Users
  • OneDrive [consumer version]
  • Salesforce
  • SharePoint Online
  • Twitter

   Establishing a connection to these services is as simple as logging into the service.  Once you establish a connection it is persisted to the PowerApps cloud and will be available on any device that you log into your account.

 

Authentication

   Speaking of logging into accounts, authentication for PowerApps is handled by Azure Active Directory.  As such you will need to have an Azure Active Directory identity / domain in order to utilize PowerApps.  Thus you can view PowerApps as an enterprise solution more than a consumer solution even though you do have access to consumed focused connections (e.g. Twitter, OneDrive, etc.).

 

PowerFlows

   PowerFlows (also called Logic Flows) are still a work in progress but the goal is to provide simple yet robust workflows for data.  Think along the lines of If This Then That (IFTTT, www.ifttt.com) which is a popular website for connecting data from disparate sources and taking action when specific triggers are met.  Ex. when the forecast is predicting rain tomorrow send me a text message and put a calendar entry on my calendar to bring an umbrella to work.  IFTTT also integrates with home automation software, smartphone devices, design websites, and more.

   On the PowerFlows side you can define a triggers and then take actions based on that incoming trigger.  Ex. when a new tweet from Twitter contains specific data create a new entry in a SharePoint list, send me an email, and then create a case in Salesforce.  When used in conjunction with apps from PowerApps this can be a powerful complimentary toolset.

 

Sharing

   When it comes time to share your PowerApps app with others you can simply type in their email address and share it with them.  No need to worry about downloading the application, incompatibility of the OS, or other traditional blockers for enterprise applications.  In the enterprise plan it is possible to restrict access to the app so that only specific users are able to view and access your app.

 

Plans

   Speaking of plans there are 3 plan levels.  They are as follows.

  • Free – create and use unlimited apps, 2 connections to SaaS data per user, shared infrastructure
  • Standard – create and use unlimited apps, unlimited connections to SaaS data per user, shared infrastructure
  • Enterprise – create and use unlimited apps, unlimited connections to SaaS data per user, dedicated infrastructure, app governance, API management

   The last piece of the Enterprise plan is interesting to me.  This allows an organization to package up an API to line of business (LOB) or other data (i.e. SQL Server, on-prem SharePoint, etc.) and publish it to Azure.  That data source can then be consumed by PowerApps apps.

 

Sign up for preview

   PowerApps is currently in private preview but I encourage everyone to request an invite to gain access at the following URL.  Note that you may not be accepted in right away but you will be added to the list for future inclusion.

 

Request invite to PowerApps

https://powerapps.microsoft.com/en-us/signup/

 

Conclusion

   I am very excited to see PowerApps finally come to private preview.  I have been following Project Siena (precursor to PowerApps) for over a year now and tinkering around with the alpha and beta builds of both.  There is no release date yet for PowerApps but I encourage you and your organization to sign up for the preview and take a look at the videos and tutorials linked below.

   Lastly a few parting thoughts.  Think of all of the LOB apps that exist in your company or organizations and all of the time, effort, and money that goes into developers and / or designers creating and maintaining those applications.  Many of these applications are simple data entry, approval workflow, or similar type applications.  By exposing enterprise data in a structured and secure manner you can empower end users to create those types of applications much more quickly while freeing up resources and people for other business needs.

 

      -Frog Out

 

Additional Resources

Introducing Microsoft PowerApps
https://powerapps.microsoft.com/en-us/blog/introducing-microsoft-powerapps/

Microsoft PowerApps main site (and registration)
https://powerapps.microsoft.com/en-us/

 

Microsoft PowerApps tutorials

https://powerapps.microsoft.com/en-us/tutorials/

 

Microsoft PowerApps videos on Channel 9 

https://channel9.msdn.com/Events/Microsoft-Azure/PowerApps

 

Microsoft takes the wraps off PowerApps
http://www.zdnet.com/article/microsoft-takes-wraps-off-powerapps-mobile-app-creation-service/

My Experience Configuring Cloud Hybrid Search Service Application for SharePoint

   In this post I’ll talk through my personal experience deploying the new cloud hybrid search service application for SharePoint 2013 (also available in SharePoint 2016).  By no means am I an expert on this topic (especially in many of the supporting technologies such as AAD Connect, AD FS, etc.) but this is more meant to increase exposure to this new offering.  For an overview of cloud hybrid search and more information about actual implementation (which I will refer back to later) please read through Cloud Hybrid Search Service Application written by two of my Microsoft peers Neil and Manas (they are the true experts).

 

Components

   Here is a list of the high level components I used for my deployment.

Note: My Azure VM configuration is not using best practices for where or how to deploy different services.  Also my mention of GoDaddy and DigiCert are purely for example purposes and not an endorsement for either company.  I just happen to use their services and products in this scenario.

  • Office 365 (O365) trial tenant (sign up for one here)
  • 4 Azure VMs
    • A1 – Active Directory Domain Services (AD DS)
    • A1 – Active Directory Federation Services (AD FS)
    • A2 – Azure Active Directory Connect (AAD Connect), Web Application Proxy (WAP)
    • A4 – SQL Server 2014, SharePoint 2013 farm with Service Pack 1 and at least Aug 2015 CU
  • Custom domain (purchased through GoDaddy but any domain registrar should work)
    • Note: Office 365 does have a partnership with GoDaddy so configuration may be easier due to automated updates that can be performed
    • Additionally I was able to modify public DNS records through GoDaddy to allow federated authentication through AD FS
  • SSL wildcard certificate purchased from DigiCert
    • Only required if want to allow Office 365 user to open / preview a search result that resides on-prem with Office Online Server (new version of Office Web Apps Server 2013, not discussed in this post)
    • I also used this certificate for other purposes such as securing AD FS communication and implementing Remote Desktop Gateway (the latter is unrelated to this post)
  • Custom result source to display O365 search results in my on-prem farm

 

   Next we’ll take a look at some of these components more in depth.

 

SharePoint Server

   The new cloud hybrid search service application is available in SharePoint Server 2013 with the August 2015 CU or later.  I have heard from my peers that there are some issues with cloud hybrid search as of the October, November, and December 2015 CUs.  As such use either the August or September 2015 CUs at the time of this writing (Dec 8, 2015) or wait until the Jan 2016 CU which should contain the fix (link).  The SharePoint Server 2016 IT Preview 1 also supports cloud hybrid search although I have not tested it out myself.

 

Cloud Search Service Application

   To provision a cloud hybrid search service application the property CloudIndex on the service application must be set to True.  This property is a read-only property and can only be set at creation time.  As such you will need to create a new search service application in order to utilize the cloud hybrid search service.

   I have not tested creating a new cloud hybrid search service application using a restored backup admin database from an existing search service application.  The thought behind this would be to retain at least a portion of your existing search service application.  If you try this and have any findings let me know in the comments below.

 

Custom Domain

   A custom domain is not a requirement for cloud hybrid search.  I used one so that I could allow end users (demo accounts) to log into Office 365 as a federated user “someUser@<fakecompany>.com” rather than the default domain “someUser@<O365TenantDomain>.onmicrosoft.com”.

 

AAD Connect

   In order to search for on-prem content that has been indexed by Office 365 the user will need to have an account that is synchronized to Azure Active Directory / Office 365.  This allows the search service in Office 365 to show content based on the Access Control List (ACL) defined on-prem.

   There are multiple options available for synchronizing accounts between on-prem and AAD but the predominate ones include DirSync, AAD Sync, and AAD Connect.  Since AAD Connect is the future looking tool of choice of these three I decided to use it.  AAD Connect automates many of the tedious tasks of configuring federated authentication by stepping through a wizard.

   That said I did run into a number of issues during configuration due to missing certificates, invalid permissions, or other steps I missed or was unaware of.  If I got part of the way through the configuration and ran into a failure that I couldn’t recover from then I had to uninstall AAD Connect (do not remove all prerequisites when prompted), wipe out the contents of “<install drive>:Program FilesMicrosoft Azure AD SyncData”, and then re-install.

 

Display Search Results On-Prem

 

***PLEASE READ AS THIS IS IMPORTANT***

    The default scenario for cloud hybrid search is to index both on-prem and O365 content which are then queried in O365.  It is possible to create or modify an on-prem result source to use the remote index from your Office 365 tenant which allows for querying and display the combined search results on-prem.  The problem though is that when you query for and click results on-prem the search analytics click data is not incorporated back to the cloud index to further influence search results.

Ex. I queried for “SharePoint” in on-prem search center and clicked the 4th result on result page.  Multiple other users also searched for “SharePoint” and clicked the same 4th result.  SharePoint search (via timer jobs and other background processes) incorporates that click data and adjusts the 4th result to now appear higher in rankings upon subsequent search queries.

   I have unsuccessfully tested a few options to manually pass the search click data up to SharePoint Online.  These include creating a ClientContext object and calling the RecordPageClick() method on SearchExecutor, modifying the display template page, and more.  I did hear from a SharePoint MVP that successfully tested out a way to push search analytics data between on-prem and O365 but it took a fair amount of customizations to accomplish.  If I find out any additional information, workaround, or updates on this topic I’ll update this post to reflect that.

 

Example

   As you can see from the below screenshots I can initiate a search query from on-prem or O365 (respectively) and get the same combined result set.

 

OnPremResults

 

SPOResults

 

 

Conclusion

   Due to my prior inexperience around AD FS, Web Application Proxy, AAD Connect, and other applications it took me a few days to get everything working end-to-end.  After that small hurdle I was very excited to be seeing combined on-prem and O365 search results in both on-prem and O365.  Do note though the section above calling out the current issue with search analytics data not being sent back and forth.  Aside from that I am looking forward to testing this out with customers and reaping the many benefits such as inclusion of content in the Microsoft Graph (formerly Office Graph) / Delve and other O365 only offerings.

 

      -Frog Out