Using Microsoft Flow to Start and Stop a Set of Azure VMs

   In this blog post I’ll walk through creating a Microsoft Flow flow for starting (and another for stopping) a set of Azure Resource Manager (ARM) VMs.  Note that this is not my own original work.  I implemented this based on the work of someone else I found online but can no longer find the original owner’s reference.  If you do find this elsewhere please feel free to let me know in the comments.

Background

   While it is possible to start and stop Azure VMs from the newly released Azure mobile app, most time I need to start up a set (3-5) VMs at a time for a SharePoint farm / app dev environment / etc.  I was able to find a sample someone wrote in Microsoft Flow to trigger the start / stop from the Flow mobile app.  The flow calls Azure AD to get an access token using an Azure AD app that has permissions to start / stop VMs.  The access token is then passed into a series of REST calls to start up VMs in order (usually domain controller, database server, app server, web front end, etc.)  Finally the flow will send a mobile push notification letting me know that the VMs have started.

Word of caution

   This solution embeds the client ID and client secret (essentially user name and password) for the Azure AD app which has permissions to the Azure VM.   This could be a security risk and as such should be cautioned from doing this.  Treat this sample as a proof of capability for development purposes only.  I’m continuing to explore alternatives (ex. Managed Service Identity, Azure connector in Microsoft Flow) which would increase security for this solution.  If anyone has any suggestions please feel free to let me know in the comments.

Solution – Start Azure VMs

   I won’t go into detail on each and every step as some of these are self explanatory or a repeat of others (ex. 2nd and 3rd VM to be started.)  Before going into the flow to be created, ensure you have an Azure AD app registered with permissions on the desired VMs to be started / stopped.

Register Azure AD App

   Log into the “new” Azure portal (portal.azure.com) and go into the Azure AD screen.  First click on Properties to view the directory ID.  Make note of this for future use.

image

   Click App registrations and create a new app of type “Web app / API”.

image

   Make note of the application ID (also known as client ID).

image

   Go into the Required Permissions setting for the app.  Add a permission for the “Windows Azure Service Management API”.  Choose the permission “Access Azure Service Management as organization users” which is currently in preview.

   Create a key for the Azure AD app and write this down.  You will only get to see this key once and cannot retrieve it at a later time.  If you lose the key value you will need to create a new one.

Assign access control to resource group

   Now that the Azure AD App has been registered it will need access control to the resource group (or individual Azure VMs, more administration if this option) so that the app can start / stop the desired VMs.  I granted Virtual Machine Contributor role to the Azure AD App but more fine grained controls might be possible if security concerns are a factor.

image

Microsoft Flow sample

  1. Manually trigger a flow
  2. Get access token for Azure
  3. Parse JSON to extract access token
  4. Start VMs (in series)
  5. Push notification if successful

image

Manually trigger a flow

   This is self explanatory.  This will let you initiate the flow from Flow web portal or the Flow mobile app.

Get access token for Azure

   This step will use an HTTP POST action to the Azure AD directory where the Azure AD app is registered.  Ideally you should send a request to this URI using Postman or a similar REST endpoint testing tool to get a sample of the JSON response to be used in the following step.

image

  • Method: POST
  • Uri: https://login.microsoftonline.com/<directoryID from previous step>/oauth2/token
  • Headers
    • Content-Type: application/x-www-form-urlencoded
  • Body: resource=https://management.azure.com/&client_id=<client ID from previous step>&grant_type=client_credentials&client_secret=<client secret from previous step>

Example JSON response using Postman:

{
   “token_type”: “Bearer”,
   “expires_in”: “3599”,
   “ext_expires_in”: “0”,
   “expires_on”: “1508115492”,
   “not_before”: “1508111592”,
   “resource”: “https://management.azure.com/”,
   “access_token”: “<removed value>”
}

Parse JSON

   Either using the sample JSON response above or your own you can define the schema of the JSON to be parsed.  Specify the “Body” of the JSON response from the prior HTTP POST action.  The important element to parse out is “access_token”.

image

{

    “type”: “object”,

    “properties”: {

        …<other properties here>…,

        “access_token”: {

            “type”: “string”

        }

    }

}

Start VM REST call

   Add another HTTP POST action this time specifying the following configuration.

image

  • Method: POST
  • Uri: https://management.azure.com/subscriptions/<Azure subscription ID>/resourceGroups/<resource group name>/providers/Microsoft.Compute/virtualMachines/<Azure VM name>/start?api-version=2016-04-30-preview
  • Headers
    • Authorization: Bearer <insert the bearer token “input” from prior Parse JSON step>

   Note that I used an older version for the “api-version=” portion of query string (highlighted in green).  A newer version might also be available and compatible but I haven’t tested anything newer.

   Create as many additional HTTP POST actions that call off to additional VMs as needed.  I hand coded the Uri for each as Microsoft Flow didn’t yet support expressions and other dynamic variables when this solution was first created.  You may want to investigate those to reduce repeated syntax if possible.

Notify when VMs started

   Straight forward action with a simple notification to let me know when flow has completed.

image

Solution – Stop Azure VMs

   The steps for stopping a set of Azure VMs will be identical to the “start” flow except that stopping VMs can be done in parallel as the order is not as important.  In your own scenario the order may be important so consider that when creating your own solution.

  1. Manually trigger a flow
  2. Get Access Token for Azure
  3. Parse JSON to extract access token
  4. Stop VMs (in parallel)
  5. Push notification if successful

image

   The other important difference will be to call to “deallocate” (highlighted in red) the VM rather than “start” using the Azure Service Management API.  See example below for the HTTP POST to a VM.

Sample Execution

   As you can see from the below sample executions of both flows the start and stop of each VM can take some time (2-3 minutes) but is still an easier process of clicking one button rather than multiple clicks within the Azure Portal or mobile app.

image

image

Conclusion

   Hopefully this walkthrough will help others who are interested in automating Azure VMs to start and stop (or any other authenticated actions against Azure resources).  I’m hoping to try out additional options to remove the need to store client ID and secret within the flow.  For the time being try out this process and let me know if you have any issues.

      -Frog Out

Slides and Demo Files from Dogfood Con 2017

   A big thanks to Cassandra, Trey, Danillo, and all of the other organizers of Dogfood Con 2017.  Thanks also to all of the folks who attended my two sessions.  Below are my slides and source code.  Feel free to let me know about any follow up questions or comments.

PowerApps and Microsoft Flow for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

Intro to Power BI for Office 365 Developers

Slides

Sample Financial Data file

http://go.microsoft.com/fwlink/?LinkID=521962

Blog posts on SMAT data report

https://aka.ms/SMAT2013BTJpart1

https://aka.ms/SMAT2013BTJpart2

-Frog Out

SharePoint CSOM to Traverse All Sites in SharePoint Online

   In the past I’ve written posts for “PowerShell to Enumerate SharePoint 2010 or SharePoint 2013 Permissions” or “PowerShell Script To Traverse All Sites In SharePoint 2010 (or 2007) Farm” to assist with traversing through all sites within a SharePoint on-prem farm.  In this post I’ll share a snippet I recently used for traversing through all site collections and subsites within a SharePoint Online tenant.

Background

   If you’ve worked with the SharePoint Online Management Shell you may know that originally it was not able to retrieve Personal Sites (also known as My Sites / OneDrive for Business sites) in SharePoint Online.  As far as I’m aware this was primarily a limitation of the underlying client side libraries (Microsoft.SharePoint.Client.*).  Fast forward to a recent release (I don’t have the specific one but I can confirm it is in the 16.1.6621.1200 release of the Microsoft.SharePointOnline.CSOM NuGet package) and now it is supported to retrieve Personal Sites.  In the management shell this is accomplished by calling the following:

Connect-SPOService -Url ‘<tenantAdminUrl>’

Get-SPOSite -Limit all -IncludePersonalSite $true

   The problem is that if you want to traverse all site collections in SharePoint Online in client side object model (CSOM) you would need to know how the SharePoint Online Management Shell implements that inclusion of Personal Sites with the rest of site collections.  In order to find this out I used a disassembler (ILSPY in my case, but there are many alternatives available as well) on the underlying libraries to recreate the process in my own code.

Solution

   The resulting CSOM that I came up with is below.  The yellow code is processing all site collections within SharePoint Online while the green code is processing sites (subsites / webs / “not site collections”).  Feel free to borrow this and use in your own code but note that it is provided as-is with no warranty.

SPOSitePropertiesEnumerable ssp = null;

SPOSitePropertiesEnumerableFilter sspFilter = new SPOSitePropertiesEnumerableFilter();

SharePointOnlineCredentials creds = new SharePointOnlineCredentials(“myUsernameGoesHere”, securePassword);

using (PnPClientContext cc = new PnPClientContext(“myURLGoesHere”))

{

    cc.Credentials = creds;

   Tenant tenant = new Tenant(cc);

    //loop through all site collections including personal sites (even though not being used)

    //borrowed this code from after decompiling SPO Management Shell assemblies

    sspFilter.IncludePersonalSite = PersonalSiteFilter.Include;

    sspFilter.IncludeDetail = true;

    sspFilter.StartIndex = null;

    ssp = tenant.GetSitePropertiesFromSharePointByFilters(sspFilter);

    cc.Load(ssp);

    cc.ExecuteQuery();

    foreach (SiteProperties sp in ssp)

    {

        //DO YOUR WORK HERE FOR EACH SITE COLLECTION, such as looping through subwebs

        cc.Load(cc.Web, w => w.NoCrawl,

                                    w => w.Webs,

                                    w => w.Url);

        cc.ExecuteQuery();

        //check subweb(s)

        foreach (var subweb in cc.Web.Webs)

        {

            cc.Load(subweb, sw => sw.NoCrawl,

                                        sw => sw.Url);

            cc.ExecuteQuery();

        }

    }

}

Conclusion

   In this post I shared a snippet for traversing all site collections in SharePoint Online with C# CSOM code.  In my daily job I’ve been using this in combination with Azure Functions for a number of interesting interactions with a SharePoint Online tenant.  More to come on those scenarios in future weeks.  For now let me know in the comments if you have any questions or issues implementing the above snippet in your own code.  Happy coding!

      -Frog Out

Slides and Demo Files from CincyDevelop() Conference

   A big thanks to Phil and all of the other organizers of Cincy.Develop() / Day of Agile Conference.  Thanks also to all of the folks who attended my Integrate All the Things: PowerApps and Flow for Developers session.  Below are my slides and source code.  Feel free to let me know about any follow up questions or comments.

PowerApps and Microsoft Flow for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

-Frog Out

Upcoming Speaking Events in 2017

I’m honored to be accepted to speak at the following upcoming events.  Here are the abstracts for these presentations.  If you are at any of these events feel free to stop by and chat.  Registration is still open as of the time of writing as well.

 

Cincy.Develop()

Website: Cincy Day of Agile and Cincy.Develop()

When: Fri. Jul 28, 2:40pm-3:40pm

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

DogFood Conference

Website: DogFoodCon

When: Oct 5-6, timeslots TBA

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

Title: Intro to Power BI for Office 365 Developers

Abstract: “Power BI is SSRS in the cloud, right?” “I’ll throw a few pie charts on the screen and call it a dashboard, right?” Wrong. In this session we will introduce Office 365 developers to the Power BI service and Power BI Desktop. We will cover data sources that can be used (Azure SQL, SharePoint, files, etc.), direct query vs. imported dataset, report and dashboard creation, and the on-prem data gateway. We will also break down misconceptions of good dashboard design (hint: pie charts and 3D are almost always a bad design). Prior experience with Power BI or Power BI desktop are not required.

 

SPTechCon DC

Website: SPTechCon DC

When: Nov 12-15, timeslots TBA

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

Title: Intro to Power BI for Office 365 Developers

Abstract: “Power BI is SSRS in the cloud, right?” “I’ll throw a few pie charts on the screen and call it a dashboard, right?” Wrong. In this session we will introduce Office 365 developers to the Power BI service and Power BI Desktop. We will cover data sources that can be used (Azure SQL, SharePoint, files, etc.), direct query vs. imported dataset, report and dashboard creation, and the on-prem data gateway. We will also break down misconceptions of good dashboard design (hint: pie charts and 3D are almost always a bad design). Prior experience with Power BI or Power BI desktop are not required.

 

      -Frog Out

Controlling Office 365 Admin Access with Azure AD Privileged Identity Management (PIM)

   Controlling, monitoring, and revoking access to privileged accounts can be a difficult process.  Recently my coworker Ken Kilty shared with me a new service for Azure Active Directory called Privileged Identity Management (Azure AD PIM).  After spending some time with it I wanted to share with a broader audience since I had never heard of it previously.

image

 

Overview

   Please read the What is Azure AD Privileged Identity Management first for a good overview of implementation, example scenario, and additional links to resources.  Note that Azure AD PIM requires Azure AD Premium P2 licenses.  If you would like to test this out there is a free 30 day trial of Azure AD Premium P2 for up to 100 users.

   Granting administrator access, for any application or server, to users should always be done with caution.  Sometimes what starts out as a temporary elevation of permissions turns into a permanent assignment.  Azure AD PIM answers many of the tough questions for Azure AD, Office 365, and related services such as:

  • Who has admin access to <service X>?
  • How do I grant truly temporary access to <service Y>?
  • How can I review all current admins to see if they still need admin access?

   The goal with Azure AD PIM is to allow administrators to define either permanent or “eligible” assignment of specific elevated permissions within Azure and Office 365.  Currently there are 21 roles that can be managed such as Global Administrator, Password Administrator, SharePoint Service Administrator, Exchange Administrator, and more.  See Assigning administrator roles in Azure Active Directory for a more complete listing of roles.  Users who are defined as “eligible” will be able to elevate themselves to roles they have been assigned for a set number of hours (1-72) defined by a Azure AD PIM administrator.  During this role elevation process the “eligible” user will need to verify their identity through a text / call verification or multifactor authentication (MFA) mechanism.  One of the key advantages is that this entire interaction is tracked and auditable.  Administrators can even require an incident or service ticket number prior to elevation and receive alerts when elevation requests are processed.

 

Conclusion

   I have seen privileged role access handled in many different ways at customers over the years.  Having a consistent and auditable process ensures that changes can be tracked and users who no longer need elevated permissions can be removed.  In the time I’ve tested out Azure AD Privileged Identity Management I am very happy with the overall process and review options.  One word of advice for users elevating yourself.  You will need to log out and log back in in order to update your claim token with the new elevated role claims.  Give Azure Active Directory Privileged Identity Management a try and share any feedback in the comments below.

 

      -Frog Out

Guest Blog Post on PowerApps Blog – Social Media Sample

   I wrote a guest blog post Introducing Brian T Jackett: PowerApps and Microsoft Flow Social Media Sample for the official PowerApps blog.  I also provided the social media sample demo code along with a video walkthrough of the demo.  Special thanks to Pratap Ladhani and Audrie Gordon for getting this published.  Please read and leave any feedback or questions in the comments on that post.

 

Introducing Brian T Jackett: PowerApps and Microsoft Flow Social Media Sample

https://powerapps.microsoft.com/en-us/blog/introducing-brian-t-jackett-powerapps-and-microsoft-flow-social-media-sample/

 

Background

    Over the past 6 months I’ve given a number of external and internal presentations on PowerApps and Flow for developers.  One of the highlights of this presentation was an interactive demo integrating a variety of cloud services such as Twitter, Azure SQL, Azure Web App, Azure API App, Microsoft Flow, PowerApps, and Custom APIs.  Attendees were able to use a hashtag on Twitter that then fed into the demo and we would track the data through multiple platforms to present the data back out.  After giving this presentation internally Pratap Ladhani (Principal Program Manager for PowerApps and Flow teams) asked me to share out this demo.  This guest blog post is the result of that.

 

 

      -Frog Out