Automate Creation of Azure AD Application with OAuth Permissions

<Update 2018-02-06>Updated with snippet to list out GUIDs for app roles that can be assigned.</Update>

In this post I will show how to automate the creation of an Azure AD Application and assign OAuth permissions to that application.  The latter part is tricky as there is not currently a PowerShell commandlet or Azure CLI command to assign OAuth permissions.  Instead we will leverage an authenticated call to the Microsoft Graph to assign the permissions.  For more in depth information about Azure AD apps, verifying the results, and more please see the following post which I am borrowing heavily from.  I had difficulty finding this information so this post is my attempt to spread the word and also add a few clarifications on the ADAL libraries used.

(Read this first!) Automating the creation of Azure AD Applications by Christer Ljung

http://www.redbaronofazure.com/?p=7197

Problem

Creating Azure AD apps typically involves logging into the Azure Portal (classic or “new” / Ibiza version) and manually clicking through multiple screens.  When developing a solution that needs to leverage Office 365 services (as is my case with a current project) it is helpful to automate the process of creating the Azure AD app and assigning the permissions.  If you happen to be assigning Admin permissions then additional steps will be required by an Azure AD domain administrator (see following screenshot).

AzureADApp1

Solution

Creating an Azure AD application can be accomplished in 2 lines of PowerShell.  Login to Azure then create the app.


Login-AzureRmAccount

$aadapp = New-AzureRmADApplication -DisplayName "Some amazing app" -HomePage https://localhost:8081/ -IdentifierUris https://localhost:8081/

***BONUS***

If you want to create an app that uses certificate based authentication you can use the following PowerShell commandlets.

Note: The commandlets for creating and exporting a certificate require Windows 8 or higher.  There are workarounds for Windows 7 or similar OS.  Feel free to reach out if you are in that scenario.


$pwd = Read-Host -AsSecureString -Prompt "Enter certificate password"

# process for Windows 8+ type OS
$ssc = New-SelfSignedCertificate -CertStoreLocation cert:\localmachine\my -Provider "Microsoft Enhanced RSA and AES Cryptographic Provider" `
                           -Subject "cn=MySuperSpecialCert" -KeyDescription "Used to access Azure Resources" `
                           -NotBefore (Get-Date).AddDays(-1) -NotAfter (Get-Date).AddYears(1)

# Export cert to PFX - uploaded to Azure App Service
Export-PfxCertificate -cert cert:\localMachine\my\$($ssc.Thumbprint) -FilePath ExportedSpecialCertFile.pfx -Password $pwd –Force
$KeyStorageFlags = [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::MachineKeySet, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::PersistKeySet
$certFile = Get-ChildItem –Path <path to certificate file>
$x509 = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$x509.Import($certFile.FullName, $pwd, $KeyStorageFlags)
$certValue = [System.Convert]::ToBase64String($x509.GetRawCertData())

# should match our certificate entries above.
$validFrom = [System.DateTime]::Now.AddDays(-1)
$validTo = [System.DateTime]::Now.AddYears(1)

$aadapp = New-AzureRmADApplication –DisplayName "Some amazing app" -HomePage "https://localhost:8080/" `
                                   -IdentifierUris "https://localhost:8080/" -CertValue $certValue `
                                   -StartDate $validFrom -EndDate $validTo

The next step involves granting OAuth permissions to the recently created Azure AD app.  As of the writing of this blog (Feb 2, 2018) there is not a PowerShell commandlet nor Azure CLI command to assign those permissions.  There is however a way to use the Microsoft Graph to assign permissions.  This is an adapted version of Christer’s example that I referenced earlier and uses a local version of the Active Directory Authentication Library (ADAL) DLLs.  Currently these are at version 3.19.1.

ADAL NuGet package

https://www.nuget.org/packages/Microsoft.IdentityModel.Clients.ActiveDirectory/

Extract the following DLLs into the folder where you are executing other PowerShell commands:

  • Microsoft.IdentityModel.Clients.ActiveDirectory.dll
  • Microsoft.IdentityModel.Clients.ActiveDirectory.Platform.dll

$Tenant = "<Office 365 tenant name, ex. Contoso>"
$aadTenant = "$Tenant.onmicrosoft.com"
$adminUser = "<admin account with access to authenticate against MS Graph>"

# load ADAL DLLs
$adal = ".\Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
$adalforms = ".\Microsoft.IdentityModel.Clients.ActiveDirectory.Platform.dll"

[System.Reflection.Assembly]::LoadFrom($adal) | Out-Null
[System.Reflection.Assembly]::LoadFrom($adalforms) | Out-Null

 $clientId = "1950a258-227b-4e31-a9cf-717495945fc2"  # Set well-known client ID for AzurePowerShell
  $redirectUri = "urn:ietf:wg:oauth:2.0:oob" # Set redirect URI for Azure PowerShell
  $resourceAppIdURI = "https://graph.windows.net/" # resource we want to use
  $adminUserId = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.UserIdentifier" -ArgumentList ($adminUser, "OptionalDisplayableId")

 # Create Authentication Context tied to Azure AD Tenant
  $authContext = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext" -ArgumentList $authority

  # Acquire token
  $authResult = $authContext.AcquireToken($resourceAppIdURI, $clientId, [Uri]$redirectUri, [Microsoft.IdentityModel.Clients.ActiveDirectory.PromptBehavior]::Always, $adminUserId)

 $authHeader = $authResult.CreateAuthorizationHeader()
  $headers = @{"Authorization" = $authHeader; "Content-Type"="application/json"}   

# make call against MS Graph to apply OAuth permissions
$url = "https://graph.windows.net/$aadTenant/applications/$($aadapp.ObjectID)?api-version=1.6"
$postData = "{'requiredResourceAccess':[
     {'resourceAppId':'00000003-0000-0ff1-ce00-000000000000','resourceAccess':[{'id':'fbcd29d2-fcca-4405-aded-518d457caae4','type':'Role'}]},
     {'resourceAppId':'00000002-0000-0000-c000-000000000000','resourceAccess':[{'id':'311a71cc-e848-46a1-bdf8-97ff7156d8e6','type':'Scope'}]}
     ]}";
$result = Invoke-RestMethod -Uri $url -Method "PATCH" -Headers $headers -Body $postData  

Note the use of specific resoureAppId and resourceAccess values above.  These two examples grant the “read and write all items in SharePoint Online” admin consent permission and the default “read user profile data” delegated permission respectively.  In order to find out the GUIDs you may need you’ll need to add the permissions through the Azure portal UI, check the manifest file, and extract the GUIDs.  See Christer’s post for more details.

<Update 2018-02-06>  I recently found out it is possible to list out the Application role permissions and GUIDs needed above by running the following PowerShell against the Azure AD module (I’m using Azure AD “V2” Preview module, haven’t verified against the existing V1 module).

Connect-AzureAD
# 00000003-0000-0ff1-ce00-000000000000 is the AppId for SharePoint Online, call Get-AzureADServicePrincipal by itself to find other AppIds
$SPOApi = Get-AzureADServicePrincipal -Filter "AppId eq '00000003-0000-0ff1-ce00-000000000000'"
$SPOApi.AppRoles

</Update>

If you happen to assign an admin consent permissions (such as the “read and write all items in SharePoint Online” permission) an Azure AD domain administrator will still need to consent to that permission by clicking “Grant permission” inside the Azure portal.  I’m not aware of a way to automate that process but if you do know please share in the comments below.

Conclusion

Originally I had hoped automating creation of an Azure AD app would be a simple process.  Creation of the Azure AD app is easy, but adding certificate authentication and / or assigning OAuth permissions adds extra work to be done.  As seen in this post though much of that can be automated.  Hopefully this post saves you time and effort.  Feel free to leave any feedback or questions in the comments below.

-Frog Out

Slides and Demo Files from SPTechCon DC 2017

   Big thanks to Stacy and her crew of organizers, all of the attendees, and fellow speakers at SPTechCon DC 2017.  This was one of the best ones I’ve attended from an engagement and networking perspective.  Below are my slides and source code.  Feel free to let me know about any follow up questions or comments.

PowerApps and Microsoft Flow for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

Intro to Power BI for Office 365 Developers

Slides

Sample Financial Data file

http://go.microsoft.com/fwlink/?LinkID=521962

Blog posts on SMAT data report

https://aka.ms/SMAT2013BTJpart1

https://aka.ms/SMAT2013BTJpart2

      -Frog Out

Using Microsoft Flow to Start and Stop a Set of Azure VMs

   In this blog post I’ll walk through creating a Microsoft Flow flow for starting (and another for stopping) a set of Azure Resource Manager (ARM) VMs.  Note that this is not my own original work.  I implemented this based on the work of someone else I found online but can no longer find the original owner’s reference.  If you do find this elsewhere please feel free to let me know in the comments.

Background

   While it is possible to start and stop Azure VMs from the newly released Azure mobile app, most time I need to start up a set (3-5) VMs at a time for a SharePoint farm / app dev environment / etc.  I was able to find a sample someone wrote in Microsoft Flow to trigger the start / stop from the Flow mobile app.  The flow calls Azure AD to get an access token using an Azure AD app that has permissions to start / stop VMs.  The access token is then passed into a series of REST calls to start up VMs in order (usually domain controller, database server, app server, web front end, etc.)  Finally the flow will send a mobile push notification letting me know that the VMs have started.

Word of caution

   This solution embeds the client ID and client secret (essentially user name and password) for the Azure AD app which has permissions to the Azure VM.   This could be a security risk and as such should be cautioned from doing this.  Treat this sample as a proof of capability for development purposes only.  I’m continuing to explore alternatives (ex. Managed Service Identity, Azure connector in Microsoft Flow) which would increase security for this solution.  If anyone has any suggestions please feel free to let me know in the comments.

Solution – Start Azure VMs

   I won’t go into detail on each and every step as some of these are self explanatory or a repeat of others (ex. 2nd and 3rd VM to be started.)  Before going into the flow to be created, ensure you have an Azure AD app registered with permissions on the desired VMs to be started / stopped.

Register Azure AD App

   Log into the “new” Azure portal (portal.azure.com) and go into the Azure AD screen.  First click on Properties to view the directory ID.  Make note of this for future use.

image

   Click App registrations and create a new app of type “Web app / API”.

image

   Make note of the application ID (also known as client ID).

image

   Go into the Required Permissions setting for the app.  Add a permission for the “Windows Azure Service Management API”.  Choose the permission “Access Azure Service Management as organization users” which is currently in preview.

   Create a key for the Azure AD app and write this down.  You will only get to see this key once and cannot retrieve it at a later time.  If you lose the key value you will need to create a new one.

Assign access control to resource group

   Now that the Azure AD App has been registered it will need access control to the resource group (or individual Azure VMs, more administration if this option) so that the app can start / stop the desired VMs.  I granted Virtual Machine Contributor role to the Azure AD App but more fine grained controls might be possible if security concerns are a factor.

image

Microsoft Flow sample

  1. Manually trigger a flow
  2. Get access token for Azure
  3. Parse JSON to extract access token
  4. Start VMs (in series)
  5. Push notification if successful

image

Manually trigger a flow

   This is self explanatory.  This will let you initiate the flow from Flow web portal or the Flow mobile app.

Get access token for Azure

   This step will use an HTTP POST action to the Azure AD directory where the Azure AD app is registered.  Ideally you should send a request to this URI using Postman or a similar REST endpoint testing tool to get a sample of the JSON response to be used in the following step.

image

  • Method: POST
  • Uri: https://login.microsoftonline.com/<directoryID from previous step>/oauth2/token
  • Headers
    • Content-Type: application/x-www-form-urlencoded
  • Body: resource=https://management.azure.com/&client_id=<client ID from previous step>&grant_type=client_credentials&client_secret=<client secret from previous step>

Example JSON response using Postman:

{
   “token_type”: “Bearer”,
   “expires_in”: “3599”,
   “ext_expires_in”: “0”,
   “expires_on”: “1508115492”,
   “not_before”: “1508111592”,
   “resource”: “https://management.azure.com/”,
   “access_token”: “<removed value>”
}

Parse JSON

   Either using the sample JSON response above or your own you can define the schema of the JSON to be parsed.  Specify the “Body” of the JSON response from the prior HTTP POST action.  The important element to parse out is “access_token”.

image

{

    “type”: “object”,

    “properties”: {

        …<other properties here>…,

        “access_token”: {

            “type”: “string”

        }

    }

}

Start VM REST call

   Add another HTTP POST action this time specifying the following configuration.

image

  • Method: POST
  • Uri: https://management.azure.com/subscriptions/<Azure subscription ID>/resourceGroups/<resource group name>/providers/Microsoft.Compute/virtualMachines/<Azure VM name>/start?api-version=2016-04-30-preview
  • Headers
    • Authorization: Bearer <insert the bearer token “input” from prior Parse JSON step>

   Note that I used an older version for the “api-version=” portion of query string (highlighted in green).  A newer version might also be available and compatible but I haven’t tested anything newer.

   Create as many additional HTTP POST actions that call off to additional VMs as needed.  I hand coded the Uri for each as Microsoft Flow didn’t yet support expressions and other dynamic variables when this solution was first created.  You may want to investigate those to reduce repeated syntax if possible.

Notify when VMs started

   Straight forward action with a simple notification to let me know when flow has completed.

image

Solution – Stop Azure VMs

   The steps for stopping a set of Azure VMs will be identical to the “start” flow except that stopping VMs can be done in parallel as the order is not as important.  In your own scenario the order may be important so consider that when creating your own solution.

  1. Manually trigger a flow
  2. Get Access Token for Azure
  3. Parse JSON to extract access token
  4. Stop VMs (in parallel)
  5. Push notification if successful

image

   The other important difference will be to call to “deallocate” (highlighted in red) the VM rather than “start” using the Azure Service Management API.  See example below for the HTTP POST to a VM.

Sample Execution

   As you can see from the below sample executions of both flows the start and stop of each VM can take some time (2-3 minutes) but is still an easier process of clicking one button rather than multiple clicks within the Azure Portal or mobile app.

image

image

Conclusion

   Hopefully this walkthrough will help others who are interested in automating Azure VMs to start and stop (or any other authenticated actions against Azure resources).  I’m hoping to try out additional options to remove the need to store client ID and secret within the flow.  For the time being try out this process and let me know if you have any issues.

      -Frog Out

Slides and Demo Files from Dogfood Con 2017

   A big thanks to Cassandra, Trey, Danillo, and all of the other organizers of Dogfood Con 2017.  Thanks also to all of the folks who attended my two sessions.  Below are my slides and source code.  Feel free to let me know about any follow up questions or comments.

PowerApps and Microsoft Flow for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

Intro to Power BI for Office 365 Developers

Slides

Sample Financial Data file

http://go.microsoft.com/fwlink/?LinkID=521962

Blog posts on SMAT data report

https://aka.ms/SMAT2013BTJpart1

https://aka.ms/SMAT2013BTJpart2

-Frog Out

SharePoint CSOM to Traverse All Sites in SharePoint Online

   In the past I’ve written posts for “PowerShell to Enumerate SharePoint 2010 or SharePoint 2013 Permissions” or “PowerShell Script To Traverse All Sites In SharePoint 2010 (or 2007) Farm” to assist with traversing through all sites within a SharePoint on-prem farm.  In this post I’ll share a snippet I recently used for traversing through all site collections and subsites within a SharePoint Online tenant.

Background

   If you’ve worked with the SharePoint Online Management Shell you may know that originally it was not able to retrieve Personal Sites (also known as My Sites / OneDrive for Business sites) in SharePoint Online.  As far as I’m aware this was primarily a limitation of the underlying client side libraries (Microsoft.SharePoint.Client.*).  Fast forward to a recent release (I don’t have the specific one but I can confirm it is in the 16.1.6621.1200 release of the Microsoft.SharePointOnline.CSOM NuGet package) and now it is supported to retrieve Personal Sites.  In the management shell this is accomplished by calling the following:

Connect-SPOService -Url ‘<tenantAdminUrl>’

Get-SPOSite -Limit all -IncludePersonalSite $true

   The problem is that if you want to traverse all site collections in SharePoint Online in client side object model (CSOM) you would need to know how the SharePoint Online Management Shell implements that inclusion of Personal Sites with the rest of site collections.  In order to find this out I used a disassembler (ILSPY in my case, but there are many alternatives available as well) on the underlying libraries to recreate the process in my own code.

Solution

   The resulting CSOM that I came up with is below.  The yellow code is processing all site collections within SharePoint Online while the green code is processing sites (subsites / webs / “not site collections”).  Feel free to borrow this and use in your own code but note that it is provided as-is with no warranty.

SPOSitePropertiesEnumerable ssp = null;

SPOSitePropertiesEnumerableFilter sspFilter = new SPOSitePropertiesEnumerableFilter();

SharePointOnlineCredentials creds = new SharePointOnlineCredentials(“myUsernameGoesHere”, securePassword);

using (PnPClientContext cc = new PnPClientContext(“myURLGoesHere”))

{

    cc.Credentials = creds;

   Tenant tenant = new Tenant(cc);

    //loop through all site collections including personal sites (even though not being used)

    //borrowed this code from after decompiling SPO Management Shell assemblies

    sspFilter.IncludePersonalSite = PersonalSiteFilter.Include;

    sspFilter.IncludeDetail = true;

    sspFilter.StartIndex = null;

    ssp = tenant.GetSitePropertiesFromSharePointByFilters(sspFilter);

    cc.Load(ssp);

    cc.ExecuteQuery();

    foreach (SiteProperties sp in ssp)

    {

        //DO YOUR WORK HERE FOR EACH SITE COLLECTION, such as looping through subwebs

        cc.Load(cc.Web, w => w.NoCrawl,

                                    w => w.Webs,

                                    w => w.Url);

        cc.ExecuteQuery();

        //check subweb(s)

        foreach (var subweb in cc.Web.Webs)

        {

            cc.Load(subweb, sw => sw.NoCrawl,

                                        sw => sw.Url);

            cc.ExecuteQuery();

        }

    }

}

Conclusion

   In this post I shared a snippet for traversing all site collections in SharePoint Online with C# CSOM code.  In my daily job I’ve been using this in combination with Azure Functions for a number of interesting interactions with a SharePoint Online tenant.  More to come on those scenarios in future weeks.  For now let me know in the comments if you have any questions or issues implementing the above snippet in your own code.  Happy coding!

      -Frog Out

Slides and Demo Files from CincyDevelop() Conference

   A big thanks to Phil and all of the other organizers of Cincy.Develop() / Day of Agile Conference.  Thanks also to all of the folks who attended my Integrate All the Things: PowerApps and Flow for Developers session.  Below are my slides and source code.  Feel free to let me know about any follow up questions or comments.

PowerApps and Microsoft Flow for Developers

GitHub link to demo project files

https://github.com/BrianTJackett/BTJ.PowerApps.AzureDBSample

Slides

-Frog Out

Upcoming Speaking Events in 2017

I’m honored to be accepted to speak at the following upcoming events.  Here are the abstracts for these presentations.  If you are at any of these events feel free to stop by and chat.  Registration is still open as of the time of writing as well.

 

Cincy.Develop()

Website: Cincy Day of Agile and Cincy.Develop()

When: Fri. Jul 28, 2:40pm-3:40pm

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

DogFood Conference

Website: DogFoodCon

When: Oct 5-6, timeslots TBA

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

Title: Intro to Power BI for Office 365 Developers

Abstract: “Power BI is SSRS in the cloud, right?” “I’ll throw a few pie charts on the screen and call it a dashboard, right?” Wrong. In this session we will introduce Office 365 developers to the Power BI service and Power BI Desktop. We will cover data sources that can be used (Azure SQL, SharePoint, files, etc.), direct query vs. imported dataset, report and dashboard creation, and the on-prem data gateway. We will also break down misconceptions of good dashboard design (hint: pie charts and 3D are almost always a bad design). Prior experience with Power BI or Power BI desktop are not required.

 

SPTechCon DC

Website: SPTechCon DC

When: Nov 12-15, timeslots TBA

Title: Integrate All the Things: PowerApps and Microsoft Flow for Developers

Abstract: Securely start a server farm of Azure dev VMs with the tap of a button on your phone. Update data in an on-prem DB from any device without writing any plumbing code or UI layer. In this session we will overview PowerApps and Microsoft Flow which are low-code / no-code solutions that allow forms and workflow development that integrate to hundreds of services including Twilio, Twitter, Azure, GitHub, and more. We will also talk about the developer story for integrating with Custom Connectors and on-prem data sources such as SQL Server and SharePoint. Lastly we will demo a number of scenarios to show how easily you can create and consume apps across Windows, iOS, Android, and web. Prior experience with PowerApps and Flow is not required.

 

Title: Intro to Power BI for Office 365 Developers

Abstract: “Power BI is SSRS in the cloud, right?” “I’ll throw a few pie charts on the screen and call it a dashboard, right?” Wrong. In this session we will introduce Office 365 developers to the Power BI service and Power BI Desktop. We will cover data sources that can be used (Azure SQL, SharePoint, files, etc.), direct query vs. imported dataset, report and dashboard creation, and the on-prem data gateway. We will also break down misconceptions of good dashboard design (hint: pie charts and 3D are almost always a bad design). Prior experience with Power BI or Power BI desktop are not required.

 

      -Frog Out