All posts by Jos

Uploading a file to onedrive for business with Python

For a Raspberry Pi project that’ll take a number of pictures of my house for an as of yet unknown period of time I’m sharing my very first Python script with you.

All it has to do is upload all files from a given folder to a given Onedrive for Business path, as obviously the Pi can’t store much data on its tiny SD card. You’ll need to register an azure ad app and give it the appropriate permissions. You’ll have to consent to the application once (url format = https://login.microsoftonline.com/common/adminconsent?client_id={client-id}).

Then the schedule below Python script on your Pi, it will retrieve an Azure token without the need for external libraries, parse the directory and upload everything in it to the given onedrive for business URL, simple as that! It can also be used for Sharepoint or Teams by adjusting the path.

import requests 
import json
directory = r"c:\temp\uploads"
data = {'grant_type':"client_credentials", 
        'resource':"https://graph.microsoft.com", 
        'client_id':'XXXXX', 
        'client_secret':'XXXXX'} 
URL = "https://login.windows.net/YOURTENANTDOMAINNAME/oauth2/token?api-version=1.0"
r = requests.post(url = URL, data = data) 
j = json.loads(r.text)
TOKEN = j["access_token"]
URL = "https://graph.microsoft.com/v1.0/users/YOURONEDRIVEUSERNAME/drive/root:/fotos/HouseHistory"
headers={'Authorization': "Bearer " + TOKEN}
r = requests.get(URL, headers=headers)
j = json.loads(r.text)
print("Uploading file(s) to "+URL)
for root, dirs, files in os.walk(directory):
    for filename in files:
        filepath = os.path.join(root,filename)
        print("Uploading "+filename+"....")
        fileHandle = open(filepath, 'rb')
        r = requests.put(URL+"/"+filename+":/content", data=fileHandle, headers=headers)
        fileHandle.close()
        if r.status_code == 200 or r.status_code == 201:
            #remove folder contents
            print("succeeded, removing original file...")
            os.remove(os.path.join(root, filename)) 
print("Script completed")
raise SystemExit

Import a PBIX to PowerBI using PowerShell

Reading up on the PowerBI API to import PBIX files to the PowerBI service and an example on the actual request I decided to write a PowerShell function to import a PowerBI PBIX file to the PowerBI service.

The PowerBI Import API is quite specific and kept giving me 400’s like:

  • “Bad Request” with no details
  • UnknownError pbi.error exceptionCulprit 1 (loved this one)
  • MultiPartMimeStreamFormatException
  • RequestedFileIsEncryptedOrCorrupted

Eventually I figured out how to import my reports directly into PowerBI, so to help you automate importing your reports into workspaces or directly to customers using PowerShell, I’ll share my PowerShell function with you:

GitLab: Import-PBIXToPowerBI.ps1

Azure AD sign in and audit log retention

Often we, as cloud admins, need our audit or sign in logs. Usually, we need real-time data because, for example, we’re debugging why that one user has conditional access issues. But sometimes, we need to go back further than 30 days. And that is not something Azure does by default, but can be enabled:


Our options when exporting logs are limited to a Storage account, Log Analytics or an Event Hub. All these options offer multiple extraction methods to cover your transport needs to other systems. The default retention period is then forever, which is nice as we might need audit info going back a bit as hacks are usually discovered after about 206 days.

If you don’t have specific tools or requirements, I recommend setting up a Log Analytics workspace and connecting that to Azure AD:

Whichever method you choose, a P1 or P2 license is required. You only need a single license for the entire tenant when using the export audit / singin log functionality of AzureAD. Once configured, the Logs option directly bring you to the Log Analytics workspace search results:

I’ve briefly shown how to configure AzureAD to send audit and sign in logs to Log Analytics so you can go back further than 30 days. Stay tuned for the next post that will utilize these logs to dive deeper into Guest User activity.

Devices that lack a bitlocker recovery key in AzureAD

With Intune’s new Bitlocker Encryption Report administrators have an effective way of seeing which of their devices have been encrypted.

But if we want to know if we can actually recover the bitlocker key of a device, we need to know if it was ever uploaded to AzureAD.

Network or local device issues can sometimes prevent the recovery key from reaching AzureAD, resulting in lost data if the device’s disk needs to be recovered for any reason. To hunt down devices that have not escrowed their recovery key to AzureAD, you can use my report function (in PowerShell as always):

GitLab source download link

Programmatically triggering a group licenses refresh for AzureAD

Azure AD allows us to assign licenses to groups, a nifty feature that has made a host of automation scripts dealing with bulk license assignment obsolete.

A problem I’ve encountered is that when you assign users to a group, license assignments are not processed right away, especially if you didn’t have enough licenses when you assigned the user to the group (and added licenses to the tenant later).

Azure AD has a button to trigger an update manually:

But of course, this can also be automated with PowerShell!

function Invoke-AzHAPIReprocessGroupLicenses{
    <#
        .SYNOPSIS
        reprocesses group license assignment

        .NOTES
        Author: Jos Lieben

        .PARAMETER AzureRMToken
        Use Get-azureRMToken to get a token for this parameter

        .PARAMETER groupGUID
        GUID of the group to reprocess licenses of
    
        Requires:
        - Global Administrator Credentials (non-CSP!)
        - AzureRM Module
        - supply result of get-azureRMToken function
    #>
    param(
        [Parameter(Mandatory = $true)]$AzureRMToken,
        [Parameter(Mandatory = $true)]$groupGUID
    )
    $header = @{
        'Authorization' = 'Bearer ' + $AzureRMToken
        'X-Requested-With'= 'XMLHttpRequest'
        'x-ms-client-request-id'= [guid]::NewGuid()
        'x-ms-correlation-id' = [guid]::NewGuid()
    }   

    $url = "https://main.iam.ad.ext.azure.com/api/AccountSkus/Group/$groupGUID/Reprocess"
    Invoke-RestMethod –Uri $url –Headers $header –Method POST -Body $Null -UseBasicParsing -ErrorAction Stop -ContentType "application/json"
}

Source on GIT: https://gitlab.com/Lieben/assortedFunctions/blob/master/invoke-AzHAPIReprocessGroupLicenses.ps1https://gitlab.com/Lieben/assortedFunctions/blob/master/invoke-AzHAPIReprocessGroupLicenses.ps1

Disclaimer: the ‘hidden azure api’ is not officially supported.

Requires output from the Get-AzureRMToken function

Get an Office365 / Azure AD tenant ID from a user’s login name or domain

I often need a tenant ID for a given customer, the usual method to get it is to log in to the Azure portal and find it there. But what if you want to get the tenant ID programmatically? Without actually logging in? And you only know the log in name of a user? Or just one of the customer’s domain names?

Then this’ll help you out!

function get-tenantIdFromLogin(){
    <#
      .SYNOPSIS
      Retrieves an Office 365 / Azure AD tenant ID for a given user login name (email address)
      .EXAMPLE
      $tenantId = get-tenantIdFromLogin -Username you@domain.com
      .PARAMETER Username
      the UPN of a user
      .NOTES
      filename: get-tenantIdFromLogin.ps1
      author: Jos Lieben
      blog: www.lieben.nu
      created: 8/3/2019
    #>
    Param(
        [Parameter(Mandatory=$true)]$Username
    )
    $openIdInfo = Invoke-RestMethod "https://login.windows.net/$($Username.Split("@")[1])/.well-known/openid-configuration" -Method GET
    return $openIdInfo.userinfo_endpoint.Split("/")[3]
}

Obviously, you can also get the tenant ID by just filling out bogus info in front of the user’s login (e.g. bogus@ogd.nl), it’ll still work as only the domain part of the login is really used.

Hope this helps someone 🙂

Git link: https://gitlab.com/Lieben/assortedFunctions/blob/master/get-tenantIdFromLogin.ps1

Deploying an embedded file (FONT) in a Powershell script through Intune MDM

Most solutions that describe how to deploy a font through Intune use an external source to host fonts such as Azure Blob storage.

If you want to KISS (keep it simple, stupid), you don’t want to maintain two different things (your script and externally accessible storage).

The following example shows how to embed a file in a PowerShell script, and then install it into the Fonts folder. This could, of course, be used for other purposes, but don’t forget that the script size limit in Intune is only 200kb.

First, execute the following in a PS window:

$inputFontPath = "C:\fonts\YourFont.ttf"
Compress-Archive $inputFontPath -CompressionLevel Optimal -DestinationPath (Join-Path $Env:TEMP -ChildPath "tempzipfontzipfile.zip") -Force
[Array]$bytes = [io.file]::ReadAllBytes((Join-Path $Env:TEMP -ChildPath "tempzipfontzipfile.zip"))
$b64 = [System.Convert]::ToBase64String($bytes)
$b64 | Set-Clipboard

You have now compressed and base64 encoded YourFont.ttf, and this is loaded in memory (clipboard).

Create a new file, e.g. YourFont.ps1 and add the following:

$b64 = "<SELECT EVERYTHING BETWEEN THESE QUOTES, THEN PRESS CTRL+V>"
$byteContent = [System.Convert]::FromBase64String($b64)
$byteContent | Set-Content (Join-Path $Env:TEMP -ChildPath "tempzipfontzipfile.zip") -Encoding Byte -Force
Expand-Archive -Path (Join-Path $Env:TEMP -ChildPath "tempzipfontzipfile.zip") -DestinationPath (Join-Path $Env:TEMP -ChildPath "tempfontsfolder") -Force

$sa =  new-object -comobject shell.application
$Fonts =  $sa.NameSpace(20)
gci (Join-Path $Env:TEMP -ChildPath "tempfontsfolder") | % {$Fonts.MoveHere($_.FullName)}

Remove-Item (Join-Path $Env:TEMP -ChildPath "tempzipfontzipfile.zip") -Force -ErrorAction SilentlyContinue
Remove-Item (Join-Path $Env:TEMP -ChildPath "tempfontsfolder") -Force -Recurse -ErrorAction SilentlyContinue

On the first line, follow the instructions (e.g. paste your b64-encoded font on that line). Now save and check if the size is below 200kb, then distribute the script to your users. For Fonts, don’t forget to let the script run in system context as administrative permissions are required when installing Fonts.

This method can be used to deploy other payloads as well, happy scripting!

Removing special characters from UTF8 input for use in email addresses or login names

When working with non-US customers, users often have characters in their names like ë, ó, ç and so on. Most of the time, a ‘human process’ converts these to their simple equivalent of e, o and c for use in computerized systems.

When searching for such a mapping of special characters to ‘safe’ characters I had a hard time finding a good list or PowerShell method to automatically convert special characters to standard A-Z characters so I wrote one:

function get-sanitizedUTF8Input{
    Param(
        [String]$inputString
    )
    $replaceTable = @{"ß"="ss";"à"="a";"á"="a";"â"="a";"ã"="a";"ä"="a";"å"="a";"æ"="ae";"ç"="c";"è"="e";"é"="e";"ê"="e";"ë"="e";"ì"="i";"í"="i";"î"="i";"ï"="i";"ð"="d";"ñ"="n";"ò"="o";"ó"="o";"ô"="o";"õ"="o";"ö"="o";"ø"="o";"ù"="u";"ú"="u";"û"="u";"ü"="u";"ý"="y";"þ"="p";"ÿ"="y"}

    foreach($key in $replaceTable.Keys){
        $inputString = $inputString -Replace($key,$replaceTable.$key)
    }
    $inputString = $inputString -replace '[^a-zA-Z0-9]', ''
    return $inputString
}

#example usage:
get-sanitizedUTF8Input -inputString "Jösè"
#result:
Jose

Edit: my colleague Gerbrand alerted me to a post by Grégory Schiro which solves this issue much more elegantly using native .NET functions. My slightly modified version to really ensure nothing non a-zA-Z0-9 gets past the function:

function Remove-DiacriticsAndSpaces
{
    Param(
        [String]$inputString
    )
    $objD = $inputString.Normalize([Text.NormalizationForm]::FormD)
    $sb = New-Object Text.StringBuilder
 
    for ($i = 0; $i -lt $objD.Length; $i++) {
        $c = [Globalization.CharUnicodeInfo]::GetUnicodeCategory($objD[$i])
        if($c -ne [Globalization.UnicodeCategory]::NonSpacingMark) {
          [void]$sb.Append($objD[$i])
        }
      }
    
    $sb = $sb.Normalize([Text.NormalizationForm]::FormC)
    return($sb -replace '[^a-zA-Z0-9]', '')
}
#example usage:
Remove-DiacriticsAndSpaces -inputString "Jösè"
#result:
Jose

And an even easier oneliner I converted to a function by John Seerden:

function Remove-DiacriticsAndSpaces
{
    Param(
        [String]$inputString
    )
    #replace diacritics
    $sb = [Text.Encoding]::ASCII.GetString([Text.Encoding]::GetEncoding("Cyrillic").GetBytes($inputString))

    #remove spaces and anything the above function may have missed
    return($sb -replace '[^a-zA-Z0-9]', '')
}

And the most advanced function I’ve found so far is by 
Daniele Catanesi (PsCustomObject): https://github.com/PsCustomObject/New-StringConversion/blob/master/New-StringConversion.ps1 in which all features of above functions are supported and parameterized.