Just for those poor souls googling this error:

get-azurermvm : Your Azure credentials have not been set up or have expired, please run Connect-AzureRmAccount to set
up your Azure credentials.

Or any other command after Connect-AzureRMAccount with the -tenantId switch specified.

Took me over an hour to figure out that the tenant ID is actually case sensitive as the error is confusing, the log in works fine but subsequent commands fail.