0

I'm trying to create a pool using a custom image I created from VM with azure python sdk. The location and resource group match.

Here's my code:

import azure.batch as batch
from azure.batch import BatchServiceClient
from azure.batch.batch_auth import SharedKeyCredentials
from azure.batch import models



account = 'mybatch'
key = 'Adgfdj1hhsdfqATc/K2fgxdfg/asYgKRP2pUdfglBce7mgmSBdfgdhC7f3Zdfgrcgkdgh/dfglA=='
batch_url = 'https://mybatch.westeurope.batch.azure.com'

creds = SharedKeyCredentials(account, key)
batch_client = BatchServiceClient(creds, base_url = batch_url)

pool_id_base = 'mypool'
idx = 1

pool_id = pool_id_base + str( idx )

while batch_client.pool.exists( pool_id ):
  idx += 1
  pool_id = pool_id_base + str( idx )

print( 'pool_id ' + pool_id )

sku_to_use =  'batch.node.ubuntu 18.04'

# 
# image_ref_to_use = models.ImageReference(
#     offer = 'UbuntuServer', 
#     publisher = 'Canonical',
#     sku = '18.04-LTS', 
#     version = 'latest'
#   )



image_ref_to_use = models.ImageReference(
    virtual_machine_image_id = '/subscriptions/1834572sd-34sd409a-sdfb-sc345csdfesourceGroups/resource-group-1/providers/Microsoft.Compute/images/my-image-1'
  )


vm_size = 'Standard_D3_v2' 

vmc = models.VirtualMachineConfiguration(
  image_reference = image_ref_to_use,
  node_agent_sku_id = sku_to_use
)

new_pool = models.PoolAddParameter(
  id = pool_id, 
  vm_size = vm_size, 
  virtual_machine_configuration = vmc,
  target_dedicated_nodes = 1
)

batch_client.pool.add(new_pool)

According to the docs I should be able to use either virtual_machine_image_id other provide marketplace image parameters. I can create a pool of standard marketplace images, but I get an error when I'm trying to use an id of my custom image.

Traceback (most recent call last):   File "create_pool.py", line 60, in <module>
    batch_client.pool.add(new_pool)   File "/root/miniconda/lib/python3.6/site-packages/azure/batch/operations/pool_operations.py", line 312, in add
    raise models.BatchErrorException(self._deserialize, response) azure.batch.models.batch_error.BatchErrorException: {'additional_properties': {}, 'lang': 'en-US', 'value': 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:0dfdf9c1-edad-4b72-8e8f-f8dbcfd0abbdf\nTime:2018-12-06T10:51:21.9417222Z'}

How can I resolve this issue?

UPDATE

I tried to use ServicePrincipalCredentials with the following:

CLIENT_ID: I created a new application in Defaut Directiry -> Add registration and got it's Application ID.

SECRET: A created a key for the new application and used its value.

TENANT_ID: az account show in the cloud shell.

RESOURCE: Used 'https://batch.core.windows.net/'.

Updated my code like this:

from azure.common.credentials import ServicePrincipalCredentials

creds = ServicePrincipalCredentials(
  client_id=CLIENT_ID,
  secret=SECRET,
  tenant=TENANT_ID,
  resource=RESOURCE
)

And I get another error:

Keyring cache token has failed: No recommended backend was available. Install the keyrings.alt package if you want to use the non-recommended backends. See README.rst for details.
Traceback (most recent call last):
  File "create_pool.py", line 41, in <module>
    while batch_client.pool.exists( pool_id ):
  File "/root/miniconda/lib/python3.6/site-packages/azure/batch/operations/pool_operations.py", line 624, in exists
    raise models.BatchErrorException(self._deserialize, response)
azure.batch.models.batch_error.BatchErrorException: Operation returned an invalid status code 'Server failed to authorize the request.'
sr9yar
  • 4,850
  • 5
  • 53
  • 59

1 Answers1

2

Try using Service Principal Credentials instead of the Shared Key Credentials

credentials = ServicePrincipalCredentials(
    client_id=CLIENT_ID,
    secret=SECRET,
    tenant=TENANT_ID,
    resource=RESOURCE
)

There seems to be an error with Shared Key Credentials.

Documentation Link: https://learn.microsoft.com/en-us/azure/batch/batch-aad-auth

Issue Link: https://github.com/Azure/azure-sdk-for-python/issues/1668

Note: Please remove your account details as it can be accessed by anyone. Replace the account name and key with ****.

Update If the Service Principal Credentials are not working, try using the User credentials and see if that works.

from azure.common.credentials import UserPassCredentials
import azure.batch.batch_service_client as batch

credentials = UserPassCredentials(
    azure_user,
    azure_pass
)
batch_client = batch.BatchServiceClient(credentials, base_url = batch_url)
Shashank Verma
  • 369
  • 1
  • 14
  • They are not real account details, no worries ;-) And I still have an error, updated my question – sr9yar Dec 06 '18 at 16:47
  • ok, I just tried what your edit suggests, but got errors. I'm using a trail subscription. We believe, this is some kind of trail version limitation. I'll come back to it, after we've moved to a paid subscription. Another thing, using a user/pass auth here seems to be odd here, I'd try to avoid storing my account creds anywhere in the code. – sr9yar Dec 17 '18 at 06:43
  • Actually the user credentials should not be used usually. But, since the other methods were not working in your case I thought of trying this. But if you have the necessary privileges, then this should have worked. If this would have worked then it means there is an issue with the Principal credentials you created. But since it's not working that means indeed there is some account limitation. – Shashank Verma Dec 17 '18 at 08:25
  • I was testing everything with my personal account I just registered for this purpose (have access to everything, to avoid possible permissions issues). So I guess, I do have all possible permissions here. I just got access to the company account, I'm moving things there. Maybe I'll come back to this question, but for the time being, we consider this a trails subscription limitation. In any case, thanks for your help. :) – sr9yar Dec 17 '18 at 17:13