0

I have an Azure ADLS storage account called eventcoadltest and I have a container called eventconnector-transformed-data-fs.

enter image description here

I have deployed this ADLS through an ARM template but I need to create a directory inside of eventconnector-transformed-data-fs as shown below (the folder debugging was created through the UI but I need to achieve the same with an ARM template):

enter image description here

I have found some posts that indicate this is not possible but it can be bypassed with some workarounds:

  1. How to create empty folder in azure blob storage
  2. Use ARM template to create directories in Azure Storage Containers?
  3. How to create a folder inside container in Azure Data Lake Storage Gen2 with the help of 'azure-storage' Package
  4. ARM template throws incorrect segments lengths for array of storage containers types
  5. how to create blob container inside Storage Account using ARM templates
  6. Microsoft Azure: How to create sub directory in a blob container
  7. How to create an azure blob storage using Arm template?
  8. How to create directories in Azure storage container without creating extra files?
  9. How to create a folder inside container in Azure Data Lake Storage Gen2 with the help of 'azure-storage' Package

I have tried to modify my ARM template as well to achieve a similar result but I haven't had any success.

{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
    "storageAccountDLName": {
        "type": "string"
    },
    "sku": {
        "type": "string"
    },
    "directoryOutput":{
        "type": "string"
    }
},
"resources": [
    {
        "type": "Microsoft.Storage/storageAccounts",
        "apiVersion": "2021-02-01",
        "sku": {
            "name": "[parameters('sku')]",
            "tier": "Standard"
        },
        "kind": "StorageV2",
        "name": "[parameters('storageAccountDLName')]",
        "location": "[resourceGroup().location]",
        "tags": {
            "Contact": "[parameters('contact')]"
        },
        "scale": null,
        "properties": {
            "isHnsEnabled": true,
            "networkAcls": {
                "bypass": "AzureServices",
                "virtualNetworkRules": [],
                "ipRules": [],
                "defaultAction": "Allow"
            }
        },
        "dependsOn": [],
        "resources": [
            {
                "type": "storageAccounts/blobServices/containers",
                "name": "[concat('default/', 'eventconnector-raw-data-fs/test')]",
                "apiVersion": "2021-02-01",
                "properties": {},
                "dependsOn": [
                    "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountDLName'))]"
                ]
            }
        ]
    }
]
}

The following code was modified for trying to create the folders inside of the containers.

"type": "storageAccounts/blobServices/containers",
"name": "[concat('default/', 'eventconnector-raw-data-fs/test')]"

The reason why I am trying to solve this problem is because I won't have access to create folders in our production environment, so that's why I need to do the deployment fully through ARM. How can I create this folder with the deployment script? Is there another alternative for achieving my desired result? Any idea or suggestion is welcome :)

abautista
  • 2,410
  • 5
  • 41
  • 72

2 Answers2

0

this doesn't make any sense, as you can not create folders in Azure Storage. They don't have folders. blobs are individual objects\entities. you are confused to believe folders exist, because UI renders them as folders, however THERE ARE NO FOLDERS in a Azure Storage Blob Container.

TLDR: you can not do this at all no matter how hard you try

4c74356b41
  • 69,186
  • 6
  • 100
  • 141
  • Thank you for your explanation and helping me out to better understand the structure of storage in Azure. Having said that, what advice could you suggest me? My only idea now is to store every single file in this eventconnector-raw-data-fs blob. – abautista Nov 02 '21 at 08:04
  • you can create container(s) using an arm template , then you can store blobs in there – 4c74356b41 Nov 02 '21 at 08:12
  • Can you elaborate this solution? I currently have a container with blobs in it but I am thinking about a container with nested blobs but I am not sure if this is possible – abautista Nov 02 '21 at 16:30
  • well, when you create blobs, they can be sorted into folders by name, kinda, but there are no actual folders. so that you can do – 4c74356b41 Nov 02 '21 at 17:38
  • The root cause of the problem is that I don't have access to the production environment, so everything must be done through ARM templates and my idea is to create a blob inside a blob through ARM - not sure if it is possible... – abautista Nov 02 '21 at 17:52
  • you still don't fully understand it. folders don't exist. blobs are individual objects. their names "create" folders, but you cant put a blob inside a blob. to put 2 blobs into the same "folder" you just name them appropriately. – 4c74356b41 Nov 03 '21 at 14:41
0

After some research I found out that it is possible to create a folder via Databricks with the following command:

dbutils.fs.mkdirs("dbfs:/mnt/folder_desktop/test/uploads")

I had to configure Databricks with my Azure Datafactory in order to run this command.

abautista
  • 2,410
  • 5
  • 41
  • 72