Im trying to do a role assignment usig BICEP. My goal is to assign the Databricks Access connector as the storage blob data contributor on Storage account.

Im deploying both, Databricks (with managed RG which contains my access connector) and storage account too.

I have the below code to do so: main.bicep

module databricks 'platform/modules/databricks/deploy.bicep' = {
  name: 'DeployDatabricksWorkspace'
  params: {
    workspaceName: workspaceName
    pricingTier: pricingTier
    location: location
    disablePublicIp: disablePublicIp
    vnetID: vNetId
    environment: environment
  }
  dependsOn: [
    virtualNetwrok
  ]
}

module accessConnectorStorageRbac 'platform/modules/roles/accessConnector/deploy.bicep' = {
  name: 'AccessConnector-rbac'
  params: {
    storageAccountName: storageAccountName
    principalId: accessConnectorPrincipalId
    roleId: 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' // Storage Blob Data Contributor
  }
  dependsOn: [
    storageAccountModule
    databricks
  ]
}

and then I have the module where the actual deployment happens:

param disablePublicIp bool
param workspaceName string
param pricingTier string
param location string = resourceGroup().location
param vnetID string
param environment string

var managedResourceGroupName = 'rg-mgd-databricks-${workspaceName}-${uniqueString(workspaceName, resourceGroup().id)}'

resource workspace 'Microsoft.Databricks/workspaces@2024-05-01' = {
  name: workspaceName
  location: location
  sku: {
    name: pricingTier
  }
  properties: {
    managedResourceGroupId: managedResourceGroup.id
    parameters: {
      customVirtualNetworkId: {
        value: vnetID
      }
      customPublicSubnetName: {
        value: 'snet-ads-public-${environment}-weu-01'
      }
      customPrivateSubnetName: {
        value: 'snet-ads-private-${environment}-weu-01'
      }
      enableNoPublicIp: {
        value: disablePublicIp
      }
    }
  }
}

resource managedResourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
  scope: subscription()
  name: managedResourceGroupName
}

resource accessConnector 'Microsoft.Databricks/accessConnectors@2024-05-01' existing = {
  name:'unity-catalog-access-connector'
  scope:resourceGroup(managedResourceGroupName)
  dependsOn: [
    managedResourceGroup
  ]
}

output accessConnectorPrincipalId string = accessConnector.identity.principalId

Error Im getting is below:

Resource group 'rg-mgd-databricks-ads-mdp-comm-dev-weu-01-ntm7hk4xxjyda' could not be found

so the rg-mgd-databricks-ads-mdp-comm-dev-weu-01-ntm7hk4xxjyda is the name where my access connector is. Not sure why I get this error since I have the dependencies.

Im trying to do a role assignment usig BICEP. My goal is to assign the Databricks Access connector as the storage blob data contributor on Storage account.

Im deploying both, Databricks (with managed RG which contains my access connector) and storage account too.

I have the below code to do so: main.bicep

module databricks 'platform/modules/databricks/deploy.bicep' = {
  name: 'DeployDatabricksWorkspace'
  params: {
    workspaceName: workspaceName
    pricingTier: pricingTier
    location: location
    disablePublicIp: disablePublicIp
    vnetID: vNetId
    environment: environment
  }
  dependsOn: [
    virtualNetwrok
  ]
}

module accessConnectorStorageRbac 'platform/modules/roles/accessConnector/deploy.bicep' = {
  name: 'AccessConnector-rbac'
  params: {
    storageAccountName: storageAccountName
    principalId: accessConnectorPrincipalId
    roleId: 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' // Storage Blob Data Contributor
  }
  dependsOn: [
    storageAccountModule
    databricks
  ]
}

and then I have the module where the actual deployment happens:

param disablePublicIp bool
param workspaceName string
param pricingTier string
param location string = resourceGroup().location
param vnetID string
param environment string

var managedResourceGroupName = 'rg-mgd-databricks-${workspaceName}-${uniqueString(workspaceName, resourceGroup().id)}'

resource workspace 'Microsoft.Databricks/workspaces@2024-05-01' = {
  name: workspaceName
  location: location
  sku: {
    name: pricingTier
  }
  properties: {
    managedResourceGroupId: managedResourceGroup.id
    parameters: {
      customVirtualNetworkId: {
        value: vnetID
      }
      customPublicSubnetName: {
        value: 'snet-ads-public-${environment}-weu-01'
      }
      customPrivateSubnetName: {
        value: 'snet-ads-private-${environment}-weu-01'
      }
      enableNoPublicIp: {
        value: disablePublicIp
      }
    }
  }
}

resource managedResourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
  scope: subscription()
  name: managedResourceGroupName
}

resource accessConnector 'Microsoft.Databricks/accessConnectors@2024-05-01' existing = {
  name:'unity-catalog-access-connector'
  scope:resourceGroup(managedResourceGroupName)
  dependsOn: [
    managedResourceGroup
  ]
}

output accessConnectorPrincipalId string = accessConnector.identity.principalId

Error Im getting is below:

Resource group 'rg-mgd-databricks-ads-mdp-comm-dev-weu-01-ntm7hk4xxjyda' could not be found

so the rg-mgd-databricks-ads-mdp-comm-dev-weu-01-ntm7hk4xxjyda is the name where my access connector is. Not sure why I get this error since I have the dependencies.

• azure-databricks
• azure-bicep
• azure-rbac

Share Improve this question Follow edited Nov 18, 2024 at 10:45 play_something_good asked Nov 18, 2024 at 10:24 play_something_goodplay_something_good 14322 silver badges1212 bronze badges 10

• Cross check your resource resource group name once? – Bhavani Commented Nov 18, 2024 at 11:20
• @Bhavani what do you mean? My RG name is correct when I compare the one generted by the error and the deployed one – play_something_good Commented Nov 18, 2024 at 11:54
• can you check the below answer, and let me know any concerns are there. – Bhavani Commented Nov 18, 2024 at 12:10
• you shouldn't be deploying the connector in the managed RG cause it is managed by the azure platform. Ff you remove the scope / dependsOn on the accessConnector that should work fine. – Thomas Commented Nov 18, 2024 at 20:49
• If you would like to deploy the accessConnector resource in the managed resource group, it has to be done in a different module because the scope of the module deployment is different from the scope of the accessConnector resource (scope:resourceGroup(managedResourceGroupName)) – Thomas Commented Nov 18, 2024 at 20:50

 |  Show 5 more comments

1 Answer 1

Sorted by: Reset to default -1

Resource group 'rg-mgd-databricks-ads-mdp-comm-dev-weu-01-ntm7hk4xxjyda' could not be found

You have provided managed resource group of databricks workspace as

rg-mgd-databricks-${workspaceName}-${uniqueString(workspaceName, resourceGroup().id)}

But it is not the correct format of managed resource group. According to this

For Azure Databricks: By default, a managed resource group is created for you when your workspace is created. It will be named as databricks-rg-<WorspaceName>-<RandomNumber>.

The managed resource group is not modifiable. you will be able to find the resource group and managed resource group in the overview page of data bricks as shown below:

Use that name as managed resource group name. Along with that instead of referring the name directly try to refer the managed rg property of workspace in a separate file this will be helpful to fetch the correct managed rg name as per the requiremnt:

param managedResourceGroupId string
 
resource accessConnector 'Microsoft.Databricks/accessConnectors@2024-05-01' = {
  name: accessConnectorName
  location: location
  scope: resourceGroup(managedResourceGroupId)
  properties: {
    // Specify any required properties here if necessary.
  }
}