Catalog access can be authenticated using tokens issued by an Azure IDP-managed service principal, and storage access is also authenticated using Azure IDP credentials. This configuration uses an Azure Active Directory service principal to provide secure OAuth2‑based authentication.
| Parameter | Description | Requirement | Example Value |
|---|---|---|---|
| catalog_type | Specifies the Iceberg catalog type | Mandatory | catalog_type ('rest') |
| rest_catalog_type | Specifies REST Iceberg catalog type Allowed value: unity |
Mandatory | rest_catalog_type ('unity') |
| catalog_location | Databricks Unity Iceberg REST API endpoint Format: "<workspace-url>/api/2.1/unity-catalog/iceberg-rest"
This format is according to Databricks Runtime 16.4 LTS and above. |
Mandatory | catalog_location ('https://adb-8074943983864086.6.azuredatabricks.net/api/2.1/unity-catalog/iceberg-rest') |
| catalog_name | Databricks Unity Iceberg Catalog Name | Mandatory | catalog_name ('reg_iceberg_db') |
| idp_type | Identity provider type Allowed value: azure_service_principal (Azure Active Directory service principal for secure OAuth2 authentication.) |
Mandatory | idp_type ('azure_service_principal') |
| idp_token_scope | Token scope for token generation requests (defaults to '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default' if not provided) | Optional | idp_token_scope ('2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default') |
| storage_location | AZURE Cloud Data Lake Storage location | Mandatory | storage_location ('abfss://reg-storage-acct-container@regicebergstorageacct.dfs.core.windows.net/9e12180b-1bd5-44bb-ab39-8f70763bb9f5/') |
| tenant_id | AZURE tenant id | Mandatory | tenant_id ('391c8c4c-6a2a-40fd-ab98-226b6baa5155') |
| storage_account_name | AZURE storage account name | Mandatory | storage_account_name('regicebergstorageacct') |
Example
Replace azure_user_id and azure_password variables with your Azure IDP credentials.
REPLACE AUTHORIZATION azure_idp_auth
USER '<<azure_user_id>>'
PASSWORD '<<azure_password>>';
REPLACE DATALAKE iceberg_unity_azure_idp_rest
EXTERNAL SECURITY CATALOG azure_idp_auth,
EXTERNAL SECURITY STORAGE azure_idp_auth
USING
catalog_type('rest')
rest_catalog_type('unity')
catalog_location('https://adb-8074943983864086.6.azuredatabricks.net/api/2.1/unity-catalog/iceberg-rest')
catalog_name('reg_iceberg_db')
idp_type('azure_service_principal')
tenant_id('391c8c4c-6a2a-40fd-ab98-226b6baa5155')
storage_location('abfss://reg-storage-acct-container@regicebergstorageacct.dfs.core.windows.net/9e12180b-1bd5-44bb-ab39-8f70763bb9f5/')
storage_account_name('regicebergstorageacct')
TABLE FORMAT iceberg;