Catalog access can also be authenticated using tokens generated from a Google Cloud Service Account, and storage access is also authenticated using the same Service Account.
| Parameter | Description | Requirement | Example Value |
|---|---|---|---|
| catalog_type | Specifies the Iceberg catalog type | Mandatory | catalog_type ('rest') |
| rest_catalog_type | Specifies REST Iceberg catalog type Allowed value: unity |
Mandatory | rest_catalog_type ('unity') |
| catalog_location | Databricks Unity Iceberg REST API endpoint Format: "<workspace-url>/api/2.1/unity-catalog/iceberg-rest"
This format is according to Databricks Runtime 16.4 LTS and above. |
Mandatory | catalog_location ('https://1440666423386241.1.gcp.databricks.com/api/2.1/unity-catalog/iceberg-rest'') |
| catalog_name | Databricks Unity Iceberg Catalog Name | Mandatory | catalog_name ('reg_iceberg_unity_gcp') |
| idp_type | Identity provider type Allowed value: gcp_service_account (Google Cloud Platform Service Account ID Token for secure OAuth2 authentication.) |
Mandatory | idp_type ('gcp_service_account') |
| idp_token_scope | Token scope for token generation requests (defaults to catalogHostLocation if not provided) | Optional | idp_token_scope ('https://1440666423386241.1.gcp.databricks.com') |
| storage_location | Google Cloud Storage location | Mandatory | storage_location ('gs://tdotf-dev/58eff1a8-e4a5-44a9-8847-1692a8cb154c') |
| project_id | Google Cloud Service Account project id | Mandatory | project_id ('tc-otf') |
| client_id | Google Cloud Service Account client id | Mandatory | client_id ('110337432395221507455') |
| client_email | Google Cloud Service Account client email | Mandatory | client_email ('sa-tc-otf-000@tc-otf.iam.gserviceaccount.com') |
Example
Replace google_cloud_service_account_client_email and google_cloud_service_account_private_key variables with the client email and private key from your Google Cloud Service Account.
REPLACE AUTHORIZATION gcp_idp_auth
USER '<<google_cloud_service_account_private_client_email>>'
PASSWORD '<<google_cloud_service_account_private_key>';
REPLACE DATALAKE iceberg_unity_gcp_idp_rest
EXTERNAL SECURITY CATALOG gcp_idp_auth ,
EXTERNAL SECURITY STORAGE gcp_idp_auth
USING
catalog_type ('rest')
rest_catalog_type('unity')
catalog_location ('https://1440666423386241.1.gcp.databricks.com/api/2.1/unity-catalog/iceberg-rest')
catalog_name ('reg_iceberg_unity_gcp')
idp_type('gcp_service_account')
storage_location ('gs://tdotf-dev/58eff1a8-e4a5-44a9-8847-1692a8cb154c')
project_id ('tc-otf')
client_id ('110337432395221507455')
client_email ('sa-tc-otf-000@tc-otf.iam.gserviceaccount.com')
TABLE FORMAT iceberg;