Example of Datalake Definition using Databricks Managed Service Principle for GCP - Teradata Vantage

The following example shows how to create a DeltaLake DATALAKE that using Databricks Managed Service Principle for the catalog connection and an GCP IDP managed principal for storage access.

Define the authorization for Catalog:

CREATE AUTHORIZATION gcs_iceberg_dbx_auth
     USER '<<dbx_client_id>>'
     PASSWORD '<<dbx_client_secret_key>>';

Define authorization for Storage access:

CREATE AUTHORIZATION gcs_iceberg_idp_auth
     USER '<<gcp_service_account_private_key_id>>'
     PASSWORD '<<gcp_service_account_private_key>>';

Create a DeltaLake DATALAKE object referencing the two AUTH objects:

CREATE DATALAKE unity_iceberg_write_01
EXTERNAL SECURITY INVOKER TRUSTED CATALOG gcs_iceberg_dbx_auth ,
EXTERNAL SECURITY INVOKER TRUSTED STORAGE gcs_iceberg_idp_auth
USING
catalog_type ('unity')
catalog_location ('https://1440666423386241.1.gcp.databricks.com/api/2.1/unity-catalog/iceberg')
unity_catalog_name ('reg_iceberg_unity_gcp')
storage_location ('gs://tdotf-dev')
project_id ('tc-otf')
client_id ('110337432395221507455')
client_email ('sa-tc-otf-000@tc-otf.iam.gserviceaccount.com')
default_cluster_id ('5410-181833-mgt9s3pw')
TABLE FORMAT iceberg ;