Note on Iceberg String Support in Teradata OTF - Teradata Vantage

Apache Iceberg and Delta Lake Open Table Format on VantageCloud Lake Getting Started

Deployment
VantageCloud
Edition
Lake
Product
Teradata Vantage
Published
December 2024
ft:locale
en-US
ft:lastEdition
2025-01-03
dita:mapPath
bsr1702324250454.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
bsr1702324250454

The Iceberg STRING column type is mapped to Teradata VARCHAR character set UNICODE. Iceberg STRING column values can be large and does not have a maximum column length defined in the table schema. This makes it harder for TDOTF engine to do the datatype mapping with correct max column length.

To minimize data loss, by default, STRING columns are mapped to maximum supported VARCHAR UNICODE size of 32000. Using this default size of 32000 for smaller strings has shown to have a performance impact especially when used in a SORT BY or GROUP BY clause. Future releases of TD OTF will allow the default VARCHAR size of a Iceberg STRING to be tunable through a NVP setting - this can be used when the lengths of the STRING columns in the OTF dataset is known and can lead to better performance.