Parquet Format with Native Object Store | NOS | Teradata Vantage - 17.10 - Parquet Format - Advanced SQL Engine - Teradata Database

Teradata Vantage™ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.10
Release Date
July 2021
Content Type
Programming Reference
Publication ID
B035-1214-171K
Language
English (United States)

When reading external Parquet data, the maximum Parquet page size supported is 16 MB.

Using NOS, the maximum record size is 16,776,192 bytes.
  • If your record consists of all character data, these are the limitations for each character set:
    • For UNICODE, 16,776,192 bytes is equivalent to 8,388,096 characters.
    • For LATIN, 16,776,192 bytes is equivalent to 16,776,192 characters.

If some of data in the record is binary, the maximum number of characters is proportionately reduced.

Parquet tables don't have a payload column. The user creates a foreign table and maps the Parquet logical data type to the corresponding Teradata data type.

Redshift is the supported format for the manifest files.

Parquet format limitations:
  • READ_NOS can be used to view the Parquet schema, using RETURNTYPE('NOSREAD_SCHEMA'). This is helpful in creating the foreign table when you do not know the schema of your Parquet data beforehand.
  • Certain complex data types are not supported, including STRUCT, MAP, LIST, and ENUM.
  • Because support for the STRUCT data type is not available, nested Parquet object stores cannot be processed by NOS.

The following examples use external Parquet data in this format:

message schema {
  optional double GageHeight2;
  optional double Flow;
  optional int64 site_no;
  optional binary datetime (UTF8);
  optional double Precipitation;
  optional double GageHeight;
}

For supported Parquet formats, see Parquet External Files.