When reading external Parquet data, the maximum Parquet page size supported is 16 MB.
Using NOS, the maximum record size is 16,776,192 bytes.
- If your record consists of all character data, these are the limitations for each character set:
- For UNICODE, 16,776,192 bytes is equivalent to 8,388,096 characters.
- For LATIN, 16,776,192 bytes is equivalent to 16,776,192 characters.
If some of data in the record is binary, the maximum number of characters is proportionately reduced.
Unlike CSV and JSON, Parquet tables don't have a payload column. The user creates a foreign table and maps the Parquet logical data type to the corresponding Teradata data type.
Parquet format limitations:
- The READ_NOS table operator does not support Parquet. However, READ_NOS can be used to view the Parquet schema, using RETURNTYPE('NOSREAD_PARQUET_SCHEMA'). This is helpful in creating the foreign table when you do not know the schema of your Parquet data beforehand.
- Certain complex data types are not supported, including STRUCT, MAP, LIST, and ENUM.
- Because support for the STRUCT data type is not available, nested Parquet object stores cannot be processed by NOS.
The following examples use external Parquet data in this format:
message schema {
optional double GageHeight2;
optional double Flow;
optional int64 site_no;
optional binary datetime (UTF8);
optional double Precipitation;
optional double GageHeight;
}
For supported Parquet formats, see Parquet External Files.