Previewing the Parquet Schema | Native Object Store ( NOS ) | Teradata Vantage - 17.10 - Previewing the Parquet Schema Using READ_NOS - Advanced SQL Engine - Teradata Database

Teradata Vantage™ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.10
Release Date
July 2021
Content Type
Programming Reference
Publication ID
B035-1214-171K
Language
English (United States)
To view the schema of your Parquet data, use the READ_NOS table operator with the RETURNTYPE('NOSREAD_SCHEMA') or the RETURNTYPE('NOSREAD_PARQUET_SCHEMA') clause.

The examples and results show a sample river flow data set. To use your own data, replace the table and column names, and authorization object.

  1. To run NOS-related commands, log on to the database as a user with the required privileges.
  2. View the schema of the Parquet data:
    SELECT * FROM  (
    LOCATION='YOUR-OBJECT-STORE-URI'
    AUTHORIZATION=authorization_object
    RETURNTYPE='NOSREAD_SCHEMA'
    ) AS d;
    

    See Variable Substitutions for Examples for the credentials and location values for the sample river flow data set.

    Your result will be similar to the following:

    ColPosition      1
            Name GageHeight2
    TdatDataType FLOAT
    PhysicalType DOUBLE
     LogicalType NONE
       Precision           0
           Scale           0
       MinLength           0
       MaxLength           0
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/
    
     ColPosition      2
            Name Flow
    TdatDataType FLOAT
    PhysicalType DOUBLE
     LogicalType NONE
       Precision           0
           Scale           0
       MinLength           0
       MaxLength           0
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/
    [...]

    The output is displayed vertically for readability.

    Or, you can view the schema using NOSREAD_PARQUET_SCHEMA, if you are using the FULLSCAN option. The FULLSCAN option is used to scan columns of variable length types to get the maximum length:

    SELECT * FROM  (
    LOCATION='YOUR-OBJECT-STORE-URI'
    AUTHORIZATION=authorization_object
    FULLSCAN='TRUE'
    RETURNTYPE='NOSREAD_PARQUET_SCHEMA'
    ) AS d;

    Your result will be similar to the following:

     ColPosition      1
            Name GageHeight2
    TdatDataType FLOAT
    PhysicalType DOUBLE
     LogicalType NONE
       Precision           0
           Scale           0
       MinLength           0
       MaxLength           0
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/09394500/2018/06/27.parquet
    
    ColPosition      1
            Name GageHeight2
    TdatDataType FLOAT
    PhysicalType DOUBLE
     LogicalType NONE
       Precision           0
           Scale           0
       MinLength           0
       MaxLength           0
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/09394500/2018/06/28.parquet
     ColPosition      1
    
    [...]
    
     ColPosition      4
            Name datetime
    TdatDataType VARCHAR
    PhysicalType BYTE_ARRAY
     LogicalType UTF8
       Precision           0
           Scale           0
       MinLength          16
       MaxLength          16
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/09394500/2018/06/27.parquet
    
    ColPosition      4
            Name datetime
    TdatDataType VARCHAR
    PhysicalType BYTE_ARRAY
     LogicalType UTF8
       Precision           0
           Scale           0
       MinLength          16
       MaxLength          16
       NullFound    1
        FileType parquet
        Location /S3/s3.amazonaws.com/td-usgs-public/PARQUETDATA/09394500/2018/06/28.parquet
     

    The output is displayed vertically for readability.