Filter External Parquet Data From a Foreign Table | NOS | Teradata Vantage - Filtering External Parquet Data From a Foreign Table - Advanced SQL Engine - Teradata Database

Teradata Vantageā„¢ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.05
Published
January 2021
Language
English (United States)
Last Update
2021-01-22
dita:mapPath
zws1595641486108.ditamap
dita:ditavalPath
hoy1596145193032.ditaval
dita:id
B035-1214
lifecycle
previous
Product Category
Software
Teradata Vantage
Parquet tables don't have a payload column. The user creates a foreign table and maps the Parquet logical data type to the corresponding Teradata data type.

The example shows filtering based on the Parquet data received from the object store. All the Parquet data from the object store at the foreign table location will be brought into the database and then filtered.

The examples and results show a sample river flow data set. To use your own data, replace the table and column names, and authorization object.

  1. To run NOS-related commands, log on to the database as a user with the required privileges.
  2. Create a foreign table to access external Parquet data:
    CREATE FOREIGN TABLE riverflow_parquet_path
    , EXTERNAL SECURITY DEFINER TRUSTED DefAuth
    (
      Location VARCHAR(2048) CHARACTER SET UNICODE CASESPECIFIC
      , GageHeight2 DOUBLE PRECISION FORMAT '-ZZZ9.99'
      , Flow DOUBLE PRECISION FORMAT '-ZZZZ9.99'
      , site_no BIGINT
      , datetime VARCHAR(16) CHARACTER SET UNICODE CASESPECIFIC
      , Precipitation DOUBLE PRECISION FORMAT '-ZZZ9.99'
      , GageHeight DOUBLE PRECISION FORMAT '-ZZZ9.99'
    )
    USING (
        LOCATION ('YOUR-STORAGE-ACCOUNT')
        PATHPATTERN ('$data/$siteno/$year/$month/$day')
        STOREDAS ('PARQUET')
    ) NO PRIMARY INDEX
    , PARTITION BY COLUMN;

    See Variable Substitutions for Examples for the credentials and location values for the sample river flow data set.

  3. Filter on a specific column in the Parquet data from external storage using the foreign table.
    SELECT TOP 2 GageHeight TheGageHeight
    , Flow TheFlow
    FROM riverflow_parquet_path
    WHERE site_no = '09394500';

    Result:

    TheGageHeight    TheFlow
    -------------  ---------
             5.56      31.20
             4.68       0.00