Efficient Retrieval of Parquet Data | Native Object Store | Teradata Vantage - 17.05 - Using Filtering to Retrieve Parquet Data More Efficiently - Advanced SQL Engine - Teradata Database

Teradata Vantageā„¢ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.05
Release Date
January 2021
Content Type
Programming Reference
Publication ID
B035-1214-175K
Language
English (United States)

Reading all the objects in a bucket can be time-consuming and expensive. An alternative solution is to filter the object store using one or both of the following methods:

Path Filtering
  • A path is a series of values that constitute a multi-level key (siteno, year, month, day) which identifies one or more objects within an object store.
  • Path filtering lets you pre-select which objects in the object store you want to bring back to the database, by providing specified values for different levels in a path key (such as "WHERE $path.$siteno = 09380000")
Column Filtering
  • Parquet supports the ability to apply filtering at the column level while the data is in the object store.
  • This can be as effective as path filtering in reducing the volume of data returned.