Using teradataml with Native Object Store - 17.00 - Using teradataml with Native Object Store - Teradata Package for Python

Teradata® Package for Python User Guide

Product
Teradata Package for Python
Release Number
17.00
Release Date
November 2021
Content Type
User Guide
Publication ID
B700-4006-070K
Language
English (United States)

Native Object Store (NOS) is a new capability included with Teradata Vantage that makes it easy for users to explore datasets that have been stored in JSON, comma-separated values (CSV), or Parquet format, located on external object stores like AWS S3 and Azure Blob Storage, using standard SQL. Because the data is being processed and analyzed inside the database, Native Object Store can take advantage of the scalability, performance, and workload management that is part of the Vantage platform. No special object storage-side compute infrastructure is required to use Native Object Store. Simply create a Native Object Store table definition within the Advanced SQL Engine, point it at any AWS-S3 or Azure Blob Storage bucket that you are authorized to access, and within minutes, you can explore the data located in that bucket using all the analytics functionality of Vantage. As a result, Native Object Store is ideally suited for data scientists, analysts, and other business users that want to use object stores on an ad-hoc basis to store interim results, prior versions, and other data sets as part of their analytics workflow.

You can use teradataml to explore this external data made available on Vantage via Native Object Store capability. This section shows how to use teradataml to explore data stored on external objects with the help of Native Object Store. The first step is to create a foreign table. The foreign table allows the external data to be easily referenced within the Advanced SQL Engine and makes the data available in a structured relational format which could include complex data types. Once the data is in a relational format, either persistently or virtually, it can be aggregated or joined to other relational tables.

Refer to the Native Object Store documentation for more details about NOS.

The following sections show different ways to explore foreign table data in Vantage via teradataml, based on the actual format of the data. Sample datasets for Native Object Store can be found at https://downloads.teradata.com/database/articles/native-object-store-example-dataset-download.