Native Object Store (NOS) is a new capability included with Vantage that makes it easy for users to explore datasets that have been stored in JSON, comma-separated values (CSV), or Parquet format, located on external object stores like AWS S3 and Azure Blob Storage, using standard SQL. Because the data is being processed and analyzed inside the database, Native Object Store can take advantage of the scalability, performance, and workload management that is part of the Vantage platform. No special object storage-side compute infrastructure is required to use Native Object Store. Simply create a Native Object Store table definition within the Advanced SQL Engine, point it at any AWS-S3 or Azure Blob Storage bucket that you are authorized to access, and within minutes, you can explore the data located in that bucket using all the analytics functionality of Vantage. As a result, Native Object Store is ideally suited for data scientists, analysts, and other business users that want to use object stores on an ad-hoc basis to store interim results, prior versions, and other data sets as part of their analytics workflow.
You can use tdplyr to explore this external data made available on Vantage via Native Object Store capability. This section shows how to use tdplyr to explore data stored on external objects with the help of Native Object Store. The first step is to create a foreign table. The foreign table allows the external data to be easily referenced within the Advanced SQL Engine and makes the data available in a structured relational format which could include complex data types. Once the data is in a relational format, either persistently or virtually, it can be aggregated or joined to other relational tables.
Refer to the Native Object Store documentation for more details about NOS.
The following sections show different ways to explore foreign table data in Vantage via tdplyr, based on the actual format of the data. Sample datasets for Native Object Store can be found at https://downloads.teradata.com/database/articles/native-object-store-example-dataset-download.