Example: Using the MANIFESTFILE Option - Advanced SQL Engine - Teradata Database

Teradata Vantageā„¢ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.05
Published
January 2021
Language
English (United States)
Last Update
2021-01-22
dita:mapPath
zws1595641486108.ditamap
dita:ditavalPath
hoy1596145193032.ditaval
dita:id
B035-1214
lifecycle
previous
Product Category
Software
Teradata Vantage

Create a manifest file listing the paths of the objects stored on external storage using the MANIFESTFILE option. MANIFESTFILE creates a manifest file on external object store at the location specified.

  1. Write all rows matching the criteria to external object store and create a manifest file listing the paths of the objects stored on external storage:
    SELECT NodeId, AmpId, Sequence, ObjectName, ObjectSize, RecordCount
    FROM WRITE_NOS_FM (
      ON  ( SELECT * FROM RiverFlowPerm WHERE DateTime = (TIMESTAMP '2018-07-01 00:00:00') )
      USING
        LOCATION('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/')
        STOREDAS('PARQUET')
        MANIFESTFILE('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json')
    ) AS d 
    ORDER BY AmpId;

    Replace the LOCATION of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/ with the URI to the external object store where you want to write the data.

    Replace the MANIFESTFILE location of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json with the URI to the manifest file on external object store.

    You can view the manifest file on external storage using the command line of your external object store.

    The entries stored in the manifest file are similar to this:

    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_6_1.parquet","meta":{"content_length":2733}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2725}}]
    }