17.05 - Example: Using the MANIFESTFILE Option - Advanced SQL Engine - Teradata Database

Teradata Vantageā„¢ - Native Object Store Getting Started Guide

Product
Advanced SQL Engine
Teradata Database
Release Number
17.05
Release Date
January 2021
Content Type
Programming Reference
Publication ID
B035-1214-175K
Language
English (United States)

Create a manifest file listing the paths of the objects stored on external storage using the MANIFESTFILE option. MANIFESTFILE creates a manifest file on external object store at the location specified.

  1. Write all rows matching the criteria to external object store and create a manifest file listing the paths of the objects stored on external storage:
    SELECT NodeId, AmpId, Sequence, ObjectName, ObjectSize, RecordCount
    FROM WRITE_NOS_FM (
      ON  ( SELECT * FROM RiverFlowPerm WHERE DateTime = (TIMESTAMP '2018-07-01 00:00:00') )
      USING
        LOCATION('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/')
        STOREDAS('PARQUET')
        MANIFESTFILE('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json')
    ) AS d 
    ORDER BY AmpId;

    Replace the LOCATION of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/ with the URI to the external object store where you want to write the data.

    Replace the MANIFESTFILE location of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json with the URI to the manifest file on external object store.

    You can view the manifest file on external storage using the command line of your external object store.

    The entries stored in the manifest file are similar to this:

    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_6_1.parquet","meta":{"content_length":2733}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2725}}]
    }