17.05 - Example: Using the MANIFESTFILE Option - Teradata Database

Teradata Vantageā„¢ - Native Object Store Getting Started Guide

prodname
Advanced SQL Engine
Teradata Database
vrm_release
17.05
created_date
January 2021
category
Programming Reference
featnum
B035-1214-175K

Create a manifest file listing the paths of the objects stored on external storage using the MANIFESTFILE option. MANIFESTFILE creates a manifest file on external object store at the location specified.

  1. Write all rows matching the criteria to external object store and create a manifest file listing the paths of the objects stored on external storage:
    SELECT NodeId, AmpId, Sequence, ObjectName, ObjectSize, RecordCount
    FROM WRITE_NOS_FM (
      ON  ( SELECT * FROM RiverFlowPerm WHERE DateTime = (TIMESTAMP '2018-07-01 00:00:00') )
      USING
        LOCATION('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/')
        STOREDAS('PARQUET')
        MANIFESTFILE('YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json')
    ) AS d 
    ORDER BY AmpId;

    Replace the LOCATION of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/ with the URI to the external object store where you want to write the data.

    Replace the MANIFESTFILE location of YOUR-STORAGE-ACCOUNT/20180701/ManifestFile/manifest1.json with the URI to the manifest file on external object store.

    You can view the manifest file on external storage using the command line of your external object store.

    The entries stored in the manifest file are similar to this:

    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_6_1.parquet","meta":{"content_length":2733}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2725}}]
    }