Example: Using the MANIFESTFILE Option - Teradata Vantage

Teradata® VantageCloud Lake

Deployment
VantageCloud
Edition
Lake
Product
Teradata Vantage
Published
January 2023
Language
English (United States)
Last Update
2024-04-03
dita:mapPath
phg1621910019905.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
phg1621910019905

Create a manifest file listing the paths of the objects stored on external object storage using the MANIFESTFILE option. MANIFESTFILE creates a manifest file on external object storage at the location specified.

  1. Write all rows matching the criteria to external object storage and create a manifest file listing the paths of the objects stored on external object storage:
    SELECT * FROM WRITE_NOS (
    ON ( SELECT * FROM RiverFlowPerm WHERE DateTime = '2018-06-27 00:00:00' ) USING
    LOCATION('YOUR-OBJECT-STORE-URI80627/ManifestFile/')
    AUTHORIZATION(MyAuthObj)
    STOREDAS('PARQUET')
    MANIFESTFILE('YOUR-OBJECT-STORE-URI/20180627/ManifestFile/manifest1.json')
    ) AS d;
    

    Replace the LOCATION of YOUR-OBJECT-STORE-URI/20180701/ManifestFile/ with the URI to the external object storage where you want to write the data.

    Replace the MANIFESTFILE location of YOUR-OBJECT-STORE-URI/20180701/ManifestFile/manifest1.json with the URI to the manifest file on external object storage.

    Your result is similar to the following:

         NodeId          33
          AmpId           0
       Sequence                    1
     ObjectName /S3/s3.amazonaws.com/iewritenostest/20180627/ManifestFile/object_33_0_1.parquet
     ObjectSize                 1327
    RecordCount                    3
    
         NodeId          33
          AmpId           3
       Sequence                    1
     ObjectName /S3/s3.amazonaws.com/iewritenostest/20180627/ManifestFile/object_33_3_1.parquet
     ObjectSize                 1346
    RecordCount                    3
    [...]
  2. You can view the manifest file on external object storage using the command line of your external object storage.

    The entries stored in the manifest file are similar to this:

    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_6_1.parquet","meta":{"content_length":2733}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
               {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2725}}]
    }