Example: Using MANIFESTFILE('file_name') and MANIFESTONLY('TRUE') - Teradata VantageCloud Lake

Lake - Manage and Move Data
Deployment
VantageCloud
Edition
Lake
Product
Teradata VantageCloud Lake
Release Number
Published
February 2025
ft:locale
en-US
ft:lastEdition
2025-05-16
dita:mapPath
atx1683670417382.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
atx1683670417382

The following creates a manifest file on external object storage.

First, create a database table containing manifest file values, then writes the rows in the table to a manifest file on external object storage.

  1. If not already done, create a manifest table called ManifestTbl in the database to store the manifest values. See Create a Manifest File in the Database.
  2. Insert data into ManifestTbl:
    INSERT INTO ManifestTbl VALUES ('YOUR-OBJECT-STORE-URI/20180627/ManifestFile/object_33_0_1.parquet', 2803);
INSERT INTO ManifestTbl VALUES ('YOUR-OBJECT-STORE-URI/20180627/ManifestFile/object_33_1_1.parquet', 3009);
INSERT INTO ManifestTbl VALUES ('YOUR-OBJECT-STORE-URI/20180627/ManifestFile/object_33_2_1.parquet', 2733);
INSERT INTO ManifestTbl VALUES ('YOUR-OBJECT-STORE-URIt/20180627/ManifestFile/object_33_3_1.parquet', 2591);
  3. Write the data in ManifestTbl to the manifest file on external object storage:
    SELECT *
FROM WRITE_NOS (
  ON  ( SELECT * FROM ManifestTbl )
  USING
    LOCATION('YOUR-OBJECT-STORE-URI/20180627/ManifestFile2/')
    AUTHORIZATION(MyAuthObj)
    STOREDAS('PARQUET')
    MANIFESTFILE('/YOUR-OBJECT-STORE-URI/20180627/ManifestFile2/manifest2.json')
    MANIFESTONLY('TRUE')
) AS d;

    Replace the LOCATION of YOUR-OBJECT-STORE-URI/20180701/ManifestFile2/ with the URI to the external object storage location where you want to write the manifest file.

    Replace the MANIFESTFILE location of YOUR-OBJECT-STORE-URI/20180701/ManifestFile2/manifest2.json with the URI to the manifest file on external object storage.

    MANIFESTONLY('TRUE') writes only a manifest file to external object storage; no data objects are written:
    • Use this option to create a new manifest file in the event that a WRITE_NOS operation fails due to a database stop or restart, or when network connectivity issues interrupt and stop a WRITE_NOS operation before all data has been written to external object storage.
    • The manifest is created from the table or query result set that is input to WRITE_NOS. The input must be a list of storage object names and sizes, with one row per object.

    Your result is similar to the following:

    ObjectName                                                       ObjectSize
------------------------------------------------------------------------- ----------
/S3/s3.amazonaws.com/iewritenostest/20180627/ManifestFile2/manifest2.json        433
  4. You can view the manifest file on external object storage using commands from the command line of your external object storage. The manifest file contains information similar to the following:
    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2733}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}}]
}