Example: Using MANIFESTFILE('file_name'), MANIFESTONLY('TRUE'), and OVERWRITE('TRUE') to Overwrite the Manifest File - Teradata VantageCloud Lake

Lake - Manage and Move Data
Deployment
VantageCloud
Edition
Lake
Product
Teradata VantageCloud Lake
Release Number
Published
February 2025
ft:locale
en-US
ft:lastEdition
2025-05-16
dita:mapPath
atx1683670417382.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
atx1683670417382

The following overwrites a manifest file on external object storage.

Create a database table containing manifest file values, then write the rows in the table to the manifest file on external object storage. The manifest file already exists on external object storage and is therefore overwritten, not appended to.

  1. If not already done, create a manifest table called ManifestTbl in the database to store the manifest values. See Create a Manifest File in the Database.
  2. Overwrite the manifest file with data from the table called ManifestTbl:
    SELECT *
FROM WRITE_NOS (
  ON  ( SELECT * FROM ManifestTbl )
  USING
    LOCATION('YOUR-OBJECT-STORE-URI/20180627/ManifestFile2/')
    AUTHORIZATION(MyAuthObj)
    STOREDAS('PARQUET')
    MANIFESTFILE('YOUR-OBJECT-STORE-URI/20180627/ManifestFile2/manifest2.json')
    MANIFESTONLY('TRUE')
    OVERWRITE('TRUE')
) AS d ;

    Replace the LOCATION of YOUR-OBJECT-STORE-URI/20180701/ManifestFile2/ with the URI to the external object storage location where you want to write the manifest file.

    Replace the MANIFESTFILE location of YOUR-OBJECT-STORE-URI/20180701/ManifestFile2/manifest2.json with the URI to the manifest file on external object storage.

    MANIFESTONLY('TRUE') writes only a manifest file to external object storage; no data objects are written:
    • Use this option to create a new manifest file if a WRITE_NOS operation fails due to a database stop or restart, or when network connectivity issues interrupt and stop a WRITE_NOS operation before all data has been written to external object storage.
    • The manifest is created from the table or query result set that is input to WRITE_NOS. The input must be a list of storage object names and sizes, with one row per object.

    OVERWRITE('TRUE') overwrites an existing manifest file. Note, OVERWRITE('TRUE') must be used with MANIFESTONLY('TRUE').

    Your result is similar to the following:

    ObjectName                                                       ObjectSize
------------------------------------------------------------------------- ----------
/S3/s3.amazonaws.com/iewritenostest/20180627/ManifestFile2/manifest2.json        433
  3. You can view the manifest file on external object storage using commands from the command line of your external object storage. The manifest file contains information similar to the following:
    {"entries":[{"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_0_1.parquet","meta":{"content_length":2803}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_2_1.parquet","meta":{"content_length":2725}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_6_1.parquet","meta":{"content_length":2733}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_7_1.parquet","meta":{"content_length":2591}},
           {"url":"s3://ie-writenos-bucket/20180701/ManifestFile/object_33_1_1.parquet","meta":{"content_length":3009}}]
}