Data Storage | VantageCloud Lake - Data Storage - Teradata Vantage

Teradata® VantageCloud Lake
Deployment
VantageCloud
Edition
Lake
Product
Teradata Vantage
Published
January 2023
ft:locale
en-US
ft:lastEdition
2024-12-11
dita:mapPath
phg1621910019905.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
phg1621910019905
Choose VantageCloud Lake options for storing data depending on cost and performance requirements:
Storage Option Storage Type Benefits and Considerations
Object File System Object Storage
  • Storage integrated with VantageCloud Lake
  • Lower cost, but higher latency
  • Reduces the cost of storing large tables
  • Optimal for data that support less time-sensitive analytics, long-running queries, or tables that are the object of ETL processes
  • Supports scaling and isolating workloads
External Object Storage Accessed Using Native Object Store (NOS) Object Storage
  • Storage maintained outside of VantageCloud Lake where users can store external data
  • Data is accessed by VantageCloud Lake using Native Object Store
Block Storage on the primary cluster Persistent block-level storage
  • Higher cost, but lower latency
  • Optimal for workloads with time-sensitive tactical or short SLA requests
Open Table Format Datalakes  
  • Low-cost and flexible and provide the following data warehouse-like properties:
    • Efficient data access and management
    • Guaranteed ACID transactions
    • Schema evolution
    • Partition evolution
    • Time travel
You can convert a Block File System table to an Object File System table using ALTER TABLE Syntax (Storage Conversion Form). To convert an Object File System table to a Block File System table, use CREATE TABLE AS Syntax (Object File System Table Form), specifying WITH NO DATA.

Object File System

Usually, the best place to store your data is in the Object File System. This object storage is integrated with VantageCloud Lake and provides these benefits:
  • Reduce cost of ownership

    The Object File System uses lower cost object storage provided by public cloud vendors. You can move data tables that are large, or that support analytics and long-running queries, or tables that are the object of ETL processes into the Object File System to lower the cost of storage.

  • Enable the use of the compute cluster architecture to run queries

    Data tables in the Object File System are accessible to the primary cluster and all compute clusters so you can scale and isolate workloads.

  • Enable blue/green upgrades with less replication

For additional benefits and considerations in using the Object File System, see Object File System.

External Object Storage Accessed Using Native Object Store

Users create and manage this object storage from public cloud vendors themselves. Users can access this external data, such as IOT data, from VantageCloud Lake by using Native Object Store functionality (READ_NOS, WRITE_NOS, and foreign table SQL).

See the following documents for how to access this data using the Native Object Store capabilities:

Block Storage

Block Storage is used for AMP storage to store user perm data and database data, such as dictionary tables, on the primary cluster. Block Storage is low-latency block-level storage suitable for time-sensitive tactical or short SLA requests.

Best Practices for Storing Data

Teradata recommends prioritizing moving user tables, especially large tables, from the primary cluster into the Object File System to reduce storage cost and to take advantage of the compute cluster architecture.

Considerations for administrators when determining where to store data:
  • Balancing cost with acceptable performance
  • Optimizing the division of work between the primary cluster and compute clusters
  • Object File System Considerations
  • Designing tables depending on whether data is stored in the Object File System or in external object storage referenced by foreign tables and Native Object Store capabilities