Using COPY with Columnar Tables - Aster Client

Teradata Aster® Client Guide

Product
Aster Client
Release Number
7.00
Published
May 2017
Language
English (United States)
Last Update
2018-04-13
dita:mapPath
hki1475000360386.ditamap
dita:ditavalPath
Generic_no_ie_no_tempfilter.ditaval
dita:id
B700-2005
lifecycle
previous
Product Category
Software
Columnar tables are not supported on a system running Aster Execution Engine.

A loading operation using the Aster Loader Tool, COPY, or INSERT can be expensive when the following conditions exist:

  • the target table uses columnar storage, AND
  • the target table has many logical partitions, AND
  • the loaded data matches many different logical partitions.

In this case, the memory allocated to perform the load may be as large as the amount of source data. To avoid high memory requirements, it is best to divide a large load into batches. There are two alternative approaches:

  • Ensure that each batch only loads a small number of logical partitions in the columnar table. For example, when inserting data into a columnar table with weekly partitions, each batch may insert data for a single month.
  • Ensure that the size of each batch is a small fraction of system memory available at the worker nodes. This should only be done if the data being loaded into the columnar table has a mixture of records matching many different logical partitions. As an example, suppose that a year's worth of data is being loaded into a columnar fact table with weekly partitions, and the cluster has four physical worker nodes with 100GB of system memory in each node. Loading data in 40GB batches will use 10GB memory per physical worker, which is 10% of the overall available memory.