We use the following terms in the text that follows:
Aster loader node: In the cluster, a loader node is a node dedicated to data loading. Many loader nodes can operate in parallel. Note that the Aster Loader node is not supported by systems running the Aster Execution Engine.
Aster Loader Tool (ncluster_loader) is the client application for initiating high-speed bulk loads.
Aster Database Load Error Logging is a feature in ncluster_loader that allows you to perform loading that is more tolerant of poorly formatted input data. Load Error Logging sends malformed rows to an error logging table.
Data staging node(s): Nodes where all the input data is present. In a typical setup the input data is stored on the local filesystem. However, other use cases where all the data is stored on a network-mounted storage device are possible.
Row(s): In any given input data file individual rows are present. The used format for the input data describes where row boundaries are. Usually a “row” refers to a logical unit of information that needs to be stored as a unit in the data warehouse. One example for rows include call records which consist of source and destination phone numbers, call duration time, and so on.
The following section highlights three different data loading scenarios and the respective best practices that should be performed to make the process as a whole as seamless as possible.