Adding Data to an IMDC with the API - Aster Execution Engine

Teradata Aster® Developer Guide

Product

Aster Execution Engine

Release Number

7.00.02

Published

July 2017

Language

English (United States)

Last Update

2018-04-13

dita:mapPath

xnl1494366523182.ditamap

dita:ditavalPath

Generic_no_ie_no_tempfilter.ditaval

dita:id

ffu1489104705746

lifecycle

Product Category

Software

When an IMDC is in the appendable state, you can add data to it and finalize it. Finalizing the IMDC puts it in the iterable state. After finalizing the IMDC, you can no longer add data to it.

You add data to an IMDC with the RowEmitter interface of the SQL-MapReduce framework.

You finalize an IMDC with the method InMemoryDataCollection::finalizeCollection().

You cannot add data to an IMDC during contract negotiation.

Add data to an appendable IMDC and finalize it:

The finalization step in the following code is for a SQL-MapReduce row function. For a SQL-MapReduce partition function, replace that step with the finalization code in Finalization Code for a Partition Function.

import java.util.List;
import java.util.ArrayList;
import com.asterdata.ncluster.sqlmr.RowFunction;
import com.asterdata.ncluster.sqlmr.data.RowEmitter;
import com.asterdata.ncluster.sqlmr.data.RowIterator;
import com.asterdata.ncluster.sqlmr.InMemoryDataCollectionRepository;
import com.asterdata.ncluster.sqlmr.data.InMemoryDataCollection;
import com.asterdata.ncluster.sqlmr.data.SqlType;
import com.asterdata.ncluster.sqlmr.data.RowView;
import com.asterdata.ncluster.sqlmr.data.RowBuilder;
...
  public void operateOnSomeRows(RowIterator inputIterator,
  outputEmitter)
  {
    ...
    //Get handle to RowEmitter
    RowEmitter IMDC_rowAppender_var = IMDC_name.getRowAppender();

    //Add input rows to IMDC
    while ( inputIterator.advanceToNextRow() )
      {
         //Construct row
         IMDC_rowAppender_var.addtype_0(inputIterator.gettype_0At(0));
         IMDC_rowAppender_var.addtype_1(inputIterator.gettype_1At(1));
         ...
         IMDC_rowAppender_var.addtype_n(inputIterator.gettype_nAt(n));
           
         //Add row
         IMDC_rowAppender_var.emitRow();
      }

    //Finalize IMDC (for a row function)
    IMDC_name_var.finalizeCollection();
      ...
  }

For example, the following code adds rows to the IMDC named myIMDC, which has columns of types integer, double, and var.

import java.util.List;
import java.util.ArrayList;
import com.asterdata.ncluster.sqlmr.RowFunction;
import com.asterdata.ncluster.sqlmr.data.RowEmitter;
import com.asterdata.ncluster.sqlmr.data.RowIterator;
import com.asterdata.ncluster.sqlmr.InMemoryDataCollectionRepository;
import com.asterdata.ncluster.sqlmr.data.InMemoryDataCollection;
import com.asterdata.ncluster.sqlmr.data.SqlType;
import com.asterdata.ncluster.sqlmr.data.RowView;
import com.asterdata.ncluster.sqlmr.data.RowBuilder;
...
  public void operateOnSomeRows(RowIterator inputIterator,
  outputEmitter)
  {
    ...
    //Get handle to RowEmitter
    RowEmitter myIMDC_rowAppender = myIMDC.getRowAppender();

    //Add input rows to IMDC
    while ( inputIterator.advanceToNextRow() )
      {
         //Construct row
         myIMDC_rowAppender.addInt(inputIterator.getIntAt(0));
         myIMDC_rowAppender.addDouble(inputIterator.getDoubleAt(1));
         myIMDC_rowAppender.addString(inputIterator.getStringAt(2));
           
         //Add row
         myIMDC_rowAppender.emitRow();
      }

    //Finalize IMDC
    myIMDC.finalizeCollection();
      ...
  }

The IMDC is now in the iterable state. You can iterate over its data, but you cannot add data to it.