Associating an IMDC with an Output Iterator with the API

Associating an IMDC with an Output Iterator with the API - Aster Execution Engine

Teradata Aster® Developer Guide

Product

Aster Execution Engine

Release Number

7.00.02

Published

July 2017

Language

English (United States)

Last Update

2018-04-13

dita:mapPath

xnl1494366523182.ditamap

dita:ditavalPath

Generic_no_ie_no_tempfilter.ditaval

dita:id

ffu1489104705746

lifecycle

Product Category

Software

During contract negotiation, you can associate an IMDC with an output iterator. Then, when an output target is associated with the IMDC, the output iterator is provided to data-producing methods. When the methods finish, the IMDC is finalized. Associating an IMDC with an output iterator is an easy way to integrate the use of the IMDC with a SQL-MapReduce function.

Associating an IMDC with an output iterator also helps with some optimizations. When the IMDC exists:

Data-producing methods associated with the IMDC (such as operateOnSomeRows, operateOnPartition, operateOnMultipleInputs and drainOutputRows) are skipped.
If the data in the IMDC can be consumed by means of an input iterator, the input iterator is passed to the SQL-MapReduce function; the IMDC data is not put into the outputEmitter.
When a SQL-MapReduce function uses the IMDC, function input is replaced with the empty table NC_SYSTEM.NC_DUMMY, avoiding the cost of scanning the input. This optimization is especially helpful when the query that calls the function is issued repeatedly.

Create the OutputInfo object, using the appropriate constructor:
- To use the default IMDC memory size:
```
public OutputInfo(
  List<ColumnDefinition> columns,
  String imdcToPopulate,
 boolean refreshIMDC)
```
- To specify the IMDC memory size:
```
public OutputInfo(
  List<ColumnDefinition> columns,
  String imdcToPopulate,
  int imdcInMemSizeMB, boolean refreshIMDC)
```
The refreshIMDC parameter specifies whether the OutputInfo object drops and recreates the IMDC if it exists. If refreshIMDC is false, the OutputInfo object uses the existing IMDC to produce the output.

Associate an IMDC with an output iterator during contract negotiation:

public my_sqlmr(RuntimeContract contract)
  {
    //Output information is taken directly from input information.

      InputInfo inputInfo = contract.getInputInfo();
      
      output_iterator_name = 
        contract.useArgumentClause("IMDC_name").getSingleValue();

      List<ColumnDefinition> imdcColDefs =
        InMemoryDataCollectionRepository.getInMemoryDataCollectionColDefs(
          output_iterator_name);

      int imdcInMemSizeInMB = -1;

      if(contract.hasArgumentClause("imdcInMemSizeInMB"))
      {
         imdcInMemSizeInMB = Integer.parseInt(contract.useArgumentClause("imdcInMemSizeInMB").getSingleValue());
      }
      imdcCols_ = inputInfo.getColumns();
      imdcInMemSizeInMB_ = imdcInMemSizeInMB;
      contract.setOutputInfo(
       new OutputInfo( imdcCols_,
       output_iterator_name,
       imdcInMemSizeInMB_, false));
        contract.setUsesSessionScopedJVM();
        contract.complete();
   }
 
   public void operateOnSomeRows(
         RowIterator inputIterator,
         RowEmitter outputEmitter
         )
   {            
      while ( inputIterator.advanceToNextRow() )
      {
         outputEmitter.addFromRow(inputIterator);
         //data is being emitted to imdc:output_iterator_name
         outputEmitter.emitRow();
      }      
   }

The bold command associates the output iterator with the IMDC.