Overview - Aster Analytics

Teradata AsterĀ® Spark Connector User Guide

Product
Aster Analytics
Release Number
7.00.00.01
Published
May 2017
Language
English (United States)
Last Update
2018-04-13
dita:mapPath
dbt1482959363906.ditamap
dita:ditavalPath
Generic_no_ie_no_tempfilter.ditaval
dita:id
dbt1482959363906
lifecycle
previous
Product Category
Software
You invoke the RunOnSpark function in a SQL-MapReduce query:
SELECT select_list FROM RunOnSpark (ON ...);
The query runs on the Aster cluster, in parallel with the Aster vworkers. The vworkers communicate with Spark, which they treat as an external processing engine. The query execution procedure is:
  1. The SQL-MapReduce instances synchronize themselves.
  2. Aster Database starts the Spark job at runtime by submitting a request to the Spark manager.
  3. Aster Database transfers the data for the job to the Spark manager.
  4. Spark processes the data.
  5. Spark sends the results to the SQL-MapReduce instances.
  6. Aster Database processes the results.
The following figure shows how Aster Database and Spark communicate. Aster Database and Spark can be on the same cluster or on different clusters.

Aster Queen Node User Session Aster Worker Node Driver Program AsterVWorker AsterVWorker Task Spark Context Task Aster Worker Node AsterVWorker AsterVWorker SQL/MR SQL/MR SQL/MRContext Cluster Manager Cluster Manager SQL/MR SQL/MR Task Task Cache In/OutPort In/OutPort In/OutPort In/OutPort Spark Worker Node Spark Worker Node Executor Executor Cache