Example - Aster Analytics

Teradata AsterĀ® Spark Connector User Guide

Product
Aster Analytics
Release Number
7.00.00.01
Published
May 2017
Language
English (United States)
Last Update
2018-04-13
dita:mapPath
dbt1482959363906.ditamap
dita:ditavalPath
Generic_no_ie_no_tempfilter.ditaval
dita:id
dbt1482959363906
lifecycle
previous
Product Category
Software

This excerpt of an execution trace of the generated script shows the effect of steps a through h in Executing the Teradata Aster Spark Queen Installation/Configuration Script.

./AsterSpark_queen_host_name_queen.sh
...
Aster vworkers execute RunOnSpark sql/mr queries as user extensibility.
For vworkers to use open ssh for spark-submit, user extensibility must have rsa key files.
We will now create new rsa key files.
Generating public/private rsa key pair.
Your identification has been saved in /home/beehive/.ssh/IDENTITYFILE.
Your public key has been saved in /home/beehive/.ssh/IDENTITYFILE.pub.
...
And Aster vworkers must be granted passwordless open ssh access to spark-cdh55-1-namenode.asterdata.com.
sparkJobSubmitter@spark-cdh55-1-namenode.asterdata.com's password:
...
Any Aster vworker can be used to spark-submit.
So, each vworker must have a copy of this IDENTITYFILE.
For security, we must limit access to IDENTITYFILE.
So, we make it accessible only by its owner: extensibility.
....
See /primary/logs/SparkConnector/sparkConnectorDeploy.out for details.
Successfully wrote /home/beehive/config/AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json.
Successfully set spark-params for AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json.
Successfully set spark-properties for AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json.
Successfully set run-on-spark for AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json.
Building the spark.config
=========================
spark.config generated
=======================
Successfully set default spark config name to AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json-socket-SSh.
remove and install RunOnSpark
create verification schema
create and populate verification table
running sanity check query for spark_cluster_id( AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json-file-SSh )...
PASSED.
running sanity check query for spark_cluster_id( AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json-socket-SSh )...
PASSED.
running analytics/correlation.sql...
correlation.sql PASSED.
running analytics/als.sql...
als.sql PASSED.
running configuration/configuration_test.sql...
configuration_test.sql PASSED.
Successfully set default spark config name to AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json-socket-SSh.
running empty_partitions/empty_partitions_test.sql...
empty_partitions_test.sql PASSED.
Successfully set default spark config name to AsterSpark_spark-cdh55-1-namenode.asterdata.com_site.json-file-SSh.
running empty_partitions/empty_partitions_test.sql...
empty_partitions_test.sql PASSED.
...
Tests summary:
52 tests PASSED.
0 tests FAILED.