The Correlations class defines a wrapper function that uses the Aster Spark API and implements the running phase of the Spark MLlib - Basic Statics Correlations algorithm. The Spark Correlations function supports Pearson's and Spearman's correlations.
Run Method Signature
run (input: RDD(DataRow), SparkFunctParams: String): RDD(DataRow)
Input
Two columns with numeric data types.
Output
Correlations between the two values in each row (DOUBLE PRECISION values).
Invocation Example
SELECT * FROM RunOnSpark (
ON (
SELECT x, y FROM corr
)
SPARKCODE ('com.teradata.aster.functions.Correlations')
OUTPUTS ('corr double precision')
);
Version
Spark 1.3 and later.