AsterSparkFunction is the abstract base class for all wrapper functions to be extended to use with the RunOnSpark function. A function developed on the Spark side to work with Aster must derive from one of these AsterSparkFunction subclasses:
Run Method Signature
run (input: RDD[DataRow], sparkFunctParams: String)
Parameters
- input
- Reads data from the source (socket or file) and persists it, using the type of persistence that you specify when submitting the Aster query.
- sparkFunctParams
- String representing the parameters specific to the function you are implementing.
Methods
Method | Description |
---|---|
run | Main method, which you can override to implement the desired behavior for the function. |
getSparkContext | Helper method. Returns a pointer to the Spark context object. |
getAsterContext() | Helper method. Returns the Aster context. |
getInputTypes() | Helper method. Returns an array that contains the input column data types. |
getOutputTypes() | Helper method. Returns an array that contains the output column data types. |
getInColNames() | Helper method. Returns an array that contains the input column names. |
getOutColNames() | Helper method. Returns an array that contains the output column names. |