This section explains how Advanced SQL Engine and Machine Learning Engine work together to process a SQL request.
SELECT * FROM kMeans ( ON (SELECT * FROM kmeanssample) as InputTable OUTPUT TABLE outputTable (‘kmeanssample_output’) ...
- Contract Phase: Advanced SQL Engine receives the SQL request.
In Advanced SQL Engine, the Parsing Engine (PE) is a virtual processor (vproc) that parses the SQL code of the query and sends a metadata message through QueryGrid fabric to the Planner Parser in ML Engine to discover metadata for function execution.
The metadata message contains information about the function, such as the function name, version, type, and description. It also includes the function parameters that specify the input and output tables and the input columns.
In the KMeans function, used to perform k-means clustering on a dataset, the metadata message defines the following tables:
- Required input table "InputTable" containing the list of features for clustering data
- Optional input table "CentroidsTable" containing the initial seed means for the clusters
- Required output table "OutputTable" in which to output the centroids of the clusters
- Optional output table "ClusteredOutput" in which to store the clustered output
Depending on the function, the metadata message may contain other parameters.
In ML Engine, the Planner Parser creates InputTable based on the metadata parameters, specifically the column data types, and executes the function against them with specific options from the user. The generated output table with specific data types is then used to create the output table in Advanced SQL Engine.
The Access Module Processors (AMPs) in Advanced SQL Engine then set up connections to the vworkers in ML Engine through the QueryGrid fabric, preparing to export data for analysis.
- Execute phase: Advanced SQL Engine exports the table to ML Engine for processing.
When Advanced SQL Engine receives the function execution metadata from ML Engine, the QueryGrid connector makes data type mapping and conversions, and initiates the export of the kmeanssample table from the AMPs through the QueryGrid fabric to the vworkers in ML Engine.
ML Engine temporarily stores the data for function execution.
- Execute phase: ML Engine executes the KMeans function.
- Execute phase: Advanced SQL Engine imports the result table from ML Engine.
After the function execution completes, Advanced SQL Engine pulls the result table from ML Engine through the QueryGrid fabric.
If additional tables are created as part of the execution output, ML Engine notifies Advanced SQL Engine, and exports the additional tables through separate Advanced SQL Engine sessions.