Teradata Package for R Function Reference | 17.20 - GLMPerSegment - Teradata Package for R - Look here for syntax, methods and examples for the functions included in the Teradata Package for R.

GLMPerSegment

Description

The td_glm_sqle() function is used to train the whole data set as one model. The td_glm_per_segment_sqle() function is a partition-by-key function to create a single model for each partition.
The following operations are supported for td_glm_per_segment_sqle():

Gaussian linear regression.
Binomial logistic regression for binary classification.
Iteration modes batch and epoch.
Regularization using L1, L2 and Elasticnet.
Mini-batch gradient descent for numeric optimization algorithm.
Training support with and without intercept.
Class-weighted modeling.
Learning rate support for mini-batch gradient descent using constant, dynamic and hybrid gradients.
Learning rate optimization algorithms for mini-batch gradient descent using plain, momentum, and nesterov gradients.

Notes:

The order column can be optionally applied to guarantee the result in each run is deterministic. The situation of indeterministic result in a partition can occur if the "batch.size" argument is less than the number of rows in the partition. However, adding order columns can influence the performance.
A model is generated from the td_glm_per_segment_sqle(), and a model from td_glm_sqle() should be the same when the "batch.size" argument is not less than the number of rows in the corresponding partition of the model.
td_glm_per_segment_sqle() takes all features as numeric input. Categorical columns must be converted to numeric columns as preprocessing step, such as using td_one_hot_encoding_fit_sqle()/td_one_hot_encoding_transform_sqle(), td_ordinal_encoding_fit_sqle()/ td_ordinal_encoding_transform_sqle(), and td_target_encoding_fit_sqle()/td_target_encoding_transform_sqle().
Any observation with a missing value in an input column is ignored and not used fortraining. You can use some imputation function, such as td_simple_impute_fit_sqle()/td_simple_impute_transform_sqle() to do imputation of missing values.
Best practice is to standardize the dataset before using td_glm_per_segment_sqle(). Standardization, also known as feature scaling, makes a better model and converges quicker.
Model evaluation metrics of MSE, Loglikelihood, AIC, and BIC are generated by td_glm_per_segment_sqle(). For additional regression and classification metrics, you should use td_regression_evaluator_sqle(), td_classification_evaluator_sqle() and td_roc_sqle() functions as post-processing step.
td_glm_per_segment_sqle() supports binary classification only.
"response.column" for classification accepts values of 0 and 1 for two classes in the response column.
A maximum of 2046 features are supported due to the limitation imposed by the maximum number of columns (2048) in a input data.
"batch.size" and "learning.rate" are directly related. With a larger "batch.size", "learning.rate" can be increased for faster training with fewer iterations.
"iter_num" and "iter.num.no.change" are criteria used to stop learning. To force the function to run through all iterations, disable the "iter.num.no.change" by specifying "iter.num.no.change" to 0.
User need to try different combinations to find the best values for a particular use case.
When an unsupported data type is passed in "input.columns" or "response.column", the error message is of the following format: Unsupported data type for column index n in argument InputColumns. In the message, n refers to the index of the column based on an input to the function comprising "input.columns" and "response.column only. This is due to the rest of the columns in input are not needed by the function and internal optimizer does not expose them to the function. Due to this, n might be different from the actual index in the input data.

Usage

  td_glm_per_segment_sqle (
      formula = NULL,
      data = NULL,
      input.columns = NULL,
      response.column = NULL,
      attribute.data = NULL,
      parameter.data = NULL,
      family = "GAUSSIAN",
      iter.max = 300,
      batch.size = 10,
      lambda1 = 0.02,
      alpha = 0.15,
      iter.num.no.change = 50,
      tolerance = 0.001,
      intercept = TRUE,
      class.weights = "0:1.0, 1:1.0",
      learning.rate = NULL,
      initial.eta = 0.05,
      decay.rate = 0.25,
      decay.steps = 5,
      momentum = 0.0,
      nesterov = TRUE,
      iteration.mode = "BATCH",
      partition.column = NULL,
      ...
  )

Arguments

`formula`	Required Argument when "input.columns" and "response.column" are not provided, optional otherwise. Specifies a string consisting of "formula" which is the model to be fitted. Only basic formula of the "col1 ~ col2 + col3 +..." form are supported and all variables must be from the same tbl_teradata object. Notes: The function only accepts numeric features. User must convert the categorical features to numeric values, before passing to the formula. In case categorical features are passed to formula, those are ignored, and only numeric features are considered. Provide either "formula" argument or "input.columns" and "response.column" arguments. Types: character
`data`	Required Argument. Specifies the tbl_teradata containing the input data. Types: tbl_teradata
`input.columns`	Required argument when "response.column" is provided and "formula" is not provided, optional otherwise. Specifies the name(s) of the tbl_teradata column(s) that need to be used for training the model (predictors, features, or independent variables). Types: character OR vector of Strings (character)
`response.column`	Required argument when "response.column" is provided and "formula" is not provided, optional otherwise. Specifies the name of the column that contains the class label for binary classification when "family" is 'Binomial', or target value (dependent variable) for 'Regression' when "family" is 'Gaussian'. Types: character
`attribute.data`	Optional Argument. Specifies the tbl_teradata containing a subset of features to be used with respect to each partition. Types: tbl_teradata
`parameter.data`	Optional Argument. Specifies the tbl_teradata containing a subset of parameters to be used with respect to each partition. Types: tbl_teradata
`family`	Optional Argument. Specifies the distribution exponential family. Permitted Values: BINOMIAL GAUSSIAN Default Value: GAUSSIAN Types: character
`iter.max`	Optional Argument. Specifies the maximum number of iterations over the training data batches. If batch size is 0, then "iter.max" equals the number of epochs. Note: The "iter.max" must be in the range [1, 10000000]. Default Value: 300 Types: integer
`batch.size`	Optional Argument. Specifies the number of observations (training samples) to be parsed in one mini-batch. Must be a non-negative integer value. A value of 0 indicates no mini-batches, so the entire input is processed in each iteration, and the algorithm becomes (Batch) Gradient Descent. A value higher than the number of rows on any partition also default to Batch Gradient Descent. Note: The "iter.max" must be in the range [0, 2147483647]. Default Value: 10 Types: integer
`lambda1`	Optional Argument. Specifies the amount of regularization to be added. The higher the value, the stronger the regularization. It is also used to compute learning rate when "learning.rate" is set to 'Optimal'. A value of '0' means no regularization. Notes: The "lambda1" must be in the range [0, 1e7]. The "lambda1" must be a non-negative float value. Default Value: 0.02 Types: float OR integer
`alpha`	Optional Argument. Specifies the Elasticnet parameter for penalty computation. It is only effective if "lambda1" is greater than 0. The value represents the contribution ratio of L1 in the penalty. A value of 1.0 indicates L1 (LASSO) only, a value of 0 indicates L2 (Ridge) only, and a value in between is a combination of L1 and L2. Note: The "alpha" must be in the range [0, 1]. Default Value: 0.15 Types: float OR integer
`iter.num.no.change`	Optional Argument. Specifies the number of iterations with no improvement in loss, including the "tolerance", to stop training. A value of 0 indicates no early stopping and the algorithm continues until "iter.max" iterations are reached. Note: The "iter.num.no.change" must be in the range [0, 2147483647]. Default Value: 50 Types: integer
`tolerance`	Optional Argument. Specifies the stopping criteria in terms of loss function improvement. Training stops when loss is greater than best_loss – tolerance for "iter.num.no.change" times. Notes: The "tolerance" must be in the range [1e-7, 1e7]. The "tolerance" works only when "iter.num.no.change" is greater than 0. The "tolerance" must be a non-negative value. Default Value: 0.001 Types: float OR integer
`intercept`	Optional Argument. Specifies intercept should be estimated based on whether data is already centered or not. Default Value: TRUE Types: logical
`class.weights`	Optional Argument. Specifies the weights associated with classes. The format is '0:weight,1:weight'. For example, '0:1.0,1:0.5' gives twice as much weight to each observation in class 0. If the weight of a class is omitted, then it is assumed to be 1.0. Note: The "class.weights" argument works only when "family" is 'Binomial'. Default Value: 0:1.0, 1:1.0 Types: character
`learning.rate`	Optional Argument. Specifies the Learning rate algorithm. Permitted Values: CONSTANT OPTIMAL INVTIME ADAPTIVE Default Value: 'INVTIME' when "family" is set to 'Gaussian' 'OPTIMAL' when "family" is set to 'Binomial' Types: character
`initial.eta`	Optional Argument. Specifies the initial value of eta for learning rate. For constant, this value is the learning rate for all iterations. Note: The "initial.eta" must be in the range [1e-7, 1e7]. Default Value: 0.05. Types: float OR integer
`decay.rate`	Optional Argument. Specifies the decay rate for learning rate (invtime and adaptive). Note: The "decay.rate" must be in the range [1e-7, 1e7]. Default Value: 0.25. Types: float OR integer
`decay.steps`	Optional Argument. Specifies the decay steps (number of iterations) for adaptive learning rate. The learning rate changes by decay rate after this many number of iterations. Note: The "decay.steps" must be in the range [1, 2147483647]. Default Value: 5 Types: integer
`momentum`	Optional Argument. Specifies the value to use for momentum learning rate optimizer. A larger value indicates higher momentum contribution. A value of 0 means momentum optimizer is disabled. For a good momentum contribution, a value between 0.6-0.95 is recommended. Note: The "momentum" must be in the range [0, 1]. Default Value: 0.0 Types: float OR integer
`nesterov`	Optional Argument. Specifies the indicator that nesterov optimization should be applied to Momentum Optimizer or not. Default is TRUE when momentum is greater than 0, otherwise FALSE. Default Value: TRUE Types: logical
`iteration.mode`	Optional Argument. Specifies the iteration mode. Permitted Values: Batch: One iteration per batch. After processing rows in a batch, update the weight of the parameters and proceed to the next iteration. Epoch: One iteration per epoch. After processing all rows in a partition (with one or more batches), update the weight of the parameters and proceed to the next epoch. Default Value: Batch Types: character OR vector of Strings (character)
`partition.column`	Optional Argument. Specifies the name of the "input.columns" on which to partition the input. The name should be consistent with the "data_partition_column". If the "data_partition_column" is unicode with foreign language characters, then it is necessary to specify "partition.column" argument. Types: character
`...`	Specifies the generic keyword arguments SQLE functions accept. Below are the generic keyword arguments: persist: Optional Argument. Specifies whether to persist the results of the function in a table or not. When set to TRUE, results are persisted in a table; otherwise, results are garbage collected at the end of the session. Default Value: FALSE Types: logical volatile: Optional Argument. Specifies whether to put the results of the function in a volatile table or not. When set to TRUE, results are stored in a volatile table, otherwise not. Default Value: FALSE Types: logical Function allows the user to partition, hash, order or local order the input data. These generic arguments are available for each argument that accepts tbl_teradata as input and can be accessed as: "<input.data.arg.name>.partition.column" accepts character or vector of character (Strings) "<input.data.arg.name>.hash.column" accepts character or vector of character (Strings) "<input.data.arg.name>.order.column" accepts character or vector of character (Strings) "local.order.<input.data.arg.name>" accepts logical Note: These generic arguments are supported by tdplyr if the underlying SQLE Engine function supports, else an exception is raised.

Value

Function returns an object of class "td_glm_per_segment_sqle" which is a named list containing object of class "tbl_teradata".
Named list member(s) can be referenced directly with the "$" operator using the name(s):result

Examples

  
    
    # Get the current context/connection.
    con <- td_get_context()$connection
    
    # Load the example data.
    loadExampleData("decisionforestpredict_example", "housing_train")
    loadExampleData("tdplyr_example", "housing_train_attribute",
                    "housing_train_parameter")
    
    # Create tbl_teradata object.
    housing_train <- tbl(con, "housing_train")
    housing_train_attribute <- tbl(con, "housing_train_attribute")
    housing_train_parameter <- tbl(con, "housing_train_parameter")
    
    # Check the list of available analytic functions.
    display_analytic_functions()
    
    # Filter the rows from train dataset with homestyle as Classic and Eclectic.
    binomial_housing_train = housing_train 

    # td_glm_per_segment_sqle() function requires features in numeric
    # format for processing, so dropping the non-numeric columns.
    drop_cols <- c("driveway", "recroom", "gashw", "airco", "prefarea",
                   "fullbase")
    binomial_housing_train <- binomial_housing_train 
    gaussian_housing_train <- binomial_housing_train 
    
    # Transform the train dataset categorical values to encoded values.
    housing_train_ordinal_encodingfit <- td_ordinal_encoding_fit_sqle(
                                          target.column='homestyle',
                                          data=binomial_housing_train)
    
    res <- td_ordinal_encoding_transform_sqle(data=binomial_housing_train,
                                              object=housing_train_ordinal_encodingfit$result,
                                              accumulate=c("sn", "price",
                                                          "lotsize", "bedrooms",
                                                          "bathrms", "stories"))
    
    # Example 1: Train the model using the 'Gaussian' family.
    GLMPerSegment_out_1 <- td_glm_per_segment_sqle(data=gaussian_housing_train,
                                                   data.partition.column="stories",
                                                   input.columns=c('garagepl',
                                                                   'lotsize',
                                                                   'bedrooms',
                                                                   'bathrms'),
                                                   response.column="price",
                                                   family="Gaussian",
                                                   iter.max=1000,
                                                   batch.size=9)
    
    # Print the result.
    print(GLMPerSegment_out_1$result)
    
    # Example 2: Train the model using the 'Binomial' family, formula argument
    #            and subset of features and parameters to be used with respect
    #            to "partition_id".
    formula <- homestyle ~ price + lotsize + bedrooms + bathrms
    GLMPerSegment_out_2 <- td_glm_per_segment_sqle(
                            data=res$result,
                            data.partition.column="stories",
                            formula = formula,
                            attribute.data=housing_train_attribute,
                            attribute.data.partition.column="partition_id",
                            parameter.data=housing_train_parameter,
                            parameter.data.partition.column="partition_id",
                            family="Binomial",
                            iter.max=100)
    
    # Print the result.
    print(GLMPerSegment_out_2$result)