Column transformations play a critical role in the machine learning pipeline. These transformations involve applying mathematical functions to columns of data to create new variables that can improve the accuracy and robustness of machine learning models.
Some of the most commonly used transformations include absolute, log, exponential, ceil, floor, sigmoid, and tanh transformations. These transformations serve a variety of purposes, such as normalizing data, handling non-linear relationships, mitigating skewness, addressing heteroscedasticity, and handling categorical data.
However, certain function transformations have limitations or conditions that must be met for them to be applied correctly. For instance, the log transformation can only be applied to positive values, as the logarithm of negative values is undefined. Similarly, the square root function can only be applied to non-negative values.
In addition, some function transformations may require preprocessing steps to be applied to the data before they can be used. For example, the inverse function transformation requires that the target variable has a linear relationship with the predictor variables, and that the residuals are normally distributed. All the transformations mentioned above can only be applied to numeric columns.
It is necessary to check if a certain transformation can be applied to a particular data type and values.
TD_FunctionFit determines whether specified numeric transformations can be applied to specified input columns and outputs a table to use as input to TD_FunctionTransform, which does the transformations.
- This function requires the UTF8 client character set for UNICODE data.
- This function does not support Pass Through Characters (PTCs).
For information about PTCs, see International Character Set Support, B035-1125.
- This function does not support KanjiSJIS or Graphic data types.
- SELECT TOP gives non-deterministic results. Therefore, identical queries including this instruction may produce different results.