The OutlierFilter function is useful for filtering a numeric data set before applying ML Engine functions for which outliers can skew the estimates of parameters and cause inaccurate predictions. Such functions include time series functions, GLM, LAR, LinReg, PCA, and KMeans. The input data set is expected to have millions of attribute-value pairs.
The OutlierFilter function filters outliers from a data set, either deleting them or replacing them with a specified value. Optionally, the function stores the outliers in their own table. The function provides these methods for filtering outliers:
- Percentile
- Tukey's test
- Carling's modification to Tukey's test
- Median absolute deviation
The method determines the criteria for an observation to classify as an outlier.