The OutlierFilter function is useful for filtering a numeric data set before applying analytic functions for which outliers can skew the estimates of parameters and lead to inaccurate predictions. Such functions include time series functions, GLM, LARS, LinReg, PCA, and KMeans. The input data set is expected to have as many as millions of attribute-value pairs.
The OutlierFilter function filters outliers from the data set, either deleting them or replacing them with a specified value. Optionally, the function stores the outliers in their own table. The methods that the function provides for filtering outliers are:
- Percentile
- Tukey's test
- Carling's modification to Tukey's test
- Median absolute deviation
The method determines the criteria for an observation to be classified as an outlier.