7.00.02 - Background - Aster Analytics

Teradata Aster® Analytics Foundation User GuideUpdate 2

Product
Aster Analytics
Release Number
7.00.02
Published
September 2017
Content Type
Programming Reference
User Guide
Publication ID
B700-1022-700K
Language
English (United States)
Last Update
2018-04-17

The AdaBoost algorithm (described by J. Zhu, H. Zou, S. Rosset and T. Hastie 2009 in https://web.stanford.edu/~hastie/Papers/samme.pdf) is iterative. It starts with a weak classifying algorithm, and each iteration gives higher weights to the data points that the previous iteration classified incorrectly—a technique called Adaptive Boosting, for which the AdaBoost algorithm is named. AdaBoost constructs a strong classifier as a linear combination of weak classifiers.

The AdaBoost_Drive function uses a single decision tree as the initial weak classifying algorithm.

The boosting process is:

  1. Train on a data set, using a weak classifier. (For the first iteration, all data points have equal weight.)
  2. Calculate the weighted training error.
  3. Calculate the weight of the current classifier to use in the final calculation (step 6).
  4. Update the weights for the next iteration by decreasing the weights of the correctly classified data points and increasing the weights of the incorrectly classified data points.
  5. Repeat steps 1 through 4 for each weak classifier.
  6. Calculate the strong classifier as a weighted vote of the weak classifiers, using the weights calculated in step 3.

Mathematically:

  1. Assume that the training set has n data points classified into K classes:

    (x 1, y 1), (x 2, y 2), ..., (x n, y n) where y i is an element of {1, 2, ..., K}

  2. In the first iteration, assign the same weight to each data point:

    w i(i) = 1/n

  3. For each of the T weak classifiers:
    1. Fit classifier h t(x) to the training data, using weights w t.
    2. Calculate the error rate for the classifier:


    3. Calculate the weight of the classifier h t:

      α t = log ((1-err i ) / err i ) + log (K - 1)

    4. Update the weights w t+ 1, increasing the weights of the data points that h t classified incorrectly:


      for all i = 1, 2, ..., n

  4. Calculate the strong classifier: