AdaBoost

AdaBoost, short for Adaptive Boosting, was formulated by Schapire. It is a meta-algorithm, and can be used in conjunction with a lot of other learning algorithms to improve their performances. AdaBoosting is adaptive in the sense that subsequent classifiers built are tweaking in favor of those instances misclassified by previous classifiers. It has been proven, in theory, that AdaBoosting is robust to noise data; in particular, it is not susceptible to the Overfitting problem inherent to a wide variety of learning algorithms. Quinlan's C5.0 implements, as commonly believed, a certain proprietary version of AdaBoosting together with decision tree induction algorithm. ==Generalised form of the algorithm== Given:

(x_{1},y_{1}),\ldots,(x_{m},y_{m})

where

x_{i} \in X,\, y_{i} \in Y = \{-1, +1\}

Initialise

D_{1}(i) = 1/m

. For

t = 1,\ldots,T

: * Train base learner using distribution

D_{t}

. * Get base classifier

h_{t} : X \to \mathbf{R}

. * Choose

\alpha_{t} \in \mathbf{R}

. * Update:

D_{t+1}(i) = \frac{D_{t}(i)\, exp(-\alpha_{t} y_{i}h_{t}(x_{i}))}{Z_{t}}

where

Z_{t}

is a normalisation factor (chosen so that

D_{t+1}

will be a distribution). Output the final classifier:

H(x) = \textrm{sign}\left( \sum_{t=1}^{T} \alpha_{t}h_{t}(x)\right)

Classification algorithms

Adaboost

#REDIRECTAdaBoost

See other meanings of words starting from letter:

A

AB | AC | AD | AE | AF | AG | AH | AI | AJ | AK | AL | AM | AN | AO | AP | AR | AS | AT | AU | AW | AX | AY | AZ |

Words begining with Adaboost:

AdaBoost
Adaboost
Adaboosting