AdaBoost (Adaptive Boosting): AdaBoost assigns weights to each training instance and adjusts them based on the errors made by the previous weak models. It places more emphasis on misclassified instances in subsequent iterations, allowing the ensemble to focus on the difficult examples.

Gradient Boosting Machines (GBMs): GBMs build an ensemble of weak models in a stage-wise manner. At each stage, a new model is trained to minimize the loss function by fitting the negative gradient of the loss function with respect to the current ensemble’s predictions. This process gradually improves the ensemble’s performance.

XGBoost (Extreme Gradient Boosting): XGBoost is an optimized implementation of gradient boosting that incorporates additional regularization techniques to improve performance and control overfitting. It includes features such as parallelization, tree pruning, and regularization terms in the objective function.

LightGBM: LightGBM is another gradient-boosting framework designed to be both efficient and accurate. It uses a technique called Gradient-based One-Side Sampling (GOSS) to select a subset of instances for training, reducing the computational cost while maintaining good performance.

CatBoost: CatBoost is a gradient-boosting algorithm that handles categorical features naturally without requiring explicit encoding. It also incorporates a novel technique called ordered boosting, which improves the learning process by generating permutations of the data during training.