ChimeraBoostClassifier

Gradient boosted oblivious trees for classification. Binary log loss for 2 classes, softmax for 3 or more, chosen automatically. See the User Guide: multiclass and calibrated probabilities.

Bases: ClassifierMixin, BaseEstimator

Gradient boosted oblivious trees for classification.

A scikit-learn compatible classifier. Uses binary logloss for 2 classes and softmax for 3 or more, chosen automatically. predict_proba is temperature scaled on the validation split for calibrated probabilities.

Read more in the User Guide.

Parameters:

Name	Type	Description	Default
n_estimators	int	Maximum number of boosting rounds (trees). With early_stopping on, this is an upper bound and the best round is selected automatically.	2000
learning_rate	float or None	Shrinkage applied to each tree. When early stopping is active, None resolves to 0.1 on data of about 15,000 training rows or more and fades to 0.07 at 5,000 or fewer; see adaptive_learning_rate.	None
depth	int	Depth of each oblivious tree; a depth-d tree makes d splits.	6
l2_leaf_reg	float	L2 regularization on leaf values.	1.0
max_bins	int	Histogram bins per numeric feature.	128
subsample	float	Row subsampling fraction per tree (Minimum Variance Sampling below 1.0).	1.0
colsample	float or None	Fraction of features eligible for each tree. None resolves to 1.0 for a single model and to the bagged-member default 0.85 inside n_ensembles > 1 fits (see member_params_).	None
cat_smoothing	float	Prior strength for ordered target statistics. Must be > 0 (a Bayesian pseudocount in the encoder denominator; 0 is undefined).	1.0
cat_n_permutations	int	Number of random orderings averaged by the ordered target encoder.	4
early_stopping_rounds	int or None	Rounds without validation improvement before stopping. None becomes 50 when early stopping is active.	None
min_child_weight	float or None	Minimum total hessian on each side of a split. None resolves to a size-adaptive value: a full veto below ~500 rows, off above ~2000.	None
thread_count	int or None	numba thread count. None or -1 uses all detected cores.	None
random_state	int or None	Seed for reproducibility (deterministic for a fixed thread_count).	None
verbose	bool	Print per-round train and validation metrics.	False
ordered_boosting	bool	Use the leave-one-out leaf training step instead of plain Newton updates.	False
cat_combinations	bool or None	Add all pairwise categorical-by-categorical features. None enables them automatically only when the data is entirely categorical (where the interaction columns help without crowding out numeric splits); set True/False to force it on/off.	None
leaf_estimation_iterations	int or None	Extra Newton refinement steps per leaf. None is the auto default and resolves to 3, which helps small and categorical-heavy binary fits. Refinement only applies to the plain constant-leaf path: it is inert while linear_leaves is active (default-on for binary at >= ~1000 rows, where the per-leaf ridge already fits the second-order-optimal leaf) and is not implemented for multiclass. An explicitly-set value that will be ignored on the path about to run warns.	None
linear_leaves	bool or None	Fit a ridge linear model per leaf over the numeric split features instead of a constant. None enables it for binary classification and disables it for multiclass (where it is unsupported). Below ~1000 rows it falls back to constant leaves.	None
linear_lambda	float	Ridge penalty on per-leaf linear slopes; larger is closer to a constant.	1.0
quantize_gradients	bool	Run the split search on quantized gradients/hessians packed into integer histograms (LightGBM-style quantized training, ~15-bit): ~20-25% faster fits at benchmark-flat accuracy. Leaf values always use the exact float gradients; the rounding noise touches only split selection and is deterministic for a fixed random_state. False restores exact float64 histograms.	True
eval_metric	callable or None	Custom validation metric metric(y_true, y_pred[, sample_weight]) -> float scored on the validation set each round and used for early stopping and the internal model selections instead of log loss. Binary: y_true is the 0/1-encoded target and y_pred the positive-class probability; multiclass: y_true is one-hot (n, K) and y_pred the probability matrix. Lower is better, unless the callable carries a greater_is_better = True attribute -- validation_history_ then records negated values so the internal lower-is-better machinery is unchanged. Temperature scaling still calibrates on log loss.	None
cross_features	bool or None	Numeric interaction columns. None (the default) refits the model with difference and product columns for the pairs of the top numeric features of the base fit and keeps whichever model reaches the lower validation loss (cross_features_selected_ records the outcome, cross_pairs_ the columns kept); needs >= 2000 rows and >= 2 numeric features. Binary judges on binary log loss, multiclass on softmax log loss. False turns it off. Costs up to ~2x fit time when the refit runs.	None
cross_top_columns	int or None	Cap on how many candidate cross columns the augmented fit carries. None (the default) carries all of them. An integer k keeps only the k candidates whose values correlate most strongly with the base fit's validation residuals, which cuts the augmented fit's per-round cost; the validation race still decides whether the surviving block is used at all. Inert wherever cross features do not apply.	None
selection_rounds	int or None	Round budget for the pre-cross base fit when the cross-features refit will run. The base fit is an audition capped at this many rounds; the candidates are judged on their best validation loss within the budget, the winner continues to full early stopping, and the base is refit in full only if the augmented model loses after being truncated by the cap. None runs the base fit to full early stopping instead (the pre-0.15 behavior).	100
early_stopping	bool	Hold out a stratified validation split and stop when it stops improving. StratifiedGroupKFold is used when groups is passed to fit.	True
validation_fraction	float	Validation fraction used when early_stopping is on and no eval_set is passed to fit.	0.2
n_ensembles	int or None	Number of bagged members. None or 1 trains a single model; >= 2 soft-votes the calibrated probabilities of members, each fit on its own random row sample (max_samples, without replacement by default).	None
ensemble_n_jobs	int	Worker processes fitting ensemble members concurrently, each on an equal share of the thread budget (same total cores as a single fit; models are identical either way, wall-clock 1.2-2x faster). -1 sizes the pool from the budget, capped at n_ensembles; 1 fits members sequentially, each with the full budget.	-1
max_samples	float	Fraction of rows each ensemble member trains on, drawn WITHOUT replacement ("subagging"). The default 0.8 beats the classic bootstrap on strength and fit time (a full-size bootstrap holds only ~0.63n unique rows at n rows of compute). 1.0 restores the classic full-size with-replacement bootstrap. Unsampled rows are each member's early-stopping eval set either way. When groups is passed to fit, the draw is over whole groups instead of rows (a cluster bootstrap at 1.0), so each member's eval set is made of groups it never trained on.	0.8
refit_full	(replay, bool)	After the automatic early-stopping split has chosen the tree budget (and model selection / temperature scaling have used it), retrain the winning configuration on 100% of the rows -- rounds scaled by the train-size ratio, learning rate pinned -- so the final model does not pay the holdout data tax. Only affects fits that used the automatic split (an explicit eval_set or early_stopping=False is unchanged); the calibrated temperature transfers to the refit model. validation_history_ keeps the early-stopped fit's curve. Costs one extra refit; on by default since 0.25.0, as the strongest single-model setting measured (benchmarks/SELECT_PLAN.md). Set False, or quality=2, for the faster pre-0.25 behaviour. The default is "replay", which gets the same thing for about two thirds of the cost. Growing trees is 83-85% of a fit and is a SEARCH; "replay" reuses the winner's tree structures and refits only the leaf values against full-data gradients, so the held-out rows still shape every leaf value while the split search is not paid for twice. Measured against True at 3 seeds: on Grinsztajn accuracy was flat (27W-32L over 59 datasets, mean +0.005%) and fit time fell 34% on all 59; on high-cardinality categorical data it ran slightly behind (mean -0.256%) for 17% less fit time. Pass True for the from-scratch refit -- marginally stronger on high-card data, and the setting benchmarked in REFIT_PLAN.md. Multiclass ignores "replay" and always uses the from-scratch refit (benchmarks/REPLAY_PLAN.md).	"replay"
refit_members	bool	The bagged analogue of refit_full, and off by default. A bag member trains on max_samples of the rows and early-stops on its out-of-bag complement, so its leaf values never see the rows it stopped on -- the full-data refit that helps a single model has never fired for it. With True each member replays its own tree structure against gradients from every row once early stopping is done. Only the leaf values move; the splits stay exactly as that member's own sample grew them, which is where a bag's diversity actually lives. Measured on the decision suites, an 8-member bag improves in every stratum, with perfect sweeps on the small-data ones (Grinsztajn at a quarter of the rows 12W-0L, +1.206%), for about 10-17% more fit time. Because each member is individually stronger you can also spend the gain on fewer members: 5 refit members beat a plain 8-member bag on accuracy while fitting about 20% faster. Ignored unless the fit is bagged (n_ensembles >= 2), and ignored for multiclass, where a member would need a full refit rather than a cheap structure replay.	False
adaptive_learning_rate	bool	Let the auto learning_rate depend on how much data it has, instead of being size-blind: a linear fade from 0.07 at 5,000 training rows or fewer up to the historical flat 0.1 at 15,000 or more. Small data is where a lower rate pays, and it is also where the extra trees it needs are cheap. Default-on since 0.30.0; above the upper threshold it is a no-op and the model is byte-identical to earlier versions, so only small-data fits move. Set False for the flat 0.1 everywhere. Measured on the decision suites, the mean is positive in six of seven strata, with sign-test passes at a quarter of the rows on both Grinsztajn (9W-3L) and high-card (3W-0L) and no losses at all on high-card at full size (6W-0L). Gains are individually small (medians of +0.13% to +0.31%) and cost 1.09x to 1.31x fit time on the sizes it touches. Only consulted when learning_rate is None and early stopping is on -- without early stopping the rate already scales with the round budget. Bagged fits (n_ensembles >= 2) are unaffected too, since their members already carry an explicit member learning rate.	True
cat_features	list of int or str, or None	Default categorical columns, given as integer positions and/or column names (names resolved against the DataFrame at fit). Used when fit is called without its own cat_features (the fit argument overrides). Provided as a constructor argument so GridSearchCV/Pipeline can carry it.	None

Attributes:

Name	Type	Description
classes_	ndarray	Class labels, in the column order of predict_proba.
feature_importances_	ndarray of shape (n_features,)	Split-gain importance per input feature, normalized to sum to 1.
best_iteration_	int	Number of trees retained after early stopping.
temperature_	float	Fitted calibration temperature; > 1 means raw scores were over-confident.
expected_value_	float	SHAP baseline (binary only); set after calling shap_values.
estimators_	list or None	Fitted members when n_ensembles > 1, otherwise None.
member_params_	dict	Bagged-mode member defaults that were auto-applied (params the user left on auto resolve to tuned member values inside a bag; explicit values always win). Set only when n_ensembles > 1.

Source code in chimeraboost/sklearn_api.py

def __init__(self, n_estimators=2000, learning_rate=None, depth=6,
             l2_leaf_reg=1.0, max_bins=128, subsample=1.0, colsample=None,
             cat_smoothing=1.0, cat_n_permutations=4,
             early_stopping_rounds=None,
             min_child_weight=None, thread_count=None, random_state=None,
             verbose=False, ordered_boosting=False,
             cat_combinations=None, leaf_estimation_iterations=None,
             linear_leaves=None, linear_lambda=1.0, cross_features=None,
             cross_top_columns=None, selection_rounds=100,
             early_stopping=True, validation_fraction=0.2,
             n_ensembles=None, ensemble_n_jobs=-1, max_samples=0.8,
             cat_features=None, quantize_gradients=True,
             eval_metric=None, refit_full="replay", refit_members=False,
             quality=None, adaptive_learning_rate=True):
    self.n_estimators = n_estimators
    self.learning_rate = learning_rate
    self.depth = depth
    self.l2_leaf_reg = l2_leaf_reg
    self.max_bins = max_bins
    self.subsample = subsample
    self.colsample = colsample
    self.cat_smoothing = cat_smoothing
    self.cat_n_permutations = cat_n_permutations
    self.early_stopping_rounds = early_stopping_rounds
    self.cat_features = cat_features
    self.min_child_weight = min_child_weight
    self.eval_metric = eval_metric
    self.thread_count = thread_count
    self.random_state = random_state
    self.verbose = verbose
    self.ordered_boosting = ordered_boosting
    self.cat_combinations = cat_combinations
    self.leaf_estimation_iterations = leaf_estimation_iterations
    self.linear_leaves = linear_leaves
    self.linear_lambda = linear_lambda
    self.cross_features = cross_features
    self.cross_top_columns = cross_top_columns
    self.selection_rounds = selection_rounds
    self.early_stopping = early_stopping
    self.validation_fraction = validation_fraction
    self.n_ensembles = n_ensembles
    self.ensemble_n_jobs = ensemble_n_jobs
    self.max_samples = max_samples
    self.quantize_gradients = quantize_gradients
    self.refit_full = refit_full
    self.refit_members = refit_members
    self.quality = quality
    # Size fade for the auto learning rate, default-on since 0.30.0; only
    # consulted when learning_rate is None. False == the historical flat 0.1.
    self.adaptive_learning_rate = adaptive_learning_rate

validation_history_ property

validation_history_

Per-round validation loss recorded during fit -- binary or softmax log loss -- as a list as long as the number of rounds run.

Empty when no eval_set or early-stopping split was available; a list of the members' histories for a bagged model (n_ensembles > 1).

fit

fit(
    X,
    y,
    cat_features=None,
    eval_set=None,
    groups=None,
    sample_weight=None,
    callbacks=None,
)

Fit the model.

Parameters:

Name	Type	Description	Default
X	array - like	Training data.	required
y	array - like	Training data.	required
cat_features	list of int or str, or None	Columns to treat as categoricals, given as integer positions and/or column names (names resolved against the DataFrame). Falls back to the cat_features constructor argument when not given here; passing it here overrides the constructor value. (The constructor form lets GridSearchCV/Pipeline carry it, which a fit-only kwarg can't.)	None
eval_set	(X_val, y_val) tuple or None	Explicit validation set with original class labels. When provided, automatic splitting is skipped.	None
groups	array-like of shape (n_samples,) or None	Group labels (e.g. df['subject_id']). When supplied and early stopping triggers an automatic split, StratifiedGroupKFold keeps groups intact and class proportions balanced across the split.	None
sample_weight	array-like of shape (n_samples,) or None	Per-sample weights, normalized to mean 1 internally. Applied throughout: the gradient/leaf fit, the categorical target encoder, the quantile bin borders, and the early-stopping metric on an automatically split (or bagged out-of-bag) validation set, so a zero-weight row never influences the model. An explicitly passed eval_set carries no weights and is scored unweighted.	None
callbacks	callable or list of callable, or None	Per-round fit hooks cb(iteration, train_loss, val_loss, model); a callback returning True requests an early stop. Used for live validation-curve capture and instrumentation. Not supported with n_ensembles > 1 (members fit in parallel worker processes).	None

Source code in chimeraboost/sklearn_api.py

def fit(self, X, y, cat_features=None, eval_set=None, groups=None,
        sample_weight=None, callbacks=None):
    """Fit the model.

    Parameters
    ----------
    X, y : array-like
        Training data.
    cat_features : list of int or str, or None
        Columns to treat as categoricals, given as integer positions and/or
        column names (names resolved against the DataFrame). Falls back to the
        ``cat_features`` constructor argument when not given here; passing it
        here overrides the constructor value. (The constructor form lets
        ``GridSearchCV``/``Pipeline`` carry it, which a fit-only kwarg can't.)
    eval_set : (X_val, y_val) tuple or None
        Explicit validation set with original class labels. When provided,
        automatic splitting is skipped.
    groups : array-like of shape (n_samples,) or None
        Group labels (e.g. ``df['subject_id']``). When supplied and early
        stopping triggers an automatic split, ``StratifiedGroupKFold`` keeps
        groups intact and class proportions balanced across the split.
    sample_weight : array-like of shape (n_samples,) or None
        Per-sample weights, normalized to mean 1 internally. Applied
        throughout: the gradient/leaf fit, the categorical target encoder,
        the quantile bin borders, and the early-stopping metric on an
        automatically split (or bagged out-of-bag) validation set, so a
        zero-weight row never influences the model. An explicitly passed
        ``eval_set`` carries no weights and is scored unweighted.
    callbacks : callable or list of callable, or None
        Per-round fit hooks ``cb(iteration, train_loss, val_loss, model)``;
        a callback returning True requests an early stop. Used for live
        validation-curve capture and instrumentation. Not supported with
        ``n_ensembles > 1`` (members fit in parallel worker processes).
    """
    cat_features = _resolve_cat_features(self, cat_features)
    cat_features = _resolve_cat_feature_names(cat_features, X)
    _validate_hyperparams(self)
    y = _validate_fit_input(self, X, y, cat_features, sample_weight,
                            classification=True)
    # Cached shap_importances describe the previous fit; drop them.
    self._shap_importances_cache_ = None

    if eval_set is not None:
        _check_eval_set(eval_set, self.n_features_in_,
                        classification=True)
        if not getattr(self, "_is_bag_member", False):
            _check_feature_names_match(self, eval_set[0])

        # Bag members are exempt: their OOB eval set may legitimately hold
        # a rare label their row sample missed, and the parent aligns
        # member probability columns to the global class set.
        if not getattr(self, "_is_bag_member", False):
            _check_eval_labels(eval_set, y)

    with _quality_applied(self):
        if self.n_ensembles and self.n_ensembles > 1:
            if callbacks is not None:
                raise ValueError(
                    "callbacks are not supported with n_ensembles > 1.")

            # Fix the global class set up front: a member's bootstrap may
            # miss a rare class, and predict_proba aligns each member's
            # columns to this.
            yarr = np.asarray(y)
            self.classes_ = np.unique(yarr)
            self.n_classes_ = self.classes_.size
            if self.n_classes_ < 2:
                raise ValueError(
                    f"Need at least 2 classes; got {self.n_classes_} "
                    "class(es).")

            self._multiclass = self.n_classes_ > 2
            self.estimators_ = _fit_bagged(self, X, yarr, cat_features,
                                           eval_set, groups, sample_weight)
            return self

        self.estimators_ = None
        return self._fit_single(X, y, cat_features, eval_set, groups,
                                sample_weight, callbacks)

report

report(X, y, sample_weight=None)

Score this model on (X, y): log loss, Brier and its skill score, accuracy, F1 macro, and how far the probabilities are from calibrated.

Returns the chimeraboost.metrics.classification_report dict; chimeraboost.metrics.format_report prints it. The miscalibration term is the one to watch after changing anything about predict_proba -- it is what temperature scaling exists to keep near zero.

Source code in chimeraboost/sklearn_api.py

def report(self, X, y, sample_weight=None):
    """Score this model on ``(X, y)``: log loss, Brier and its skill
    score, accuracy, F1 macro, and how far the probabilities are from
    calibrated.

    Returns the `chimeraboost.metrics.classification_report` dict;
    `chimeraboost.metrics.format_report` prints it. The miscalibration
    term is the one to watch after changing anything about
    ``predict_proba`` -- it is what temperature scaling exists to keep
    near zero.
    """
    from . import metrics
    return metrics.classification_report(
        y, self.predict_proba(X), self.classes_, sample_weight)

shap_values

shap_values(X, X_background=None)

Exact interventional TreeSHAP contributions in MARGIN space.

Binary returns (n_samples, n_features) in pre-temperature log-odds of the positive class; multiclass returns (n_samples, n_features, n_classes) in raw softmax scores. Rows sum to predict_raw(X) - expected_value_ in both cases, with expected_value_ set as an attribute -- a float for binary, a (n_classes,) array otherwise.

Attributions stay in margin space because the link, sigmoid or softmax, is not linear, so no exact additive decomposition survives it. That is where the wider SHAP ecosystem puts them too. Linear-leaf slopes are included exactly. Averaged across the bag when n_ensembles > 1, an additive surrogate for the soft-voted probability. X_background overrides the reference distribution.

Source code in chimeraboost/sklearn_api.py

def shap_values(self, X, X_background=None):
    """Exact interventional TreeSHAP contributions in MARGIN space.

    Binary returns ``(n_samples, n_features)`` in pre-temperature log-odds
    of the positive class; multiclass returns ``(n_samples, n_features,
    n_classes)`` in raw softmax scores. Rows sum to ``predict_raw(X) -
    expected_value_`` in both cases, with ``expected_value_`` set as an
    attribute -- a float for binary, a ``(n_classes,)`` array otherwise.

    Attributions stay in margin space because the link, sigmoid or softmax,
    is not linear, so no exact additive decomposition survives it. That is
    where the wider SHAP ecosystem puts them too. Linear-leaf slopes are
    included exactly. Averaged across the bag when ``n_ensembles > 1``, an
    additive surrogate for the soft-voted probability. ``X_background``
    overrides the reference distribution.
    """
    Xv = _check_predict_input(self, X)
    X = X if Xv is None else Xv

    if X_background is not None:
        bg = _check_predict_input(self, X_background)
        X_background = X_background if bg is None else bg

    members = self.estimators_ if self.estimators_ is not None else None
    if members is not None:
        out = [m.model_.shap_values(X, background=X_background)
               for m in members]
        base = np.mean([b for _, b in out], axis=0)
        # Binary members carry a scalar baseline, multiclass a (K,) one.
        self.expected_value_ = base if base.ndim else float(base)
        return np.mean([p for p, _ in out], axis=0)

    phi, base = self.model_.shap_values(X, background=X_background)
    self.expected_value_ = base
    return phi

shap_importances

shap_importances(
    X, feature_names=None, n_features=None, prettified=False
)

Global SHAP importance: mean(abs(shap_values(X))) per feature.

Returns a structured (feature, importance) array sorted descending, or a {feature: importance} dict when prettified=True (CatBoost's flag). feature holds feature_names when given, else the names captured from a DataFrame at fit, else column indices. n_features truncates to the top N. Attributions are in margin space; for multiclass the per-class magnitudes are averaged, so the ranking is over features rather than (feature, class) pairs.

The expensive SHAP pass is cached on the instance, keyed by the data's content, so repeated calls on the same X -- including with different formatting options -- reuse it. The cache is dropped on refit.

Source code in chimeraboost/sklearn_api.py

def shap_importances(self, X, feature_names=None, n_features=None,
                     prettified=False):
    """Global SHAP importance: ``mean(abs(shap_values(X)))`` per feature.

    Returns a structured ``(feature, importance)`` array sorted descending,
    or a ``{feature: importance}`` dict when ``prettified=True``
    (CatBoost's flag). ``feature`` holds ``feature_names`` when given,
    else the names captured from a DataFrame at fit, else column indices.
    ``n_features`` truncates to the top N. Attributions are in margin
    space; for multiclass the per-class magnitudes are averaged, so the
    ranking is over features rather than (feature, class) pairs.

    The expensive SHAP pass is cached on the instance, keyed by the data's
    content, so repeated calls on the same X -- including with different
    formatting options -- reuse it. The cache is dropped on refit.
    """
    return _shap_importances(self, X, feature_names, n_features,
                             prettified)