ChimeraBoostClassifier¶
Gradient boosted oblivious trees for classification. Binary log loss for 2 classes, softmax for 3 or more, chosen automatically. See the User Guide: multiclass and calibrated probabilities.
Bases: ClassifierMixin, BaseEstimator
Gradient boosted oblivious trees for classification.
A scikit-learn compatible classifier. Uses binary logloss for 2 classes and
softmax for 3 or more, chosen automatically. predict_proba is temperature
scaled on the validation split for calibrated probabilities.
Read more in the User Guide.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_estimators
|
int
|
Maximum number of boosting rounds (trees). With |
2000
|
learning_rate
|
float or None
|
Shrinkage applied to each tree. When early stopping is active, |
None
|
depth
|
int
|
Depth of each oblivious tree; a depth-d tree makes d splits. |
6
|
l2_leaf_reg
|
float
|
L2 regularization on leaf values. |
1.0
|
max_bins
|
int
|
Histogram bins per numeric feature. |
128
|
subsample
|
float
|
Row subsampling fraction per tree (Minimum Variance Sampling below 1.0). |
1.0
|
colsample
|
float or None
|
Fraction of features eligible for each tree. |
None
|
cat_smoothing
|
float
|
Prior strength for ordered target statistics. Must be > 0 (a Bayesian pseudocount in the encoder denominator; 0 is undefined). |
1.0
|
cat_n_permutations
|
int
|
Number of random orderings averaged by the ordered target encoder. |
4
|
early_stopping_rounds
|
int or None
|
Rounds without validation improvement before stopping. |
None
|
min_child_weight
|
float or None
|
Minimum total hessian on each side of a split. |
None
|
thread_count
|
int or None
|
numba thread count. |
None
|
random_state
|
int or None
|
Seed for reproducibility (deterministic for a fixed |
None
|
verbose
|
bool
|
Print per-round train and validation metrics. |
False
|
ordered_boosting
|
bool
|
Use the leave-one-out leaf training step instead of plain Newton updates. |
False
|
cat_combinations
|
bool or None
|
Add all pairwise categorical-by-categorical features. |
None
|
cat_count_features
|
bool
|
Add, for every categorical column with at least 256 training
categories, a numeric column holding the category's training-row count
(the weight total when |
True
|
leaf_estimation_iterations
|
int or None
|
Extra Newton refinement steps per leaf. |
None
|
linear_leaves
|
bool or None
|
Fit a ridge linear model per leaf over the numeric split features instead
of a constant. |
None
|
linear_lambda
|
float
|
Ridge penalty on per-leaf linear slopes; larger is closer to a constant. |
1.0
|
quantize_gradients
|
bool
|
Run the split search on quantized gradients/hessians packed into
integer histograms (LightGBM-style quantized training, ~15-bit):
~20-25% faster fits at benchmark-flat accuracy. Leaf values always
use the exact float gradients; the rounding noise touches only
split selection and is deterministic for a fixed |
True
|
eval_metric
|
callable or None
|
Custom validation metric |
None
|
cross_features
|
(bool, always or None)
|
Numeric interaction columns. |
None
|
cross_top_columns
|
int or None
|
Cap on how many candidate cross columns the augmented fit carries.
|
None
|
selection_rounds
|
int or None
|
Round budget for the pre-cross base fit when the cross-features refit
will run. The base fit is an audition capped at this many rounds;
the candidates are judged on their best validation loss within the
budget, the winner continues to full early stopping, and the base is
refit in full only if the augmented model loses after being
truncated by the cap. |
100
|
early_stopping
|
bool
|
Hold out a stratified validation split and stop when it stops improving.
|
True
|
validation_fraction
|
float
|
Validation fraction used when |
0.2
|
n_ensembles
|
int or None
|
Number of bagged members. |
None
|
ensemble_n_jobs
|
int
|
Worker processes fitting ensemble members concurrently, each on an
equal share of the thread budget (same total cores as a single fit;
models are identical either way, wall-clock 1.2-2x faster). -1 sizes
the pool from the budget, capped at |
-1
|
max_samples
|
float
|
Fraction of rows each ensemble member trains on, drawn WITHOUT
replacement ("subagging"). The default 0.8 beats the classic
bootstrap on strength and fit time (a full-size bootstrap holds
only ~0.63n unique rows at n rows of compute). 1.0 restores the
classic full-size with-replacement bootstrap. Unsampled rows are
each member's early-stopping eval set either way. When |
0.8
|
refit_full
|
(replay, bool)
|
After the automatic early-stopping split has chosen the tree budget
(and model selection / temperature scaling have used it), retrain the
winning configuration on 100% of the rows -- rounds scaled by the
train-size ratio, learning rate pinned -- so the final model does not
pay the holdout data tax. Only affects fits that used the automatic
split (an explicit The default is Measured against |
"replay"
|
refit_members
|
bool
|
The bagged analogue of Measured on the decision suites, an 8-member bag improves in every
stratum, with perfect sweeps on the small-data ones (Grinsztajn at a
quarter of the rows 12W-0L, +1.206%), for about 10-17% more fit time.
Because each member is individually stronger you can also spend the
gain on fewer members: 5 refit members beat a plain 8-member bag on
accuracy while fitting about 20% faster. Ignored unless the fit is
bagged ( |
False
|
adaptive_learning_rate
|
bool
|
Let the auto Measured on the decision suites, the mean is positive in six of seven
strata, with sign-test passes at a quarter of the rows on both
Grinsztajn (9W-3L) and high-card (3W-0L) and no losses at all on
high-card at full size (6W-0L). Gains are individually small (medians
of +0.13% to +0.31%) and cost 1.09x to 1.31x fit time on the sizes it
touches. Only consulted when |
True
|
cat_features
|
list of int or str, or None
|
Default categorical columns, given as integer positions and/or column
names (names resolved against the DataFrame at fit). Used when |
None
|
store_training_data
|
bool
|
Keep the rows the final booster's leaves came from, so |
False
|
Attributes:
| Name | Type | Description |
|---|---|---|
classes_ |
ndarray
|
Class labels, in the column order of |
feature_importances_ |
ndarray of shape (n_features,)
|
Split-gain importance per input feature, normalized to sum to 1. |
best_iteration_ |
int
|
Number of trees retained after early stopping. |
temperature_ |
float
|
Fitted calibration temperature; > 1 means raw scores were over-confident. |
expected_value_ |
float
|
SHAP baseline (binary only); set after calling |
estimators_ |
list or None
|
Fitted members when |
member_params_ |
dict
|
Bagged-mode member defaults that were auto-applied (params the user
left on auto resolve to tuned member values inside a bag; explicit
values always win). Set only when |
n_samples_trained_ |
int
|
Number of rows the final booster's leaves came from: all rows when
the full-data refit ran, else the post-split training rows. Grown
by |
Source code in chimeraboost/sklearn_api.py
validation_history_
property
¶
Per-round validation loss recorded during fit -- binary or softmax
log loss -- as a list as long as the number of rounds run.
Empty when no eval_set or early-stopping split was available; a list
of the members' histories for a bagged model (n_ensembles > 1).
fit
¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
array - like
|
Training data. |
required |
y
|
array - like
|
Training data. |
required |
cat_features
|
list of int or str, or None
|
Columns to treat as categoricals, given as integer positions and/or
column names (names resolved against the DataFrame). Falls back to the
|
None
|
eval_set
|
(X_val, y_val) tuple or None
|
Explicit validation set with original class labels. When provided, automatic splitting is skipped. |
None
|
groups
|
array-like of shape (n_samples,) or None
|
Group labels (e.g. |
None
|
sample_weight
|
array-like of shape (n_samples,) or None
|
Per-sample weights, normalized to mean 1 internally. Applied
throughout: the gradient/leaf fit, the categorical target encoder,
the quantile bin borders, and the early-stopping metric on an
automatically split (or bagged out-of-bag) validation set, so a
zero-weight row never influences the model. An explicitly passed
|
None
|
callbacks
|
callable or list of callable, or None
|
Per-round fit hooks |
None
|
Source code in chimeraboost/sklearn_api.py
3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 | |
refresh
¶
Fold new rows into this fitted binary model without a from-scratch refit.
Appends (X, y[, sample_weight]) to the rows stored at fit time
(store_training_data=True), rebuilds a raw-equivalent training
matrix from the store, and replays the FITTED booster's configuration
on it -- the same structure-transfer replay the default full-data
refit uses. Every tree's structure, the round count and the learning
rate stay pinned; leaf values and linear-leaf coefficients are refit
against the combined gradients.
refresh replays the fitted configuration and ignores any
set_params made after fit. The binner borders, the count and
cross selections, the validation history, the fitted selections and
the frozen temperature_ are unchanged; n_samples_trained_
grows by the number of new rows.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
array - like
|
New rows with the same features the model was fit on. |
required |
y
|
array - like
|
New rows with the same features the model was fit on. |
required |
sample_weight
|
array-like of shape (n_samples,) or None
|
Weights for the new rows, in the same units as the fit weights. Required when the model was fit with weights, rejected when it was not. |
None
|
Returns:
| Type | Description |
|---|---|
self
|
|
Source code in chimeraboost/sklearn_api.py
report
¶
Score this model on (X, y): log loss, Brier and its skill
score, accuracy, F1 macro, and how far the probabilities are from
calibrated.
Returns the chimeraboost.metrics.classification_report dict;
chimeraboost.metrics.format_report prints it. The miscalibration
term is the one to watch after changing anything about
predict_proba -- it is what temperature scaling exists to keep
near zero.
Source code in chimeraboost/sklearn_api.py
shap_values
¶
Exact interventional TreeSHAP contributions in MARGIN space.
Binary returns (n_samples, n_features) in pre-temperature log-odds
of the positive class; multiclass returns (n_samples, n_features,
n_classes) in raw softmax scores. Rows sum to predict_raw(X) -
expected_value_ in both cases, with expected_value_ set as an
attribute -- a float for binary, a (n_classes,) array otherwise.
Attributions stay in margin space because the link, sigmoid or softmax,
is not linear, so no exact additive decomposition survives it. That is
where the wider SHAP ecosystem puts them too. Linear-leaf slopes are
included exactly. Averaged across the bag when n_ensembles > 1, an
additive surrogate for the soft-voted probability. X_background
overrides the reference distribution.
Source code in chimeraboost/sklearn_api.py
shap_importances
¶
Global SHAP importance: mean(abs(shap_values(X))) per feature.
Returns a structured (feature, importance) array sorted descending,
or a {feature: importance} dict when prettified=True
(CatBoost's flag). feature holds feature_names when given,
else the names captured from a DataFrame at fit, else column indices.
n_features truncates to the top N. Attributions are in margin
space; for multiclass the per-class magnitudes are averaged, so the
ranking is over features rather than (feature, class) pairs.
The expensive SHAP pass is cached on the instance, keyed by the data's content, so repeated calls on the same X -- including with different formatting options -- reuse it. The cache is dropped on refit.