ChimeraBoostRegressor¶
Gradient boosted oblivious trees for regression. See the User Guide: regression basics, quantile regression, and custom objectives.
Bases: RegressorMixin, BaseEstimator
Gradient boosted oblivious trees for regression.
A scikit-learn compatible regressor supporting squared-error, absolute-error, and quantile losses, native categorical features, sample weights, bagging, and exact SHAP attributions.
Read more in the User Guide.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_estimators
|
int
|
Maximum number of boosting rounds (trees). With |
2000
|
learning_rate
|
float or None
|
Shrinkage applied to each tree. When early stopping is active, |
None
|
depth
|
int or None
|
Depth of each oblivious tree; a depth-d tree makes d splits. |
None
|
l2_leaf_reg
|
float
|
L2 regularization on leaf values. |
1.0
|
max_bins
|
int
|
Histogram bins per numeric feature. |
128
|
subsample
|
float
|
Row subsampling fraction per tree. Below 1.0, rows are drawn by Minimum Variance Sampling (gradient-weighted, unbiased) rather than uniformly. |
1.0
|
colsample
|
float or None
|
Fraction of features eligible for each tree. |
None
|
cat_smoothing
|
float
|
Prior strength for ordered target statistics; higher shrinks rare categories harder toward the global mean. Must be > 0 -- it is the Bayesian pseudocount in the encoder denominator, so 0 is undefined. |
1.0
|
cat_n_permutations
|
int
|
Number of random orderings averaged by the ordered target encoder. |
4
|
early_stopping_rounds
|
int or None
|
Rounds without validation improvement before stopping. |
None
|
loss
|
str or object
|
Training objective. Built in: |
"RMSE"
|
alpha
|
float
|
Quantile level for |
0.5
|
delta
|
float
|
Huber transition point for |
1.0
|
tweedie_variance_power
|
float
|
Variance power for |
1.5
|
eval_metric
|
callable or None
|
Custom validation metric |
None
|
min_child_weight
|
float
|
Minimum total hessian required on each side of a split. |
1.0
|
thread_count
|
int or None
|
numba thread count. |
None
|
random_state
|
int or None
|
Seed for reproducibility (deterministic for a fixed |
None
|
verbose
|
bool
|
Print per-round train and validation metrics. |
False
|
ordered_boosting
|
bool
|
Use the leave-one-out leaf training step instead of plain Newton updates. |
False
|
cat_combinations
|
bool or None
|
Add all pairwise categorical-by-categorical features. |
None
|
leaf_estimation_iterations
|
int
|
Newton refinement steps per leaf. |
1
|
linear_leaves
|
bool or None
|
Fit a ridge linear model per leaf over the numeric split features instead
of a constant value, adding local slope where step leaves underfit. Leaves
with too few rows fall back to a constant. Not available with MAE or
quantile loss. |
None
|
linear_lambda
|
float
|
Ridge penalty on per-leaf linear slopes; larger is closer to a constant. |
1.0
|
quantize_gradients
|
bool
|
Run the split search on quantized gradients/hessians packed into
integer histograms (LightGBM-style quantized training, ~15-bit):
~20-25% faster fits at benchmark-flat accuracy. Leaf values always
use the exact float gradients; the rounding noise touches only
split selection and is deterministic for a fixed |
True
|
cross_features
|
(bool, always or None)
|
Numeric interaction columns. |
None
|
cross_top_columns
|
int or None
|
Cap on how many candidate cross columns the augmented fit carries.
|
None
|
selection_rounds
|
int or None
|
Round budget for the internal selection fits. The constant/linear-leaf
variants and the pre-cross base fit run at most this many rounds
(auditions, judged on their best validation loss within the budget);
the winning candidate continues to full early stopping, and the
audition winner is refit in full only when the cross-augmented model
loses or cross features do not apply. An audition that early-stops
before the budget is the full fit already (no extra cost). |
100
|
early_stopping
|
bool
|
Hold out a validation split and stop when its score stops improving. |
True
|
validation_fraction
|
float
|
Validation fraction used when |
0.2
|
n_ensembles
|
int or None
|
Number of bagged members. |
None
|
ensemble_n_jobs
|
int
|
Worker processes fitting ensemble members concurrently, each on an
equal share of the thread budget (same total cores as a single fit;
models are identical either way, wall-clock 1.2-2x faster). -1 sizes
the pool from the budget, capped at |
-1
|
max_samples
|
float
|
Fraction of rows each ensemble member trains on, drawn WITHOUT
replacement ("subagging"). The default 0.8 beats the classic
bootstrap on strength and fit time (a full-size bootstrap holds
only ~0.63n unique rows at n rows of compute). 1.0 restores the
classic full-size with-replacement bootstrap. Unsampled rows are
each member's early-stopping eval set either way. When |
0.8
|
refit_full
|
(replay, bool)
|
After the automatic early-stopping split has chosen the tree budget
(and model selection / calibration have used it), retrain the winning
configuration on 100% of the rows -- rounds scaled by the train-size
ratio, learning rate pinned -- so the final model does not pay the
holdout data tax. Only affects fits that used the automatic split
(an explicit The default is Measured against |
"replay"
|
refit_members
|
bool
|
The bagged analogue of Measured on the decision suites, an 8-member bag improves in every
stratum, with perfect sweeps on the small-data ones (Grinsztajn at a
quarter of the rows 12W-0L, +1.206%), for about 10-17% more fit time.
Because each member is individually stronger you can also spend the
gain on fewer members: 5 refit members beat a plain 8-member bag on
accuracy while fitting about 20% faster. Ignored unless the fit is
bagged ( |
False
|
adaptive_learning_rate
|
bool
|
Let the auto Measured on the decision suites, the mean is positive in six of seven
strata, with sign-test passes at a quarter of the rows on both
Grinsztajn (9W-3L) and high-card (3W-0L) and no losses at all on
high-card at full size (6W-0L). Gains are individually small (medians
of +0.13% to +0.31%) and cost 1.09x to 1.31x fit time on the sizes it
touches. Only consulted when |
True
|
cat_features
|
list of int or str, or None
|
Default categorical columns, given as integer positions and/or column
names (names resolved against the DataFrame at fit). Used when |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
feature_importances_ |
ndarray of shape (n_features,)
|
Split-gain importance per input feature, normalized to sum to 1. |
best_iteration_ |
int
|
Number of trees retained after early stopping. |
expected_value_ |
float
|
SHAP baseline in the same additive space as |
estimators_ |
list or None
|
Fitted members when |
member_params_ |
dict
|
Bagged-mode member defaults that were auto-applied (params the user
left on auto resolve to tuned member values inside a bag; explicit
values always win). Set only when |
quantile_offset_ |
float
|
Split-conformal correction added to every prediction when
|
linear_leaves_selected_ |
bool or None
|
With |
Source code in chimeraboost/sklearn_api.py
validation_history_
property
¶
Per-round validation score recorded during fit.
The training loss (RMSE space for regression), or the custom
eval_metric when one was set -- negated if the metric declares
greater_is_better = True, so lower is always better here.
A list as long as the number of rounds run. Empty when no eval_set
or early-stopping split was available; a list of the members' histories
for a bagged model (n_ensembles > 1).
fit
¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
array - like
|
Training data. |
required |
y
|
array - like
|
Training data. |
required |
cat_features
|
list of int or str, or None
|
Columns to treat as categoricals, given as integer positions and/or
column names (names resolved against the DataFrame). Falls back to the
|
None
|
eval_set
|
(X_val, y_val) tuple or None
|
Explicit validation set. When provided, automatic splitting is skipped regardless of the early_stopping setting. |
None
|
groups
|
array-like of shape (n_samples,) or None
|
Group labels for the samples (e.g. |
None
|
sample_weight
|
array-like of shape (n_samples,) or None
|
Per-sample weights, normalized to mean 1 internally. Applied
throughout: the gradient/leaf fit, the categorical target encoder,
the quantile bin borders, and the early-stopping metric on an
automatically split (or bagged out-of-bag) validation set, so a
zero-weight row never influences the model. An explicitly passed
|
None
|
callbacks
|
callable or list of callable, or None
|
Per-round fit hooks |
None
|
Source code in chimeraboost/sklearn_api.py
2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 | |
predict_raw
¶
Raw additive score before the loss link and conformal quantile offset.
For the identity-link losses (RMSE/MAE/Huber) this equals predict;
for loss="Quantile" it is predict minus the fitted
quantile_offset_. For the log-link losses (Poisson/Gamma/Tweedie)
and custom losses with a non-identity transform it is the pre-link
score -- the space shap_values reconstructs exactly. Averaged across
the bag when n_ensembles > 1, mirroring predict.
Source code in chimeraboost/sklearn_api.py
staged_predict
¶
Yield the prediction after each successive tree.
The conformal quantile offset is a post-fit constant and is included in
every stage, so the final stage equals predict.
Source code in chimeraboost/sklearn_api.py
report
¶
Score this model on (X, y): RMSE, MAE and the R2 skill score.
Returns the chimeraboost.metrics.regression_report dict;
chimeraboost.metrics.format_report prints it. baseline sets what
the skill score is measured against -- pass the training targets to
score against the mean the model actually had, rather than the
hindsight mean of y.
Source code in chimeraboost/sklearn_api.py
shap_values
¶
Exact interventional TreeSHAP contributions in the model's additive space.
Returns an array of shape (n_samples, n_features) whose rows sum to
the model's additive score minus expected_value_. For the identity-
link losses this is predict(X) - expected_value_; for
loss="Quantile" the fitted conformal offset is folded into
expected_value_ too, so the rows still sum to predict(X) -
expected_value_. For the log-link losses (Poisson/Gamma/Tweedie) and
custom losses with a non-identity transform, attributions stay in raw
(link) space and rows sum to predict_raw(X) - expected_value_.
expected_value_ (set as an attribute by this call) is the mean
additive score over the background. Each entry is a feature's signed
additive contribution; linear-leaf slopes are included exactly. Averaged
across the bag when n_ensembles > 1 -- the bag prediction is the
members' mean, so the averaged attribution stays exact. X_background
overrides the reference distribution (default: a sample of the training
data).
Source code in chimeraboost/sklearn_api.py
shap_importances
¶
Global SHAP importance: mean(abs(shap_values(X))) per feature.
Returns a structured (feature, importance) array sorted descending,
or a {feature: importance} dict when prettified=True
(CatBoost's flag). feature holds feature_names when given,
else the names captured from a DataFrame at fit, else column indices.
n_features truncates to the top N.
The expensive SHAP pass is cached on the instance, keyed by the data's
content, so repeated calls on the same X -- including with different
formatting options -- reuse it. The cache is dropped on refit. See
shap_values for the attribution space and cost.