Recipes¶
Imports used throughout:
Regression and classification¶
reg = ChimeraBoostRegressor(random_state=0).fit(X_train, y_train)
y_pred = reg.predict(X_test)
clf = ChimeraBoostClassifier(random_state=0).fit(X_train, y_train)
labels = clf.predict(X_test) # original label values
proba = clf.predict_proba(X_test) # columns follow clf.classes_
A plain fit(X, y) early-stops on an internal holdout — see Early stopping.
Categorical features¶
Pass your categoricals as cat_features, by integer position or — for a DataFrame — by
column name (or a mix of both). They are encoded with ordered target statistics
(CatBoost-style), so there is no one-hot or LabelEncoder step. Categorical columns can be
strings or objects; the rest of the matrix stays numeric.
# columns 0 and 3 are categorical (e.g. "city", "device_type")
clf = ChimeraBoostClassifier(random_state=0)
clf.fit(X, y, cat_features=[0, 3])
# equivalently, by name when X is a DataFrame
clf.fit(df, y, cat_features=["city", "device_type"])
cat_combinations adds all pairwise category-by-category features. They help when the
target depends on categorical interactions but can crowd out numerics on mixed data, so
the default (None) turns them on automatically only when the data is entirely
categorical. Force them with cat_combinations=True (e.g. on mixed data where you know
the interactions matter) or disable with False.
Missing values¶
NaNs route to a dedicated histogram bin — no imputation needed. This works for both numeric and categorical columns, at fit and at predict time.
Quantile regression¶
Set loss="Quantile" and the level alpha. For a prediction interval, fit one model
per quantile:
lo = ChimeraBoostRegressor(loss="Quantile", alpha=0.05, random_state=0).fit(X_train, y_train)
md = ChimeraBoostRegressor(loss="Quantile", alpha=0.50, random_state=0).fit(X_train, y_train)
hi = ChimeraBoostRegressor(loss="Quantile", alpha=0.95, random_state=0).fit(X_train, y_train)
lower, median, upper = lo.predict(X_test), md.predict(X_test), hi.predict(X_test)
loss="MAE" gives median regression; loss="RMSE" (default) is squared error.
Quantile models default to a shallower tree (depth=4) than the squared-error
default (depth=6): an extreme conditional quantile is estimated from the points in
each leaf, so deep, sparse leaves overfit the tails and the predicted quantiles
collapse toward the median on held-out data. Predictions also include a
split-conformal correction (quantile_offset_) fitted on the early-stopping
validation split, which restores near-nominal marginal coverage at the tails.
With early_stopping=False and no eval_set there is no split to calibrate on,
and the raw (typically under-dispersed) quantiles are returned.
Multiclass classification¶
No configuration needed — the classifier switches to softmax when it sees 3 or more
classes, and classes_ preserves your original labels.
clf = ChimeraBoostClassifier(random_state=0).fit(X, y) # 3+ classes
proba = clf.predict_proba(X_test) # shape (n_samples, n_classes)
linear_leaves and shap_values are binary/regression only; multiclass uses constant
leaves and raises NotImplementedError from shap_values.
Sample weights¶
w = np.where(y_train == 1, 5.0, 1.0) # upweight the positive class
clf = ChimeraBoostClassifier(random_state=0)
clf.fit(X_train, y_train, sample_weight=w)
Weights are normalized to mean 1 internally and apply to training only; the early-stopping metric stays unweighted.
Bagging¶
n_ensembles trains that many models on random row samples and averages them —
regressors average predictions, classifiers soft-vote calibrated probabilities.
Each member trains on max_samples (default 0.8) of the rows drawn without
replacement — measurably stronger and faster than the classic bootstrap —
and early-stops on its own unsampled rows.
Recommended size is n_ensembles=8 (benchmarked stronger than 5 at similar
cost). Avoid n_ensembles=2: two members measure worse than one model.
Inside a bag, parameters left on auto resolve to tuned member defaults —
currently learning_rate=0.15 and colsample=0.85 — because averaging
tolerates coarser, cheaper members. The fit warns once when this happens
(a filterable UserWarning), member_params_ records what was applied,
and passing explicit values disables it.
Members fit in parallel worker processes by default, splitting the thread
budget so a bagged fit uses the same cores a single fit would; pass
ensemble_n_jobs=1 to fit them sequentially instead.
feature_importances_ and shap_values average across the bag automatically.
Early stopping¶
Early stopping is on by default. With no eval_set, the estimator holds out a
validation split (validation_fraction=0.2, stratified for classifiers), stops after a
plateau, and keeps the best round.
# default: automatic internal holdout
m = ChimeraBoostRegressor(random_state=0).fit(X_train, y_train)
print(m.best_iteration_)
# explicit validation set (overrides the internal split)
m = ChimeraBoostRegressor(random_state=0)
m.fit(X_train, y_train, eval_set=(X_val, y_val))
# grouped split: keep each group entirely in train or validation
m.fit(X_train, y_train, groups=subject_ids)
# fixed number of trees, no stopping
m = ChimeraBoostRegressor(early_stopping=False, n_estimators=500, random_state=0)
m.fit(X_train, y_train)
After fitting, validation_history_ holds the per-round validation loss, and the
regressor's staged_predict(X) yields the prediction after each successive tree
(not defined for a bagged ensemble).
Calibrated probabilities¶
predict_proba is temperature-scaled on the validation split to minimize log loss. The
scaling is monotonic, so predict(), AUC, and accuracy are unchanged while the
probabilities themselves are better calibrated.
clf = ChimeraBoostClassifier(random_state=0).fit(X_train, y_train)
proba = clf.predict_proba(X_test) # already calibrated
print(clf.temperature_) # > 1 means raw scores were over-confident
Feature importance¶
feature_importances_ is total split gain per input column, normalized to sum to 1
(averaged across the bag when n_ensembles > 1).
m = ChimeraBoostRegressor(random_state=0).fit(X_train, y_train)
for j in np.argsort(m.feature_importances_)[::-1][:5]:
print(f"feature {j}: {m.feature_importances_[j]:.3f}")
Gain reflects what the trees split on, not how much each feature moves a given prediction, and it ignores the per-leaf linear models. For a faithful decomposition of the output, use SHAP.
Cross-validation and hyperparameter search¶
The estimators are standard scikit-learn objects:
from sklearn.model_selection import cross_val_score, GridSearchCV
scores = cross_val_score(
ChimeraBoostRegressor(random_state=0), X, y, cv=5,
scoring="neg_root_mean_squared_error",
)
search = GridSearchCV(
ChimeraBoostRegressor(random_state=0),
{"depth": [6, 8, 10], "l2_leaf_reg": [1.0, 3.0]},
cv=5,
)
search.fit(X, y)
print(search.best_params_)
To pass cat_features through a search, set it on the constructor —
ChimeraBoostClassifier(cat_features=["city", "brand"]) — so the meta-estimator
carries it (a fit-only kwarg can't be).
Save and load a model¶
A fitted estimator pickles like any scikit-learn object:
Interaction-heavy regression¶
The default depth=6 is conservative to protect small data. On large, interaction-heavy
problems, raise depth to give each tree more interaction capacity:
Per-leaf linear models add local slope inside each leaf (on by default for binary classification; the regression default picks the better of linear and constant on the validation split — force one variant to skip the double fit):