ChimeraBoostQuantileRegressor

A whole grid of conditional quantiles from a single booster, with levels that cannot cross. See the User Guide: predictive distributions.

Bases: BaseEstimator

Gradient boosting for a whole predictive distribution at once.

One booster, one tree structure per round, and a K-vector in every leaf, one entry per level in quantiles. Against one quantile regressor per level that is roughly K times less split-search work, and the predictions cannot cross: the 30% quantile is never returned above the 70%.

Ordering is enforced per row by monotone rearrangement of the delivered scores. It is exact for every row and holds at every intermediate stage of staged_predict. Rearrangement cannot cost accuracy -- sorting a crossing quantile curve never increases pinball loss at any level (Chernozhukov, Fernandez-Val & Galichon 2010).

Deliberately not a RegressorMixin: predict returns a matrix, so the inherited score (which assumes one number per row) would be wrong. score here is negative CRPS, so higher is better, as sklearn requires.

Read more in the User Guide.

Parameters:

Name	Type	Description	Default
quantiles	array - like or None	Ascending, unique levels strictly inside (0, 1). Default 0.05, 0.10, ... 0.95. Column k of predict is level k.	None
split_projection	(rotate, sum, gram)	How split gain is scored across the quantile levels. Keep the default unless you are experimenting. "rotate" alternates a location and a spread contrast, two location rounds per spread round, and measured best. "sum" adds the levels up, which makes it blind to a change in spread. "gram" picks the strongest contrast each round and measured no better than rotating.	"rotate"
exact_splits	bool	Score splits exactly across every level instead of on a projection. More faithful, but the fit gets slower and more memory-hungry as the grid grows -- a reference setting, not one for routine use.	False
conformalize	bool	Calibrate the intervals by conformalized quantile regression. Carves calibration_fraction of the rows off BEFORE the early-stopping split, so that fold influences neither the fit nor the stopping point. Raises if the fold is too small to certify the requested levels.	False
calibration_fraction	float	Share of training rows reserved for conformalization. Ignored unless conformalize=True.	0.2

Attributes:

Name	Type	Description
quantiles_	ndarray of shape (n_quantiles,)	The resolved grid.
conformal_scale_	ndarray of shape (n_quantiles,)	Per-level conformal scale about the predicted median; all ones unless conformalize=True.

Notes

Every other parameter carries its usual ChimeraBoost meaning. Two defaults are set for this head rather than inherited: depth is 4, because deep leaves overfit tail quantiles, and min_child_weight follows a floor implied by the most extreme level on the grid.

Source code in chimeraboost/quantile_api.py

def __init__(self, quantiles=None, n_estimators=2000, learning_rate=None,
             depth=None, l2_leaf_reg=1.0, max_bins=128, subsample=1.0,
             colsample=None, cat_smoothing=1.0, cat_n_permutations=4,
             early_stopping_rounds=None, min_child_weight=None,
             thread_count=None, random_state=None, verbose=False,
             cat_features=None, cat_combinations=None,
             quantize_gradients=True, early_stopping=True,
             validation_fraction=0.2, split_projection="rotate",
             exact_splits=False, conformalize=False,
             calibration_fraction=0.2):
    self.quantiles = quantiles
    self.n_estimators = n_estimators
    self.learning_rate = learning_rate
    self.depth = depth
    self.l2_leaf_reg = l2_leaf_reg
    self.max_bins = max_bins
    self.subsample = subsample
    self.colsample = colsample
    self.cat_smoothing = cat_smoothing
    self.cat_n_permutations = cat_n_permutations
    self.early_stopping_rounds = early_stopping_rounds
    self.min_child_weight = min_child_weight
    self.thread_count = thread_count
    self.random_state = random_state
    self.verbose = verbose
    self.cat_features = cat_features
    self.cat_combinations = cat_combinations
    self.quantize_gradients = quantize_gradients
    self.early_stopping = early_stopping
    self.validation_fraction = validation_fraction
    self.split_projection = split_projection
    self.exact_splits = exact_splits
    self.conformalize = conformalize
    self.calibration_fraction = calibration_fraction

validation_history_ property

validation_history_

Per-round validation CRPS from fit (empty without a validation set).

fit

fit(
    X,
    y,
    cat_features=None,
    eval_set=None,
    groups=None,
    sample_weight=None,
    callbacks=None,
)

Fit the model. Arguments carry the same meaning as ChimeraBoostRegressor.fit.

Source code in chimeraboost/quantile_api.py

def fit(self, X, y, cat_features=None, eval_set=None, groups=None,
        sample_weight=None, callbacks=None):
    """Fit the model. Arguments carry the same meaning as
    `ChimeraBoostRegressor.fit`."""
    cat_features = _resolve_cat_features(self, cat_features)
    cat_features = _resolve_cat_feature_names(cat_features, X)

    _validate_hyperparams(self)
    if self.split_projection not in ("rotate", "sum", "gram"):
        raise ValueError(
            'split_projection must be "rotate", "sum" or "gram"; got '
            f"{self.split_projection!r}.")
    if not 0.0 < self.calibration_fraction < 1.0:
        raise ValueError("calibration_fraction must be in (0, 1); got "
                         f"{self.calibration_fraction!r}.")

    y = _validate_fit_input(self, X, y, cat_features, sample_weight,
                            classification=False)
    if eval_set is not None:
        _check_eval_set(eval_set, self.n_features_in_)
        _check_feature_names_match(self, eval_set[0])

    taus = _resolve_quantiles(self.quantiles)
    self.quantiles_ = taus
    self._median_idx_ = _median_index(taus)
    self.conformal_scale_ = np.ones(taus.shape[0])

    X = as_model_array(X, bool(cat_features))
    y = np.asarray(y, dtype=np.float64)
    if sample_weight is not None:
        sample_weight = np.asarray(sample_weight, dtype=np.float64)

    cal, X, y, sample_weight, groups = self._carve_calibration_fold(
        X, y, sample_weight, groups)
    X, y, sample_weight, eval_set, es_rounds = self._carve_es_split(
        X, y, sample_weight, eval_set, groups)

    self.model_ = self._make_mq_booster(taus, es_rounds, cat_features,
                                        len(y))
    self.model_.fit(X, y, cat_features=cat_features, eval_set=eval_set,
                    sample_weight=sample_weight, callbacks=callbacks)

    # 3. Conformalize on the pristine fold.
    if cal is not None:
        mi, mw = self._median_idx_
        self.conformal_scale_ = _cqr_scales(
            self.model_.predict_raw(cal[0]), cal[1], taus, mi, mw)

    return self

predict

predict(
    X,
    kind="quantiles",
    alpha=None,
    thresholds=None,
    n_samples=None,
    random_state=None,
)

Predict the conditional distribution.

kind="quantiles" (default) returns (n_samples, n_quantiles), column k being level quantiles_[k], non-decreasing along axis 1.

kind="interval" returns (n_samples, 2): the central 1 - alpha interval, read off the alpha/2 and 1 - alpha/2 levels, which must both be on the grid. No interpolation -- an interval the model was not fitted for is an error, not a guess.

kind="mean" returns (n_samples,), the integral of the quantile function over tau: trapezoid across the grid plus flat extension of the edge levels out to 0 and 1. The flat extension is the honest reading of a finite grid, assuming nothing about tails the model never estimated.

kind="median" returns (n_samples,), the predicted median -- the 0.5 level when the grid carries it, interpolated between its neighbours when it does not. This is the centre conformalization rescales about.

kind="cdf" returns (n_samples, n_thresholds): P(y <= t) for each t in thresholds, by inverting the grid. A 1-D thresholds is shared by every row; a 2-D (n_samples, T) array is read row against row -- the rule is dimensionality, never length. Clamped to the outermost fitted levels rather than to 0 and 1, for the same reason "mean" extends flat. Warns when the fitted grid is too coarse to carry a CDF (any inter-level gap above 0.2).

kind="sample" returns (n_samples_rows, n_samples): inverse- transform draws from the predicted distribution, for feeding a downstream simulation. random_state seeds them; draws stay inside the fitted level range.

Unlike "interval", "cdf" and "sample" interpolate between levels. That is a different question -- reading a fitted curve at a point, rather than claiming a level was fitted when it was not.

Source code in chimeraboost/quantile_api.py

def predict(self, X, kind="quantiles", alpha=None, thresholds=None,
            n_samples=None, random_state=None):
    """Predict the conditional distribution.

    ``kind="quantiles"`` (default) returns (n_samples, n_quantiles),
    column k being level ``quantiles_[k]``, non-decreasing along axis 1.

    ``kind="interval"`` returns (n_samples, 2): the central ``1 - alpha``
    interval, read off the ``alpha/2`` and ``1 - alpha/2`` levels, which
    must both be on the grid. No interpolation -- an interval the model was
    not fitted for is an error, not a guess.

    ``kind="mean"`` returns (n_samples,), the integral of the quantile
    function over tau: trapezoid across the grid plus flat extension of the
    edge levels out to 0 and 1. The flat extension is the honest reading of
    a finite grid, assuming nothing about tails the model never estimated.

    ``kind="median"`` returns (n_samples,), the predicted median -- the
    0.5 level when the grid carries it, interpolated between its
    neighbours when it does not. This is the centre conformalization
    rescales about.

    ``kind="cdf"`` returns (n_samples, n_thresholds): ``P(y <= t)`` for
    each ``t`` in ``thresholds``, by inverting the grid. A 1-D
    ``thresholds`` is shared by every row; a 2-D (n_samples, T) array is
    read row against row -- the rule is dimensionality, never length.
    Clamped to the outermost fitted levels rather than to 0 and 1, for
    the same reason ``"mean"`` extends flat. Warns when the fitted grid
    is too coarse to carry a CDF (any inter-level gap above 0.2).

    ``kind="sample"`` returns (n_samples_rows, n_samples): inverse-
    transform draws from the predicted distribution, for feeding a
    downstream simulation. ``random_state`` seeds them; draws stay inside
    the fitted level range.

    Unlike ``"interval"``, ``"cdf"`` and ``"sample"`` interpolate between
    levels. That is a different question -- reading a fitted curve at a
    point, rather than claiming a level was fitted when it was not.
    """
    Xv = _check_predict_input(self, X)
    Q = self._conformalize(
        self.model_.predict_raw(X if Xv is None else Xv))

    if kind == "quantiles":
        return Q

    if kind == "interval":
        i, j = self._interval_levels(alpha)
        return np.column_stack([Q[:, i], Q[:, j]])

    if kind == "mean":
        return self._mean_from_quantiles(Q)

    if kind == "median":
        mi, mw = self._median_idx_
        return _centre(Q, mi, mw)

    if kind == "cdf":
        return self._cdf_from_quantiles(Q, thresholds)

    if kind == "sample":
        return self._sample_from_quantiles(Q, n_samples, random_state)

    raise ValueError(
        'kind must be "quantiles", "interval", "mean", "median", "cdf" '
        f'or "sample"; got {kind!r}.')

predict_thresh

predict_thresh(X, thresholds, direction='greater')

Probability of the target landing beyond thresholds.

direction="greater" returns P(y > t); "less" returns P(y <= t). Both read the same fitted quantile function as predict(kind="cdf"): linear between grid levels, clamped to the outermost fitted levels outside them -- on the default grid no probability reads below 0.05 or above 0.95, because the model never estimated those tails. A coarse grid (any inter-level gap above 0.2, e.g. quantiles=[0.1, 0.5, 0.9]) warns: the probabilities would be mostly interpolation, not estimates.

thresholds may be a scalar (one probability per row, returned 1-D), a 1-D array of T values shared by every row (returns (n_samples, T)), or a 2-D (n_samples, T) array read row against row (returns (n_samples, T)). The rule is dimensionality, never length: stack several per-row threshold lists with np.column_stack.

Source code in chimeraboost/quantile_api.py

def predict_thresh(self, X, thresholds, direction="greater"):
    """Probability of the target landing beyond ``thresholds``.

    ``direction="greater"`` returns ``P(y > t)``; ``"less"`` returns
    ``P(y <= t)``. Both read the same fitted quantile function as
    ``predict(kind="cdf")``: linear between grid levels, clamped to the
    outermost fitted levels outside them -- on the default grid no
    probability reads below 0.05 or above 0.95, because the model never
    estimated those tails. A coarse grid (any inter-level gap above 0.2,
    e.g. ``quantiles=[0.1, 0.5, 0.9]``) warns: the probabilities would
    be mostly interpolation, not estimates.

    ``thresholds`` may be a scalar (one probability per row, returned
    1-D), a 1-D array of T values shared by every row (returns
    (n_samples, T)), or a 2-D (n_samples, T) array read row against row
    (returns (n_samples, T)). The rule is dimensionality, never length:
    stack several per-row threshold lists with ``np.column_stack``.
    """
    if direction not in ("greater", "less"):
        raise ValueError(
            f'direction must be "greater" or "less"; got {direction!r}.')
    if thresholds is None:
        raise ValueError(
            "predict_thresh needs `thresholds`: the values t to compare "
            "the target against.")
    Xv = _check_predict_input(self, X)
    Q = self._conformalize(
        self.model_.predict_raw(X if Xv is None else Xv))
    cdf = self._cdf_from_quantiles(Q, thresholds)
    if np.ndim(thresholds) == 0:
        cdf = cdf[:, 0]
    return 1.0 - cdf if direction == "greater" else cdf

staged_predict

staged_predict(X)

Yield the (n, K) quantile matrix after each successive tree. The conformal rescaling is a post-fit transform, so it is applied at every stage and the last one equals predict.

Source code in chimeraboost/quantile_api.py

def staged_predict(self, X):
    """Yield the (n, K) quantile matrix after each successive tree. The
    conformal rescaling is a post-fit transform, so it is applied at every
    stage and the last one equals ``predict``."""
    Xv = _check_predict_input(self, X)
    X = X if Xv is None else Xv

    for staged in self.model_.staged_predict_raw(X):
        yield self._conformalize(staged)

score

score(X, y, sample_weight=None)

Negative CRPS (mean pinball loss over the grid). Higher is better, per the sklearn convention.

Source code in chimeraboost/quantile_api.py

def score(self, X, y, sample_weight=None):
    """Negative CRPS (mean pinball loss over the grid). Higher is better,
    per the sklearn convention."""
    return -quantile_metrics.crps(y, self.predict(X), self.quantiles_,
                                  sample_weight)

report

report(X, y, sample_weight=None, baseline=None)

quantile_metrics.quantile_report on this model's predictions: CRPS and its skill score, per-level pinball, coverage plus width plus interval score for every symmetric interval, the PIT histogram, and the crossing rate.

baseline sets what the skill score is measured against -- pass the training targets to score against the marginal distribution the model actually had, rather than the hindsight marginal of y.

Source code in chimeraboost/quantile_api.py

def report(self, X, y, sample_weight=None, baseline=None):
    """`quantile_metrics.quantile_report` on this model's predictions:
    CRPS and its skill score, per-level pinball, coverage plus width plus
    interval score for every symmetric interval, the PIT histogram, and
    the crossing rate.

    ``baseline`` sets what the skill score is measured against -- pass the
    training targets to score against the marginal distribution the model
    actually had, rather than the hindsight marginal of ``y``.
    """
    return quantile_metrics.quantile_report(y, self.predict(X),
                                            self.quantiles_, sample_weight,
                                            baseline)

shap_values

shap_values(
    X,
    X_background=None,
    kind="quantiles",
    alpha=None,
    quantile=None,
    space="delivered",
)

Exact interventional TreeSHAP for a predicted quantile grid.

Explains what predict returned. Contributions plus expected_value_ (set by this call) reconstruct it, level by level.

kind selects the explained quantity:

• "quantiles" -- (n_samples, n_features, n_quantiles), or (n_samples, n_features) when quantile names one fitted level.
• "mean" -- the tau-integrated point prediction.
• "width" -- the width of the central 1 - alpha interval: which features make this row's prediction more uncertain, as opposed to higher or lower. Shapley values are linear in the value function, so the difference of two levels' attributions is exactly the attribution of their difference.

space is for one specific job and most callers can ignore it. Predictions are rearranged on delivery, which relabels a row's levels, so the default "delivered" measures each row against its own reordering of the background and expected_value_ is (n_samples, n_quantiles). Aggregating those across rows mixes rows that were reordered differently. space="raw" explains the pre-rearrangement levels instead, against one shared (n_quantiles,) baseline, which is what makes a cross-row average meaningful -- shap_importances uses it for exactly that reason. The two agree on any row whose levels were already in order. "mean" and "width" are order-dependent by construction and always read the delivered grid.

Source code in chimeraboost/quantile_api.py

def shap_values(self, X, X_background=None, kind="quantiles", alpha=None,
                quantile=None, space="delivered"):
    """Exact interventional TreeSHAP for a predicted quantile grid.

    Explains what ``predict`` returned. Contributions plus
    ``expected_value_`` (set by this call) reconstruct it, level by level.

    ``kind`` selects the explained quantity:

    * ``"quantiles"`` -- ``(n_samples, n_features, n_quantiles)``, or
      ``(n_samples, n_features)`` when ``quantile`` names one fitted level.
    * ``"mean"`` -- the tau-integrated point prediction.
    * ``"width"`` -- the width of the central ``1 - alpha`` interval: which
      features make this row's prediction more uncertain, as opposed to
      higher or lower. Shapley values are linear in the value function, so
      the difference of two levels' attributions is exactly the attribution
      of their difference.

    ``space`` is for one specific job and most callers can ignore it.
    Predictions are rearranged on delivery, which relabels a row's levels,
    so the default ``"delivered"`` measures each row against its own
    reordering of the background and ``expected_value_`` is
    ``(n_samples, n_quantiles)``. Aggregating those across rows mixes rows
    that were reordered differently. ``space="raw"`` explains the
    pre-rearrangement levels instead, against one shared
    ``(n_quantiles,)`` baseline, which is what makes a cross-row average
    meaningful -- `shap_importances` uses it for exactly that reason. The
    two agree on any row whose levels were already in order.
    ``"mean"`` and ``"width"`` are order-dependent by construction and
    always read the delivered grid.
    """
    if space not in ("raw", "delivered"):
        raise ValueError(
            f'space must be "raw" or "delivered"; got {space!r}.')
    if kind not in ("quantiles", "mean", "width"):
        raise ValueError(
            f'kind must be "quantiles", "mean" or "width"; got {kind!r}.')

    Xv = _check_predict_input(self, X)
    X = X if Xv is None else Xv
    if X_background is not None:
        # Consumed positionally, so a reordered DataFrame would silently
        # skew every baseline.
        bg = _check_predict_input(self, X_background)
        X_background = X_background if bg is None else bg

    phi, base = self.model_.shap_values(X, background=X_background)

    if kind == "quantiles" and space == "raw":
        if quantile is not None:
            k = self._level_index(quantile)
            phi, base = phi[:, :, k], base[k]
        self.expected_value_ = base
        return phi

    phi, base = self._delivered_shap(phi, base,
                                     self.model_._raw_scores(X))

    if kind == "mean":
        w = self._mean_from_quantiles(np.eye(self.quantiles_.shape[0]))
        self.expected_value_ = base @ w
        return phi @ w

    if kind == "width":
        i, j = self._interval_levels(
            alpha, caller='shap_values(kind="width")')
        self.expected_value_ = base[:, j] - base[:, i]
        return phi[:, :, j] - phi[:, :, i]

    if quantile is not None:
        k = self._level_index(quantile)
        phi, base = phi[:, :, k], base[:, k]
    self.expected_value_ = base
    return phi

shap_importances

shap_importances(
    X,
    feature_names=None,
    n_features=None,
    prettified=False,
    quantile=None,
)

Global SHAP importance: mean(abs(shap_values(X))) per feature.

Averaged over the whole grid by default, or over one level when quantile names it. Uses space="raw" so that every row is measured on the same footing -- see shap_values -- which is what a cross-row average needs.

Returns a structured (feature, importance) array sorted descending, or a {feature: importance} dict when prettified=True.

Source code in chimeraboost/quantile_api.py

def shap_importances(self, X, feature_names=None, n_features=None,
                     prettified=False, quantile=None):
    """Global SHAP importance: ``mean(abs(shap_values(X)))`` per feature.

    Averaged over the whole grid by default, or over one level when
    ``quantile`` names it. Uses ``space="raw"`` so that every row is
    measured on the same footing -- see `shap_values` -- which is what a
    cross-row average needs.

    Returns a structured ``(feature, importance)`` array sorted descending,
    or a ``{feature: importance}`` dict when ``prettified=True``.
    """
    phi = self.shap_values(X, quantile=quantile, space="raw")
    imp = np.abs(phi).mean(axis=0)
    if imp.ndim == 2:                      # (n_features, n_quantiles)
        imp = imp.mean(axis=1)
    return _format_shap_importances(
        self, imp, feature_names=feature_names, n_features=n_features,
        prettified=prettified)