bluegreencity-ki

GAM-Based Explainability (XAI) for Drought Risk Scoring

⚠️ Research concept — NOT implemented

This document describes a proposed GAM-based explainability component. Neither bluegreencity_ki/ml/xai.py nor the pygam dependency nor tests/test_gam_explainer.py exists in the codebase; the code blocks below are illustrative pseudo-code for the idea. The explainability actually in use is SHAP-based (bluegreencity_ki/ml/explainability.py, see ../risk_assessment_ml/ml_methodology_story.md, Kapitel 4). Treat this page as a dated research note, not current-system documentation.

Overview

The GAMExplainer provides interpretable explanations of drought risk predictions using Generalized Additive Models (GAMs). Unlike black-box machine learning models, GAMs decompose predictions into a sum of smooth, univariate functions — making it transparent how each criterion contributes to the overall risk score.

Risk = β₀ + f₁(age_risk) + f₂(vitality_risk) + f₃(street_type_risk) + ...
         ↑          ↑              ↑                    ↑
      intercept   smooth         smooth              smooth
                  function       function             function

Each smooth function fᵢ is a spline that captures the (potentially non-linear) relationship between a single criterion score and the overall drought risk. This is the key advantage over linear models: GAMs can reveal non-linear effects, while remaining fully interpretable.

Why GAM as an XAI Component?

Property GAM Linear Regression Random Forest
Interpretable ✅ Per-feature shape functions ✅ Coefficients ❌ Black box
Non-linear effects ✅ Spline-based ❌ Linear only ✅ Tree-based
Confidence intervals ✅ Per point ✅ Global ❌ Not native
Feature importance ✅ Variance-based ✅ Coefficient magnitude ✅ Impurity/permutation
Per-tree explanations ✅ Additive decomposition ✅ Additive ⚠️ Requires SHAP

GAMs serve as an ideal explainability baseline for the BlueGreenCity-KI project:

Quick Start

from bluegreencity_ki.ml.xai import GAMExplainer
import pandas as pd

# Assume you have criterion scores and overall risk from the pipeline
features = pd.DataFrame({
    "age_risk": [...],
    "vitality_risk": [...],
    "street_type_risk": [...],
    "air_temperature_risk": [...],
})
risk_scores = pd.Series([...], name="overall_drought_risk")

# Fit the explainer
explainer = GAMExplainer(n_splines=20, lam=0.6)
explainer.fit(features, risk_scores)

# 1. Model summary
print(explainer.summary())
# {'r_squared': 0.95, 'n_features': 4, ...}

# 2. Feature importance
importance = explainer.feature_importance()
print(importance)
#            feature  importance
# 0         age_risk    0.38
# 1    vitality_risk    0.32
# 2  air_temperature    0.18
# 3  street_type_risk   0.12

# 3. Partial dependence for a single feature
pdp = explainer.partial_dependence("age_risk")
# Returns DataFrame with columns: feature_value, partial_dependence, ci_lower, ci_upper

# 4. Per-tree explanations
contributions = explainer.explain(features)
# Returns DataFrame with contribution of each feature to each tree's risk score

# 5. Predict risk scores
predictions = explainer.predict(features)

API Reference

GAMExplainer(n_splines=20, lam=0.6)

Parameters:

fit(X, y, **kwargs)

Fit the GAM to criterion scores and overall drought risk.

summary()

Return a dictionary with model statistics (R², GCV score, feature count).

partial_dependence(feature, n_points=100)

Compute the shape function for one feature.

feature_importance()

Compute normalized feature importance based on partial dependence variance.

explain(X)

Decompose predictions into per-feature, per-tree contributions.

predict(X)

Predict risk scores using the fitted GAM.

Configuration

The GAM explainer can be used after the risk scoring pipeline completes. Future integration into config.yaml:

xai:
  enabled: true
  method: "gam"
  gam:
    n_splines: 20
    lam: 0.6

How It Works

1. Fitting

The explainer takes the criterion scores (e.g., age_risk, vitality_risk, street_type_risk) as input features and the overall_drought_risk as the target. It fits a Generalized Additive Model that learns a smooth function per feature.

2. Partial Dependence

For each feature, the shape function shows how the risk changes as that criterion varies from 0 to 1, holding all other criteria at their average values. This reveals:

3. Feature Importance

Importance is computed as the standard deviation of each partial dependence curve, normalized so importances sum to 1. Features whose shape function varies more have greater influence on the risk score.

4. Per-Tree Explanations

For individual trees, the model decomposes the predicted risk into additive contributions from each criterion. This enables targeted explanations like:

“This tree’s high risk (0.82) is primarily driven by vitality_risk (+0.25) and air_temperature_risk (+0.20).”

Dependencies

Testing

# Run XAI tests
pytest tests/test_gam_explainer.py -v

# Run all ML tests
pytest tests/test_ml_labels.py tests/test_gam_explainer.py -v

References