bluegreencity-ki

Soil-Water Balance (SWBM) — 7-Day Forecast

This document describes the operational soil-water-balance model, its 7-day forward forecast, and how the daily pipeline simulates, persists, and exposes that forecast. It complements the broader Soil_Water_Balance_Integration.md integration contract (schema, calibration boundary, standalone numeric regression).

Overview

The daily SWBM run is a physical, day-by-day, multi-layer soil-water balance for every street tree. In addition to simulating observed weather, the operational run now simulates forecast_days (default 7) days into the future using the Open-Meteo forecast API, so every tree carries a real forward 7-day soil-moisture and stress projection alongside its observed state.

This mirrors the standalone h_da step3_realtime.py behaviour (run_end = today + FORECAST_DAYS) and is the change that closes issue #225 (review of the SWBM column set and parity with the standalone h_da real-time script).

Key properties of the operational window:

• Observed days are driven by DWD HYRAS/HOSTRADA data (with Open-Meteo gap-fill). This is the high-accuracy historical/real-time path.
• Forecast days (today … today + 7) are driven by the Open-Meteo weather forecast. They are ordinary rows — there is no special “forecast table”; the daily state table simply extends past today.
• soil_water_stress_forecast_max_7d on every row is a strictly forward-looking maximum stress over the next 7 simulated days (see “Forecast semantics” below).
• The model resumes from the most recent observed state dated on or before today, so the 7 forecast rows are re-simulated (and replaced) on every daily run while the observed history stays stable.

Implementation details

1. True forward forecast window

forward_forecast_max_stress(stress_series, index, forecast_days) (bluegreencity_ki/water_balance/simulation.py) returns the maximum layer-1 soil-water stress over the days after day index: specifically days index+1 … index+forecast_days.

• The window is strictly forward — a day’s own stress is not part of its own forecast. This is the core correction from #225: previously the forecast used a rolling window over already-simulated / current days (intermediate[index : index+forecast_days]), which is not a forward projection.
• If fewer than forecast_days future days exist (a day near the end of the window), the window is shortened to the remaining days.
• If no future day exists, the day’s own stress is returned, so the value is always defined and stays within [0.0, 1.0].

This helper is used by both public entry points, run_tree_weather_sequence (operational batch) and simulate_daily_water_balance (single-tree / scenario path), so the forecast semantics are identical for the whole pipeline, the irrigation scenario functions, and the tests.

2. Weather forcing for forecast days

bluegreencity_ki/water_balance/weather_grid.py (DwdGridWeatherProvider) supplies observed DWD rows; dates beyond the DWD cache (including future days) are not served by DWD. Those dates are filled by the Open-Meteo service via OpenMeteoWeatherProvider / HistoricalForecastWeatherProvider / PriorityWeatherProvider in bluegreencity_ki/water_balance/openmeteo.py, following the configured climate.water_balance.meteorology.openmeteo_fallback block in config/config.yaml. The forecast/archive split is governed by forecast_past_days (default 5): all dates from today − 5 onward are served by the forecast API, older dates by the archive API. The same weather pipeline therefore serves observed and forecast days uniformly; forecast rows are flagged for traceability via data_quality (forecast quality factor) and the recorded meteorology_source / model_version.

3. Operational window length

scripts/run_daily_water_balance.py now ends the run at end_date = max(configured end, today + forecast_days) when running the current operational year. run_scheduled_water_balance.ps1 (the Windows scheduled task) passes --end-date <today>, and the Python CLI extends it to today + forecast_days automatically. Historical backfill runs (--year in the past) remain purely observed and do not extend into the future.

The DWD + Open-Meteo weather providers are date-agnostic, so requesting a start_date..end_date that crosses today and reaches into the future is supported: DWD serves the observed range and Open-Meteo serves the rest.

4. Continuation / resume point

run_tree_batch(..., resume_up_to: date) (default date.today()) (bluegreencity_ki/water_balance/runner.py) picks the latest persisted state dated on or before resume_up_to as the continuation state:

• WaterBalanceStore.load_latest_states(model_versions, up_to=…) and ParquetWaterBalanceStore.load_latest_states(model_versions, up_to=…) accept an optional up_to bound (PostGIS + Parquet + mirrored stores all forward it) so the query returns the most recent observed state rather than the most recent forecast row.
• The 7 forecast rows written on the previous run are therefore not treated as history. Each daily run re-simulates today’s state forward and upserts the refreshed forecast rows (ON CONFLICT (tree_id, state_date, model_version) DO UPDATE), keeping them current without duplicating history.
• A run whose window is entirely in the past (backfill) resumes from the last state with no up_to bound, preserving historical semantics.

5. Storage

• swbm_daily_state gains no new columns for the forecast: forecast days are ordinary rows keyed by (tree_id, state_date, model_version) with state_date in the future. All soil-moisture / stress / demand columns are populated the same way as for observed days.
• Because a future state_date sorts after today, any consumer that computes “latest state = max(state_date)” must cap at today to avoid mistaking a forecast row for the current observed state (see consumers below). This is the reason for the up_to bound and the view/CLI changes.

6. Consumers that must read the observed state

The following readers anchor on the observed (≤ today / ≤ planning date) state, not the forecast tail:

Consumer	Anchor	Change
WFS view bgcki.wfs_trees_current (sql/migrations/2026-09-14_create_wfs_trees_view.sql + 2026-09-16_fix_wfs_trees_view_forecast_rows.sql)	state_date <= CURRENT_DATE	Added AND (d.state_date <= CURRENT_DATE) to the LATERAL join so the “current” row shown to partners is the observed row (which already carries _forecast_max_7d).
Operational products scripts/refresh… / bluegreencity_ki/operational_products.py	state_date <= today	build_operational_tree_products caps the per-tree latest state at pd.Timestamp.today().normalize().
Irrigation planning bluegreencity_ki/irrigation_planning/adapters.py	state_date <= planning_date	prepare_planning_candidates caps the per-tree latest state at the planning date.

Each of these readers therefore still exposes a single, stable “current” state per tree; the 7-day forecast is delivered inside that row via soil_water_stress_forecast_max_7d, plus (optionally) the raw future rows in swbm_daily_state.

7. Irrigation avoidable by rain (irrigation_avoidable_by_rain)

The flag records whether, on a day whose irrigation trigger is active, forecast rain within the forward window makes the irrigation avoidable. It mirrors the standalone h_da real-time script (step3_realtime.py), which simulates today + FORECAST_DAYS with forecast weather and evaluates the irrigation trigger on every simulated day: on a forecast day the trigger simply does not fire when forecast rain keeps the soil above the trigger threshold, so the irrigation that was needed on a previous day is effectively avoided.

irrigation_avoidable_by_rain(demand_series, index, forecast_days) (bluegreencity_ki/water_balance/simulation.py) returns True only when:

• the irrigation demand on day index is positive (trigger active — for the operational v3 demand: both layer 1 and layer 2 below trigger_soil_water_fraction_layer_1), and
• at least one of the strictly forward days index + 1 through index + forecast_days is simulated without irrigation at zero demand, i.e. rain has recharged the soil back above the trigger threshold.

If the trigger is not active on the day (there is nothing to avoid) or no future day exists, the flag is False. The same helper is used by both public entry points (run_tree_weather_sequence and simulate_daily_water_balance), so operational rows, scenario runs, and tests share identical semantics. No additional configuration is required: the horizon is the existing forecast_days window, and the “minimum rain” is implicit in the physics — rain must actually push the soil back above the trigger threshold.

Historical rows. In a backfill run the forward window contains observed follow-up days, so the flag is deterministically computable for history as well: it then reads retrospectively as “if today’s irrigation had been skipped, rain within the following 7 days would have recharged the soil above the trigger”. Rows written before this feature stay false (never computed) and can be refreshed by re-backfilling the affected years (issue #237).

Data sources

Source	Used for	Provider
DWD HYRAS (daily) + HOSTRADA (hourly)	observed weather (high accuracy, precedence on overlap)	DwdGridWeatherProvider
Open-Meteo forecast archive API	forecast / gap-fill weather, incl. future days	OpenMeteoWeatherProvider → PriorityWeatherProvider
Irrigation event table bgcki.irrigation_event	known watering events (observed + estimated)	IrrigationEvent load path
Prepared tree features GeoPackage	profiles, forcing, SVF, sunlit anchors	prepare_tree_profiles, prepare_hannover_simulation_trees

The weather selection chain is configurable under climate.water_balance.meteorology in config/config.yaml (primary → gap-fill / forecast, quality factors).

Outputs

swbm_daily_state (per tree_id, per day, per model_version)

Columns produced per simulated day (observed and forecast). Raw h_da-like soil-water values plus the forward forecast:

• soil_water_m3_total, soil_water_m3_layer_1/2/3 — layer soil water, m³ (raw h_da Bodenwassergehalt_gesamt_mm-equivalent in m³)
• soil_water_fraction_total, soil_water_fraction_layer_1/2/3
• soil_water_stress_today — layer-1 stress for the day
• soil_water_stress_forecast_max_7d — max layer-1 stress over the next 7 simulated days (strictly forward; own stress as fallback)
• irrigation_demand_liters, irrigation_trigger_date
• irrigation_avoidable_by_rain — True on trigger days where forecast rain within the 7-day forward window recharges the soil above the trigger threshold (see section 7); False otherwise
• surface_wet_fraction — throughfall-rain-driven evaporation state variable: >=10 mm throughfall → 1.0; >=4 mm → min 0.6; else carry-forward. It gates soil evaporation.
• data_quality, model_version, meteorology_source

The observed row for today carries the live 7-day forecast in soil_water_stress_forecast_max_7d. The future rows (today+1 … today+7) in the same table each additionally expose their own soil_water_stress_today (that future day’s stress) so a full 7-day trajectory can be plotted.

swbm_soil_water_stress_forecast_max_7d (WFS / operational product)

This is the single forward-forecast stress shown to end users and to the irrigation planner. It equals the observed today row’s soil_water_stress_forecast_max_7d, i.e. the maximum stress the tree is expected to reach over the next simulated 7 days given the current forecast weather.

Verification

• pytest -q tests/ -k "water_balance or operational or planning or irrigation or sunlight or products" — 181 passed (forecast-window unit tests, avoidable-by-rain flag tests, up_to resume, continuation anchor, runner / simulation / storage, operational products, irrigation planning).
• The daily scheduled task writes today … today+7 rows to swbm_daily_state; after a day, the 7 future rows are refreshed in place (same primary key) rather than appended, and “current” consumers continue to report the observed today row.

Related

• Soil_Water_Balance_Integration.md — full integration contract, standalone numeric regression, calibration boundary
• Model_Parameter_Uebersicht_2026-09-16.md — exact parameter snapshot
• config/config.yaml → water_balance — run window, forecast_days, weather chain
• scripts/run_daily_water_balance.py, scripts/run_scheduled_water_balance.ps1 — operational entry points