bluegreencity-ki

Data Maintenance Tasks

This directory contains documentation for routine data maintenance tasks that should be performed periodically to keep the BlueGreenCity-KI system up-to-date and running smoothly.

Overview

Regular data maintenance ensures:

Current Maintenance Tasks

Tree Inventory Enrichment

Frequency: After changes to enrichment logic, matching rules, or upstream source schemas
Documentation: enrichment.md

Documents the runtime enrichment applied during tree loading, including OBSkH spatial joins and Citree species-profile mapping from ga_lang to citree_* output fields.

Tree Inventory Import

Frequency: After changes to CSV schema, loader logic, or domain interpretation
Documentation: Tree_Inventory_Import.md

Documents how the Hannover tree inventory CSV is loaded at runtime, how long field names are mapped to pipeline columns, how invalid geometries are handled, and how raw rows are separated from analysis rows.

Baumkataster v6 – LHH-Zahlendiskrepanz

Frequency: One-off note, updated when the LHH confirms its counting basis
Documentation: Baumkataster_v6_Diskrepanz.md

Documents the discrepancy between raw v6 row counts and the tree counts communicated by the LHH (108.368 vs. 89.510 current; 96.674 vs. 82.481 historical 2025) and its consequences for reporting.

Tree Inventory Status Matrix

Frequency: After changes to domain rules for planned, removed, or placeholder entries
Documentation: Tree_Inventory_Status_Matrix.md

Defines the operational status taxonomy derived from the raw CSV, including planned_not_planted, removed_tree, replacement_marker, object_control_placeholder, erroneous_record, and active_tree.

Open review backlog: citree_mapping_review.md

Citree Species Extraction

Frequency: As needed after upstream Citree changes
Documentation: Citree_Species_Extraction.md

Refresh the locally stored Citree species criteria catalog used for tolerance and profile-based downstream mappings.

Quick command:

python scripts/extract_citree_species_catalog.py

OSM Highway Data Update

Frequency: Annually (October recommended)
Documentation: OSM_Highway_Data_Update.md

Update OpenStreetMap highway data used for street type risk assessment. This captures new roads, road type changes, and removes deleted roads.

Quick command:

python scripts/import_osm_highways_to_postgis.py

Climate Data Import

Frequency: As needed (monthly to annually)
Documentation: Climate_Data_Import.md

Import historical and current climate data from DWD CDC (HYRAS/HOSTRADA) or Open-Meteo API for drought risk assessment and water balance modeling.

Quick commands:

# DWD CDC (HYRAS daily data)
python scripts/import_climate_data_dwd.py --dataset hyras --variable precipitation --year 2022

# Open-Meteo API (hourly data)
python scripts/import_climate_data_openmeteo.py --variables temperature_2m,precipitation --start-date 2022-01-01 --end-date 2022-12-31

Irrigation Planning

Frequency: Daily after the SWBM state and current risk outputs are available Documentation: Irrigation_Planning.md

Creates a versioned, resource-constrained multi-day irrigation draft from SWBM demand, risk products, weather signals, and aggregate daily capacities.

Irrigation Review and Execution

Frequency: After each planning run and during field execution Documentation: Irrigation_Review_Taipy.md

Documents Taipy resource scenarios, mandatory review reasons, approval-derived work orders, and structured actual irrigation reports.

Model Parameter Overview (Point-in-Time)

Frequency: After any change to model code or config/config.yaml
Documentation: Model_Parameter_Uebersicht_2026-09-16.md (code-verified parameter snapshot; the 2026-08-29 snapshot is archived)

Code-verified snapshot of all exact parameters in the three models (SWBM, ML, rule-based) as of commit 554e391 (2026-09-16): criteria weights, SWBM soil/vegetation/meteorology parameters, and the AutoGluon training/inference configuration. The previous snapshot (881e42f, 2026-08-29) is archived in risk_assessment_ml/archive/. Re-create this snapshot whenever model parameters change.

Scheduling Recommendations

The following timeline is an operational recommendation for recurring maintenance work. Some items already have dedicated runbooks in this directory, while others depend on the currently available upstream delivery format and can be documented in more detail when the workflow is standardized.

Suggested Timeline

Frequency Task Recommended Timing Reason
Annual OSM Highway Update October Aligns with tree inventory cycles
Annual Tree Inventory Import October Coordinate with city data releases and downstream refresh cycles
Annual Climate Data Import (Historical) November After growing season ends
Monthly/Weekly Climate Data Import (Current) As needed For real-time monitoring
Quarterly Database Optimization March, June, September, December Regular performance maintenance for indexes, statistics, and cache hygiene
Every 2-3 years Raster Data Update As available Triggered by publication of newer land-use, NDVI, or climate rasters
Every 2-3 years Species Factors Review As needed Re-check species mappings against updated source tables or domain knowledge

Best Practices

  1. Create backups before performing updates
  2. Document changes and update dates in change logs
  3. Verify data quality after each update
  4. Test in development before production updates
  5. Coordinate timing with related system updates
  6. Monitor data freshness using cache expiration dates

Contributing

If you identify additional data maintenance tasks, please:

  1. Document the procedure in this directory
  2. Update this README with the new task
  3. Add to the suggested timeline with appropriate frequency
  4. Clarify whether the item is a documented runbook or an operational recommendation