bluegreencity-ki

Tree Inventory Enrichment

Overview

The enrichment module spatially joins external datasets onto the main tree inventory during data loading, before any criteria are evaluated. This ensures all downstream criteria benefit from enriched attributes (e.g., crown diameter, tree height) without needing to know about the external data source.

Currently supported enrichment sources:

Source Dataset Features Key Attributes
OBSkH pkt_baeume_obskh.shp ~143,748 Crown diameter, tree height, NDVI, distance to buildings
Citree citree_species_criteria.csv 365 profiles Species tolerances, crown form, root pattern, growth rate

Architecture

Where enrichment fits in the pipeline

┌─────────────────────────────────────────────────────────────┐
│  Context.from_yaml(config.yaml)                             │
│  └── readers.load_inputs(cfg)                               │
│                                                             │
│      ① v.load_layer(trees_spec)    ← Load tree inventory    │
│           ↓                                                 │
│      ② enrich_trees(trees, cfg)    ← ENRICHMENT MODULE      │
│           │                                                 │
│           ├─ Load OBSkH shapefile                           │
│           ├─ Spatial nearest-neighbor join (≤ 3 m)          │
│           ├─ Crown diameter merge with QA tracking          │
│           ├─ Load Citree CSV                                │
│           ├─ Parse ga_lang → botanisch/common/cultivar      │
│           ├─ Match against botanischer_name + sorte         │
│           └─ Return enriched GeoDataFrame                   │
│           ↓                                                 │
│      ③ vector = {"trees": enriched_trees}                   │
│      ④ Load rasters (unchanged)                             │
│                                                             │
│  → ctx.vector["trees"] now has obskh_* columns              │
└─────────────────────────────────────────────────────────────┘
          ↓
┌─────────────────────────────────────────────────────────────┐
│  risk_scoring.run(ctx)                                      │
│                                                             │
│  All criteria read from ctx.vector["trees"]:                │
│    • NDVI criterion uses enriched kr_durchm for             │
│      zonal statistics over crown buffer                     │
│    • Other criteria unaffected (enrichment is additive)     │
└─────────────────────────────────────────────────────────────┘

Design decisions

  1. Enrichment in I/O layer, not in criteria – The spatial join is a data preparation step, not a scoring step. Placing it in readers.py ensures clean separation of concerns.

  2. Additive columns only – OBSkH attributes are added with an obskh_ prefix and Citree profile attributes with a citree_ prefix. Original inventory columns are never deleted.

Taxonomy helper fields derived from ga_lang are also added additively: species_genus, species_binomial, species_cultivar, species_common_name, species_is_placeholder.

  1. Crown diameter is the exception – kr_durchm is updated in-place because the enriched value should be used by all criteria. The original inventory value is backed up in kr_durchm_inventory_original.

  2. Configurable and optional – Enrichment is controlled via config.yaml. Setting enabled: false or removing the section skips enrichment entirely.

Source code

File Purpose
bluegreencity_ki/io/enrichment.py Enrichment logic (spatial join, crown merge, QA)
bluegreencity_ki/io/readers.py Integration point (enrich_trees() call after tree loading)
tests/test_enrichment.py 20 unit tests

Configuration

config.yaml – Enrichment section

inputs:
  enrichment:
    obskh:
      enabled: true
      # Path relative to raster_base_dir (or absolute)
      path: "20250729_pkt_baeume_obskh/pkt_baeume_obskh.shp"
      # Maximum distance (meters) for nearest-neighbor spatial join.
      # Accounts for GPS inaccuracies between datasets.
      join_tolerance_m: 3.0
      # OBSkH column containing crown diameter (meters)
      crown_diameter_field: "krone"
    citree:
      enabled: true
      path: "./data/derived/species/citree_species_criteria.csv"
      species_field: "ga_lang"
      columns:
        "Lichtanspruch": "citree_lichtanspruch"
        "Trockenheitstoleranz": "citree_trockenheitstoleranz"
        "Hitzeverträglichkeit": "citree_hitzevertraeglichkeit"
        "Kronenform": "citree_kronenform"
Parameter Type Default Description
enabled bool false Enable/disable OBSkH enrichment
path string – Path to OBSkH shapefile (relative to raster_base_dir or absolute)
join_tolerance_m float 3.0 Max distance in meters for spatial match
crown_diameter_field string "krone" Column name for crown diameter in OBSkH dataset
crown_diameter_max_m float 35.0 Plausibility limit: values above this are set to NaN
citree.path string – Repository-relative path to the extracted Citree CSV
citree.species_field string "ga_lang" Tree column used for species-profile matching
citree.columns mapping – Citree source label → output column name (citree_*)

OBSkH Dataset

Source

Methodology & data quality

The dataset was derived by the Fachbereich Geoinformation using remote sensing techniques based on DGM (Digital Terrain Model) and DSM (Digital Surface Model) data. Tree locations, heights, and crown dimensions were extracted from the difference between DSM and DGM (normalized Digital Surface Model, nDSM).

The exact methodology is not documented in detail. However, it is important to note that the data should be treated with caution: remote-sensing-derived attributes can suggest a level of precision that does not actually exist at the individual-tree scale. In particular:

This is why the enrichment module applies QA tracking (deviation flags, source labels) and preserves the original inventory values – so that downstream analysis can distinguish between field-measured and remote-sensing-derived data.

Columns carried over

After enrichment, these columns are available on ctx.vector["trees"]:

Target column OBSkH source Fill rate Description
obskh_id id 100% OBSkH feature ID
obskh_hoehe hoehe 100% Tree height in meters
obskh_krone krone 100% Crown diameter in meters (original OBSkH value)
obskh_stamm stamm 3.1% Trunk diameter in meters
obskh_bsatzung bsatzung 100% Tree protection statute (bool)
obskh_lageklasse lageklasse 100% Provider-defined location class 1–4; currently not used in scoring
obskh_area AREA 59.6% Crown projection area in m²
obskh_volume VOLUME 59.6% Crown volume in m³
obskh_meanNDVI meanNDVI 59.6% Mean NDVI over crown area
obskh_meanGRI meanGRI 59.6% Mean Green-Red Index
obskh_distToBd distToBd 59.6% Distance to nearest building in meters
obskh_heightDiff heightDiff 59.6% Height difference (nDSM-derived)
obskh_pointcount PointCount 59.6% Number of LiDAR points in crown
obskh_baumid_67 baumid_67 0.1% Legacy tree ID
obskh_gfmsid_67 gfmsid_67 18.8% GFMS ID

Note: Several provider-defined columns are retained for traceability and QA, but they are not required by the current scoring pipeline.

QA / join metadata columns

Column Type Description
obskh_match_found bool Whether an OBSkH feature was found within tolerance
obskh_spatial_distance_m float Distance to matched OBSkH feature in meters
kronendurchmesser_source string Origin of the final kr_durchm value (see below)
kronendurchmesser_diff_percent float Deviation between inventory and OBSkH crown diameter (%)
kr_durchm_inventory_original float Backup of the original inventory crown diameter

Citree Species Profile Mapping

Source

Goal

Citree is not joined spatially. Instead, the enrichment matches each inventory value from ga_lang to one Citree profile and attaches only the requested profile attributes as citree_* columns in the pipeline result.

Independent of the Citree match, the pipeline also derives reusable taxonomy columns directly from ga_lang so they are available in the general output, in visual filters, and in downstream ML feature tables.

Derived column Example value Purpose
species_genus Quercus Broad grouping/filtering by genus
species_binomial Quercus robur Botanical species grouping without cultivar overfitting
species_cultivar Fastigiata Separate cultivar/sorten field for filters and optional ML usage
species_common_name Säuleneiche Human-readable display/filter label from the German name part
species_is_placeholder true Flags labels such as spec., Art nicht bekannt, unbekannt

The difficult part is that ga_lang is not isomorphic to the Citree schema:

Matching workflow

For each unique ga_lang value, the enrichment performs the following steps:

  1. Split ga_lang at the first comma or semicolon into a botanical part and a common-name part.
  2. Normalize quotes, umlauts, separators, and hybrid markers (× → x) to a comparison-safe ASCII form.
  3. Extract cultivar candidates from quoted substrings, including malformed single-quote edge cases.
  4. Build a botanical core key from the left side using only genus + species or genus + x + hybrid epithet. Author abbreviations and rank suffixes are ignored at this stage.
  5. Normalize the common-name side as a fallback signal after removing generic stop words such as Sorte, Baum, or nicht bekannt.
  6. Score all Citree candidates against the normalized signals:
    • botanical_core: exact match on genus/species core key
    • cultivar: exact cultivar match from quotes vs. Citree sorte
    • allgemein: prefer sorte = Allgemein when the inventory has no cultivar
    • general_fallback: allow fallback to Allgemein when the inventory has a cultivar but Citree has no cultivar-specific row
    • common_exact / common_partial: use the German name only as a secondary tiebreaker, never as the sole high-confidence signal
    • cultivar_without_species: allow genus + cultivar matches when the inventory says only Ulmus-Hybride 'New Horizon'
  7. Reject placeholders and ambiguities. Genus-only placeholders such as Aesculus spe. [Art und Sorte nicht bekannt] remain unmatched. Tied top scores are marked as ambiguous instead of guessing.

Explicit botanical alias rules

The runtime matcher also applies a small set of hard-coded botanical aliases where the inventory and Citree use different but fachlich equivalent names:

Inventory form Citree form Reason
Platanus x acerifolia Platanus x hispanica Equivalent naming for the common hybrid plane tree
Ulmus hollandica Ulmus x hollandica Hybrid marker is omitted in the inventory but present in Citree
Liquidambar styraciflua Liquidambar styraciflura Citree contains the same taxon under a spelling variant
Quercus petraea Quercus robur subsp. petraea / ... sessiliflora ... Citree uses an older taxonomic framing for Trauben-Eiche

These alias rules are intentionally narrow and documented here so later updates remain auditable. They should only be extended when a mismatch is verified against the real inventory and Citree export.

Intentionally unmatched groups

Some high-frequency inventory labels currently remain empty on purpose because Citree does not expose a single safe target profile for them:

Inventory group Why it remains unmatched
Populus nigra, Populus nigra 'Italica', Populus canadensis No direct Citree profiles for these taxa; mapping to other poplars would be speculative
Tilia spec. Citree has only species-specific lime profiles, no generic Tilia spec. target
Ulmus carpinifolia Historical synonym of Ulmus minor, but Citree currently has no direct Feld-Ulme profile

Malus domestica is also treated conservatively in interpretation: if future Citree updates still expose only ornamental crabapple profiles, Kulturapfel should not be auto-mapped to them without an explicit fachliche decision.

Practical mapping examples

ga_lang in inventory Citree target Match rule
Acer campestre, Feldahorn Feld-Ahorn: Allgemein botanical_core+allgemein
Acer campestre 'Elsrijk', Feldahorn 'Elsrijk' Feld-Ahorn: Elsrijk botanical_core+cultivar
Fraxinus pennsylvanica, Esche 'Pennsylvanica' Esche: Pennsylvanica botanical_core+cultivar
Carpinus betulus spec., Hainbuche, Sorte nicht bekannt general profile for Carpinus betulus botanical_core+allgemein
Ulmus-Hybride 'New Horizon'; Ulme 'New Horizon' Ulme: New Horizon genus+common_exact+cultivar+cultivar_without_species
Aesculus spe. [Art und Sorte nicht bekannt] no match no_match

Output columns

The pipeline output receives these metadata columns for traceability:

Column Meaning
citree_art_id Matched Citree profile ID
citree_name Matched Citree profile label
citree_match_rule Concatenated rule tokens that explain why the match was chosen
citree_match_score Internal candidate score used to separate strong from weak matches

The requested profile attributes are attached with an explicit Citree prefix:

Citree source column Output column
Lichtanspruch citree_lichtanspruch
Bodenverdichtungstoleranz citree_bodenverdichtungstoleranz
Staunässetoleranz citree_staunaessetoleranz
Salzverträglichkeit citree_salzvertraeglichkeit
Gründigkeit citree_grundigkeit
Krankheiten / Schädlinge citree_krankheiten_schaedlinge
Bodeneigenschaft citree_bodeneigenschaft
Bodenfeuchtetoleranz citree_bodenfeuchtetoleranz
Wuchsgeschwindigkeit citree_wuchsgeschwindigkeit
Kronendurchlässigkeit citree_kronendurchlaessigkeit
Kronenform citree_kronenform
oberflächennahe Wurzeln citree_oberflaechennahe_wurzeln
Trockenheitstoleranz citree_trockenheitstoleranz
Hitzeverträglichkeit citree_hitzevertraeglichkeit

Operational notes

Crown Diameter Merge Logic

The merge follows priority rules defined in Issue #80:

Case Inventory kr_durchm OBSkH obskh_krone Result in kr_durchm kronendurchmesser_source
1 missing 7.0 7.0 "obskh"
2 10.0 10.5 (<10% dev.) 10.25 (average) "validated"
3 10.0 12.0 (10–30% dev.) 10.0 (inventory) "discrepancy_high"
4 5.0 12.0 (>30% dev.) 5.0 (inventory) "red_flag"
5 8.0 missing 8.0 (unchanged) "inventory"
6 missing missing NaN "missing"

After the merge, a plausibility filter caps unrealistic values:

Case kr_durchm after merge Limit (default 35 m) Result kronendurchmesser_source
7 187.0 35.0 NaN "clamped"
7 58.0 35.0 NaN "clamped"
– 12.0 35.0 12.0 (unchanged) (original source)

Rationale

Spatial Join Details

Method

Performance characteristics

Impact on Criteria

Criterion Impact
NDVI Uses enriched kr_durchm for zonal statistics (mean NDVI over crown buffer instead of point sampling)
Age No direct impact (uses pflanzjahr)
Vitality No direct impact (uses vitalitaet)
All others No direct impact; enriched columns remain available for QA and supplementary analysis

Troubleshooting

Enrichment not running

Low match rate

“Cannot insert objectid, already exists” error

This occurs when the tree inventory has objectid as both index and column. The enrichment module handles this automatically by resetting the index before the join.

Many red flags

A high number of red_flag entries (>30% crown diameter deviation) may indicate:

Changelog

Date Change
2026-02-25 Initial implementation: spatial join, crown diameter merge, QA tracking (#80)
2026-02-25 Issue #123 created for column clarifications with Stadt Hannover