bluegreencity-ki

Climate Data Import

This document describes the process of importing climate data into the PostGIS database for the BlueGreenCity-KI project.

Overview

Climate data is imported from two main sources:

1. DWD (Deutscher Wetterdienst): German Weather Service providing HYRAS, HOSTRADA, and Evaporation datasets
2. Open-Meteo: Historical weather API for additional climate variables

Both sources provide raster data that is imported into PostGIS tables for efficient spatial queries.

Import Scripts

DWD Import (import_climate_data_dwd.py)

Purpose: Import NetCDF climate rasters from DWD into PostGIS

Location: scripts/import_climate_data_dwd.py

Usage:

python scripts/import_climate_data_dwd.py --dataset <dataset> --variable <variable> --year <year> [--hyras-version <version>]

Parameters:

• --dataset: Dataset type (hyras, hostrada, or evaporation)
• --variable: Climate variable (e.g., precipitation, temperature_mean)
• --year: Year to import
• --hyras-version: HYRAS version (6-0 or 6-1, default: 6-1)
• --list-variables: List all available variables for a dataset

Examples:

# Import HYRAS precipitation for 2023 (version 6-1)
python scripts/import_climate_data_dwd.py --dataset hyras --variable precipitation --year 2023

# Import HYRAS temperature with version 6-0
python scripts/import_climate_data_dwd.py --dataset hyras --variable temperature_mean --year 2023 --hyras-version 6-0

# Import HOSTRADA wind speed
python scripts/import_climate_data_dwd.py --dataset hostrada --variable wind_speed --year 2024

# List available HYRAS variables
python scripts/import_climate_data_dwd.py --dataset hyras --list-variables

Open-Meteo Import (import_climate_data_openmeteo.py)

Purpose: Download and import historical weather data from Open-Meteo API

Location: scripts/import_climate_data_openmeteo.py

Usage:

python scripts/import_climate_data_openmeteo.py --variables <v1,v2> --start-date <YYYY-MM-DD> --end-date <YYYY-MM-DD> [--lat <lat>] [--lon <lon>] [--resolution hourly|daily] [--dataset-name <name>] [--dry-run]

Parameters:

• --variables: Comma-separated variables (e.g., temperature_2m,precipitation)
• --start-date: Start date (YYYY-MM-DD)
• --end-date: End date (YYYY-MM-DD)
• --lat / --lon: Latitude/longitude in decimal degrees (default: study area center from config/config.yaml)
• --resolution: hourly (default) or daily
• --dataset-name: Dataset label for the target tables (default: weather)
• --dry-run: Fetch from the API but do not write to PostGIS
• --list-variables: List all available variables

Examples:

# Download hourly temperature + precipitation for 2023 to the default location
python scripts/import_climate_data_openmeteo.py --variables temperature_2m,precipitation --start-date 2023-01-01 --end-date 2023-12-31

# Daily resolution, custom location
python scripts/import_climate_data_openmeteo.py --variables temperature_2m --start-date 2024-01-01 --end-date 2024-12-31 --resolution daily --lat 52.5 --lon 9.8

# Check the API response without importing
python scripts/import_climate_data_openmeteo.py --variables precipitation --start-date 2025-06-01 --end-date 2025-06-30 --dry-run

# List available variables
python scripts/import_climate_data_openmeteo.py --list-variables

Database Schema

Imported climate data is stored in the bgcki schema with the following naming convention:

DWD Tables

• Format: bgcki.climate_dwd_{dataset}_{version}_{variable}_{resolution}
• Examples:
  • bgcki.climate_dwd_hyras_v6-1_precipitation_daily
  • bgcki.climate_dwd_hyras_v6-0_temperature_mean_daily
  • bgcki.climate_dwd_hostrada_temperature_hourly
  • bgcki.climate_dwd_evaporation_evaporation_fao_daily

Open-Meteo Tables

• Format: bgcki.climate_openmeteo_{dataset-name}_{hourly|daily} (dataset name defaults to weather)
• Examples:
  • bgcki.climate_openmeteo_weather_hourly
  • bgcki.climate_openmeteo_weather_daily

Table Metadata

Each imported table includes a PostgreSQL comment with metadata:

• Variable name
• Version
• Resolution (daily/hourly)
• Units
• Description
• Import timestamp

Example:

COMMENT ON TABLE climate.dwd_hyras_v6_1_precipitation_daily IS 
'Climate data from DWD
Variable: precipitation
Version: v6_1
Resolution: daily
Units: mm
Description: HYRAS precipitation data version 6-1
Imported: 2025-12-30 14:30:00';

Data Sources

DWD (Deutscher Wetterdienst)

Base URL: https://opendata.dwd.de/climate_environment/CDC/grids_germany/

Datasets:

1. HYRAS (v6-0, v6-1):
   • Precipitation, temperature (mean, min, max), humidity, radiation
   • Daily resolution
   • 1 km grid
2. HOSTRADA (v1-0):
   • Wind speed, pressure, cloud cover
   • Daily resolution
   • Variable grid
3. Evaporation (v1-0, v1-1):
   • Potential and actual evapotranspiration
   • Daily resolution
   • 1 km grid

Open-Meteo

API URL: https://archive-api.open-meteo.com/v1/archive

Variables:

• temperature_2m: 2m air temperature (°C)
• precipitation: Total precipitation (mm)
• relative_humidity_2m: Relative humidity (%)
• surface_pressure: Surface pressure (hPa)
• And many more (use --list-variables to see all)

Maintenance Schedule

Climate data should be updated annually or as new versions become available:

1. Annual Updates: Import new year’s data each January
2. Version Updates: When DWD releases new dataset versions (e.g., HYRAS v6-2)
3. Historical Backfill: Import historical data for validation and trend analysis

Usage in Pipeline

Climate data is accessed in criteria modules using the AirTemperature criterion as a reference:

from bluegreencity_ki.criteria.air_temperature import AirTemperature

# In pipeline configuration (config.yaml):
criteria:
  - name: air_temperature
    weight: 0.2
    params:
      raster_key: "airtemp_raster_14h"
      temp_low: 20.0
      temp_high: 30.0

The raster key refers to a PostGIS raster table defined in the data sources configuration.

Troubleshooting

Import Errors

Error: NetCDF file not found

• Solution: Check if the file exists in data/derived/weather/ or download from DWD

Error: PostGIS connection failed

• Solution: Verify database credentials in .env file and ensure PostGIS is running

Error: Version not detected for Evaporation

• Solution: Ensure filename follows DWD convention (e.g., grids_germany_daily_evaporation_fao_2024_v1.1.netcdf)

Performance Issues

If imports are slow:

1. Ensure PostGIS has adequate resources (memory, CPU)
2. Create spatial indexes after import
3. Use chunked imports for very large datasets

See Also

• DWD Open Data Portal
• Open-Meteo API Documentation
• PostGIS Raster Documentation
• Scripts: scripts/ (repo root; not part of the docs site)