Release v1.1.0: Pyproject migration, Pixi support, and CMIP7 CVs #231
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add cmip7 extra to lint_and_format job installation - Add cmip7 extra to test job installation - Ensures CMIP7 interface is tested in CI pipeline
- Exclude prototype/ directory from test collection (experimental code with missing deps) - Skip test_cmip7_from_vendored_json (vendored JSON has limited data) - Remove test_version_compatibility.py (tests removed wrapper) - All CMIP7 interface tests pass (44 tests in test_cmip7_interface.py)
- Add CMIP7-CVs as git submodule (src-data branch) - Implement CMIP7ControlledVocabularies class with support for: - Loading from vendored submodule - Directory-based CV structure (experiment/, project/) - JSON-LD format handling - Add comprehensive test suite (18 tests) - Add documentation in Sphinx format
- Fix import order (isort) - Apply black formatting (whitespace, line breaks) - Remove unused 'ref' variable in load_from_git method - All linting checks now pass
- Add comprehensive CMIP7 global attributes implementation - Fix get_frequency() to fall back to rule_dict when drv is None - Update get_Conventions() to include CMIP-7.0 - Convert variant label indices to strings for netCDF compliance - Add 11 unit tests for CMIP7 global attributes (all passing) - Update validation schema for CMIP7 YAML configurations - Add CMIP7_DReq_metadata field to general schema - Make CMIP_Tables_Dir optional for CMIP7 - Add compound_name field for CMIP7 compound names - Make cmor_variable optional when compound_name provided - Add CMIP7-specific fields: realm, frequency, table_id, grid, nominal_resolution - Fix institution_id typo (was instition_id) - Add 10 integration tests for YAML validation (all passing) - Add comprehensive CMIP7 documentation - New doc/cmip7_configuration.rst: Complete YAML configuration guide - Update doc/quickstart.rst: Add note directing CMIP7 users to new guide - Update doc/index.rst: Add cmip7_configuration to TOC - Update README.rst: Mention CMIP6 and CMIP7 support - New examples/cmip7-example.yaml: Working configuration examples Key features: - CMIP7 compound names support (e.g., atmos.tas.tavg-h2m-hxy-u.mon.GLB) - Backward compatibility with CMIP6 maintained - All 21 tests passing (11 unit + 10 integration) - User-focused documentation with migration guide
Resolved conflicts in src/pycmor/core/validate.py by keeping both: - CMIP7 fields (frequency, table_id, grid, nominal_resolution) - Time coordinate fields from main (time_units, time_calendar) All tests passing after merge.
Migrates from setup.py/setup.cfg to modern pyproject.toml configuration. Key changes: - Removes setup.py, setup.cfg, pytest.ini (moved to pyproject.toml) - Consolidates all project metadata and dependencies in pyproject.toml - Maintains versioneer for version management - Preserves all entry points and CLI subcommands - Updates CI configuration to exclude CMIP7 DReq submodule from linting Closes #211 Closes #205 Co-Authored-By: Paul Gierz <pgierz@awi.de>
Adds pixi package manager support for better cross-platform dependency management. Key changes: - Adds pixi.lock for reproducible environments - Configures pixi in pyproject.toml with conda-forge channel - Supports Python 3.10-3.12 on osx-arm64, osx-64, linux-64 - Installs pycmor as editable PyPI dependency - Includes dev environment with pytest This enables developers to use `pixi install` and `pixi shell` for environment setup. Co-Authored-By: Paul Gierz <pgierz@awi.de>
Adds comprehensive CMIP7 Controlled Vocabularies support. Key changes: - Adds CMIP7-CVs as git submodule (WCRP-CMIP/CMIP7-CVs, src-data branch) - Enhances ControlledVocabularies class to support both CMIP6 and CMIP7 - Adds comprehensive unit tests for CV functionality - Updates yamllint config to exclude CMIP7-CVs directory - Adds documentation for CMIP7 CV implementation This enables pycmor to validate and process data for CMIP7 experiments. Co-Authored-By: PavanSiligam <pavankumar.siligam@awi.de>
The CMIP7-CVs submodule contains Python files that don't meet our linting standards. Add it to the exclude list for flake8, isort, and black.
versioneer.py is generated code and should not be linted.
Move flake8, black, and isort configuration from CI workflow to pyproject.toml for better maintainability. Set line length to 120 characters and configure all tool exclusions in one place.
Removed duplicate [tool.flake8], [tool.black], and [tool.isort] sections that were causing TOML parse errors. Added CMIP7-CVs to exclusion lists in the existing configuration sections to prevent linting the submodule.
- flake8 doesn't natively read pyproject.toml, so added .flake8 config file - Ran isort and black to fix all formatting issues - All linting checks now pass locally
Linting tools don't need the package installed to check code style.
The CMIP7-CVs submodule contains Python files that don't meet our linting standards. Add it to the exclude list for flake8, isort, and black.
- Build Docker images once per Python version and cache based on Dockerfile/pyproject.toml changes - Share built images across all test jobs (meta, unit, integration, doctest) - Use persistent cache in $HOME/.cache/pycmor for test data instead of ephemeral /tmp - Mount cache directory into Docker containers to avoid re-downloading test data per job - Reduces CI build time and bandwidth usage for static test data
- Updated import statement from pymor to pycmor - Renamed all test classes and methods from Pymor to Pycmor - Updated all accessor references from .pymor to .pycmor - Updated docstrings and comments to reflect new package name
dataset accessors
Code by @mzapponi, minor cleanup to better combine the error messages and properly handle the fallback.
Remerge of 219
- Apply black formatting to coordinate_attributes.py and dimension_mapping.py - All linting checks (flake8, isort, black) now pass
Co-authored-by: Paul Gierz <pgierz@awi.de>
Co-authored-by: Paul Gierz <pgierz@awi.de>
Co-authored-by: Paul Gierz <pgierz@awi.de>
Co-authored-by: Paul Gierz <pgierz@awi.de>
Co-authored-by: Paul Gierz <pgierz@awi.de>
Co-authored-by: Paul Gierz <pgierz@awi.de>
…cumentation examples
CF-Compliant Coordinate Metadata with Validation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR consolidates several major improvements for the pycmor 1.1.0 release:
Major Changes
✅ PR #212 - Pyproject Migration
setup.py/setup.cfgto modernpyproject.tomlconfiguration✅ PR #224 - Pixi Support
pixi.lockfor reproducible environments✅ PR #222 - CMIP7 Controlled Vocabularies Implementation
Already Incorporated
The following PRs were already merged into prep-release:
Breaking Changes
Testing
Checklist
pixiconda environment