Skip to content

Conversation

@xylar
Copy link
Collaborator

@xylar xylar commented Aug 18, 2025

MPAS meshes include an empty, unlimited Time dimension but the latest NetCDF4 drops this dimension since no variable use it. The create_from_generic_mpas_grid() funciton in landice.create was taking advantage of Time being a dimension in the MPAS mesh but this should no longer be assumed. This PR fixes that assumption by explicitly adding Time to the list of dimensions to create if it is not already in the input dataset.

xylar added 2 commits August 18, 2025 04:28
MPAS meshes include an empty, unlimited `Time` dimension but the
latest NetCDF4 drops this dimension since no variable use it.
the `create_from_generic_mpas_grid()` funciton in `landice.create`
was taking advantage of `Time` being a dimension in the MPAS mesh
but this should no longer be assumed.  This merge fixes that
assumption by explicitly adding `Time` to the list of dimensions
to create if it is not already in the input dataset.
@xylar xylar self-assigned this Aug 18, 2025
@xylar xylar added the bug label Aug 18, 2025
@xylar
Copy link
Collaborator Author

xylar commented Aug 18, 2025

@matthewhoffman and @trhille, a new version of JIGSAW was released last week:
dengwirda/jigsaw-python#29

In the process of testing it, I learned that new newest version of xarray breaks several of our workflows. I reported this to them:
pydata/xarray#10647
it has been fixed in:
pydata/xarray#10648
and I added a workaround for us in:
#662
and, after performing Polaris, MPAS-Ocean and MPAS-Analysis testing, I made a new release of MPAS-Tools with the fix:
https://github.com/MPAS-Dev/MPAS-Tools/releases/tag/1.3.0

I didn't do any MALI testing that that has proven to be a dumb move. MALI breaks for a different reason. There has also been an update to NetCDF4:
https://github.com/Unidata/netcdf4-python/releases/tag/v1.7.2rel
I'm not sure if the change was in this release or a previous one but I suspect the most recent release. In any case, it seems that Time is now being dropped when NetCDF4 reads in an MPAS mesh dataset as described above (because it is not used in any variables). As a result, we need to add it explicitly or we get errors like:

Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.xylar/chrysalis/miniforge3/envs/dev_compass_1.8.0-alpha.2/bin/create_landice_grid_from_generic_mpas_grid", line 9, in <module>
    sys.exit(create_from_generic_mpas_grid())
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/gpfs/fs1/home/ac.xylar/chrysalis/miniforge3/envs/dev_compass_1.8.0-alpha.2/lib/python3.13/site-packages/mpas_tools/landice/create.py", line 284, in create_from_generic_mpas_grid
    newvar = fileout.createVariable('thickness', datatype, ('Time', 'nCells'))
  File "src/netCDF4/_netCDF4.pyx", line 3005, in genexpr
  File "src/netCDF4/_netCDF4.pyx", line 3005, in genexpr
  File "/gpfs/fs1/home/ac.xylar/chrysalis/miniforge3/envs/dev_compass_1.8.0-alpha.2/lib/python3.13/site-packages/netCDF4/utils.py", line 45, in _find_dim
    raise ValueError("cannot find dimension %s in this group or parent groups" % dimname)
ValueError: cannot find dimension Time in this group or parent groups

      Failed
Exception raised while running the steps of the test case
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.xylar/compass/update-to-mpas-tools-1.3.0/compass/run/serial.py", line 322, in _log_and_run_test
    _run_test(test_case, available_resources)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.xylar/compass/update-to-mpas-tools-1.3.0/compass/run/serial.py", line 419, in _run_test
    _run_step(test_case, step, test_case.new_step_log_file,
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              available_resources)
              ^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.xylar/compass/update-to-mpas-tools-1.3.0/compass/run/serial.py", line 470, in _run_step
    step.run()
    ~~~~~~~~^^
  File "/gpfs/fs1/home/ac.xylar/compass/update-to-mpas-tools-1.3.0/compass/landice/tests/dome/setup_mesh.py", line 78, in run
    check_call(args, logger)
    ~~~~~~~~~~^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.xylar/chrysalis/miniforge3/envs/dev_compass_1.8.0-alpha.2/lib/python3.13/site-packages/mpas_tools/logging.py", line 66, in check_call
    raise subprocess.CalledProcessError(process.returncode, print_args)

Note: MPAS-Tools lints and formats any files that we change using ruff-format (a standard linting tool also used in Polaris and across the E3SM ecosystem of python packages). I have updated landice.create to conform to this standard in a separate commit. Please ignore that commit in your review unless you are interested in the linting.

Copy link
Member

@matthewhoffman matthewhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xylar , thank you for taking care of this. I imagine it was a lot of work to untangle the new issues with xarray and netcdf4. I reviewed the first commit by inspection and it makes sense to me.

@xylar xylar merged commit 5d55cb9 into MPAS-Dev:master Aug 18, 2025
6 checks passed
@xylar xylar deleted the fix-landice-create branch August 18, 2025 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants