Skip to content

Conversation

@jatkinson1000
Copy link
Member

Closes #126

This adds a tutorial for new users to the docs, with associated scripts in the GitHub repo.

It works through

  • Installing TCTrack with TE and TSTORMS
  • Fetching data
  • Preprocessing data
  • Running TCTrack (TE and TSTORMS)
  • Visualising tracks

Potential issues:

  • TSTORMS does not detect any tracks for the input data. Consider using a different period where it does?
  • Data pre-processing can take a long time and requires conda env which feels clunky
  • We use a mixture of sources for input data when ideally we would use a single source
  • We use wget to fetch data, but I would prefer if we used OpenDAP or THREDDS. Perhaps improve later. Open an issue?

On CSD3 I use the following environment:

module purge
module load rhel8/cclake/base
module load intel-oneapi-compilers/2023.2.4/gcc/4lbvg4hv
module load netcdf-cxx4/4.3.1/intel/intel-oneapi-mpi/gxfwrfnl
module load netcdf-fortran/4.6.1/intel/intel-oneapi-mpi/kqukipdf
module load python/3.11.9/gcc/nptrdpl

To run the regridding script I submitted as a batch job (approx 30mins to run) using the following:

#!/bin/bash
#SBATCH -J regrid
#SBATCH -A ICCS-SL2-CPU
#SBATCH -p icelake
#SBATCH --output=regrid_%A_%a.out
#SBATCH --error=regrid_%A_%a.err
#SBATCH --nodes=1
#SBATCH --ntasks=2
#SBATCH --time=01:00:00

. /etc/profile.d/modules.sh

module purge
module load rhel8/cclake/base

cd /home/jwa34/rds/rds-inspire-tc-TqEGHMWTn8A/TCTrack/tutorial

source miniforge3/bin/activate
conda activate regrid

python regrid.py

conda deactivate
conda deactivate

which assumes you have already cloned TCTrack and placed your miniforge installation in the tutorial/ directory.

It would be good if you could work through this when back @sjavis to see how well it works or any sticking points/points for clarification/improvement.

@jatkinson1000 jatkinson1000 self-assigned this Dec 24, 2025
Copy link
Collaborator

@sjavis sjavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to mostly work for me locally, with the exception of TSTORMS since I don't have ifort. I have just a few suggested changes.

I think the conda environment is fine, especially since we discussed that this will probably be what we recommend. The pre-processing time also didn't seem too dreadful to me. I think, ultimately, is going to take time for the pre-processing for any reasonable.

I agree that the multiple sources and using wget can probably also be made an issue to look at improving later.

git clone git@github.com:Cambridge-ICCS/TCTrack.git
cd TCTrack
pip install .

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instructions for initialising the conda environment for cf-python are given in the pre-processing section. Should those be moved here? Otherwise we will also need to add the pip install instruction in the pre-processing section

# This script will clone, build, and install TSTORMS.

# It is assumed that the following dependencies are installed:
# - Fortran compiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# - Fortran compiler
# - Intel Fortran compiler (ifort)

This data can be obtained from the `ESGF CEDA archive <https://esgf.ceda.ac.uk/thredds/catalog/catalog.html>`_
or direct from `CEDA <https://data.ceda.ac.uk/>`_ using the included fetch data script::

./fetch_data.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetch_data.sh is not executable


# zg has to be fetched direct from CEDA as not present on ESGF
wget https://dap.ceda.ac.uk/badc/cmip6/data/PRIMAVERA/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Prim3hrPt/zg7h/gn/files/d20180730/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195008010000-195008302100.nc?download=1 -O data/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195008010000-195008302100.nc
wget https://dap.ceda.ac.uk/badc/cmip6/data/PRIMAVERA/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Prim3hrPt/zg7h/gn/files/d20180730/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195009010000-195009302100.nc?download=1 -O data/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195010090000-195009302100.nc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
wget https://dap.ceda.ac.uk/badc/cmip6/data/PRIMAVERA/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Prim3hrPt/zg7h/gn/files/d20180730/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195009010000-195009302100.nc?download=1 -O data/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195010090000-195009302100.nc
wget https://dap.ceda.ac.uk/badc/cmip6/data/PRIMAVERA/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Prim3hrPt/zg7h/gn/files/d20180730/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195009010000-195009302100.nc?download=1 -O data/zg7h_Prim3hrPt_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn_195009010000-195009302100.nc

Comment on lines +42 to +43
plt.figure(figsize=(10, 3))
ax = plt.axes(projection=ccrs.PlateCarree())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
plt.figure(figsize=(10, 3))
ax = plt.axes(projection=ccrs.PlateCarree())
plt.figure(figsize=(8, 4))
ax = plt.axes(projection=ccrs.PlateCarree())
ax.set_extent([-180, 180, -90, 90])

The current plot is hard to see since the latitude range is so small and the title and legend are larger than the axes.

Comment on lines +58 to +60
pl = ax.plot(
lons[i], lats[i], "--", transform=ccrs.PlateCarree(), label=f"{label}"
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pl = ax.plot(
lons[i], lats[i], "--", transform=ccrs.PlateCarree(), label=f"{label}"
)
mask = np.isfinite(lons[i])
pl = ax.plot(
lons[i, mask], lats[i, mask], "--", transform=ccrs.PlateCarree(), label=f"{label}"
)

This avoids the warning appearing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add tutorial for new users

3 participants