Add support to model AA haplotypes

[huddlej]

Description of proposed changes

The main goal of this PR is to automatically fit the MLR to more granular amino acid haplotypes based on current clade annotation and all HA1 substitutions from that parent clade. This finer granularity will allow us to detect new haplotypes that we should be tracking.

To accomplish this goal, I've reorganized the workflow to support fitting the MLR with different variant classifications as in the forecasts-ncov workflow.

Changes include:

• Config YAML and workflow allows definition of one or more variant classifications (including the current emerging_haplotype and new aa_haplotype)
• Config YAML and workflow allows definition of one or more data provenances (gisaid)
• Local MLR JSONs live in new directory structure: results/{data_provenance}/{variant_classification}/{lineage}/{geo_resolution}/mlr/MLR_results.json
• Remote MLR JSONs live in a new structure:
  • Trial path for this PR: s3://nextstrain-data/files/workflows/forecasts-flu/trial/forecast-aa-haplotypes/gisaid/emerging_haplotype/h3n2/region/mlr/MLR_results.json
  • Production path: s3://nextstrain-data/files/workflows/forecasts-flu/gisaid/emerging_haplotype/h3n2/region/mlr/MLR_results.json
• Forecasts viz displays frequencies and GAs for both emerging haplotypes and amino acid haplotypes for the selected subtype and geographic resolution
• Reduced the minimum number of sequences required per geographic location from 150 to 100, allowing us to catch new haplotypes in locations with sparser sequencing
• Moved min_date and max_date to top-level config, since we want the same time periods for all models in a given run

Testing locally

To test the new visualization interface locally, run the following commands from inside this repo's directory and this branch:

cd viz

# Install dependencies.
npm ci

# Run local viz server.
npm run start

Open http://127.0.0.1:8000/

The following screenshot shows H1N1pdm regional results with log transform and raw data turned on for both variant classifications:

Outstanding issues

• The current workflow assumes that we can use the same pivot for both emerging haplotype and AA haplotypes. This assumption is reasonable, as long as the pivot is a high-frequency variant that is likely to appear in both analyses. It is possible that the AA haplotype analysis could partition a high-frequency variant into more granular variants such that the ancestral variant no longer has enough counts of its own to appear in the analysis. If this happens, we would still need to manually update the pivot or just accept the default fallback behavior of setting the pivot to the last variant.
• Forecasts viz hardcodes the data provenance and does not provide an interface to select an alternate provenance.

Checklist

• Update changelog
• Drop trial S3 path commit (159027e)
• Deploy MLR JSONs to new S3 paths with run model action on this branch
• Deploy the new viz site by merging this PR

[jameshadfield]

The viz part is working well. A nice way to finish 2025, hopefully we can make 2026 the year we move this forward

In terms of the modelling, I was surprised at the ~1y forecast window of given (only) ~6m of fitted data. The CIs also look a little odd, e.g. H3N2 / Spain / K:88I is unexpectedly jagged, and many CIs rise before then abruptly ending (maybe because the mean has reached zero?)

[joverlee521]

Can't comment on the actual results, but the workflow + viz changes look reasonable to me! Left a question about whether we should centralize where emerging/aa haplotypes get derived.

[joverlee521]

This script looked familiar to me and I realized it's almost exactly the same as add_derived_haplotypes.py in seasonal-flu. Should we be running this (and assign_haplotypes.py) as part of run-nextclade.smk in seasonal-flu so that they can just be part of the Nextclade TSV that is downloaded here?

[huddlej]

Good point, @joverlee521. That's probably the right place to end up. For now, I like having the flexibility to reannotate the original Nextclade file in each workflow, but I made an issue in the seasonal-flu repo for this proposed change.

[huddlej]

Thanks, @jameshadfield!

In terms of the modelling, I was surprised at the ~1y forecast window of given (only) ~6m of fitted data.

The 1-year horizon is there to match our original eLife model's horizon. We will most likely change this in the near future (ha!), since there is so much uncertainty that it's often not informative to have that long of a horizon.

The CIs also look a little odd, e.g. H3N2 / Spain / K:88I is unexpectedly jagged, and many CIs rise before then abruptly ending (maybe because the mean has reached zero?)

I'll look into this more, but I think these are separate issues. I only notice the last one you mention when the HPDIs converge to the same value as the median as a variant is predicted to fix. There is some jaggedness in the log-transform that you don't see as much in the standard view which probably reflects some numerical rounding issues.