Schema validation for pathogen.json does not reject unknown fields or catch typos

[nneune]

The JSON schema (v3.0.0) used to validate pathogen.json files for Nextclade datasets is too permissive: adding unexpected properties (extra fields) or minor typos in key names does not trigger validation errors. As a result, malformed pathogen.json files can be accepted silently, leading to downstream misbehavior or confusing data issues. For example, the EV-D68 dataset included a misspelling in the penalityGapOpen field that was not detected.

I attached a screen recording demonstrating the problem. Autofill suggestions work, and some fields (e.g., gapAlignmentSide) are validated for format, but extra keys or misspelled properties are neither highlighted nor cause the file to fail validation.

Schema_validation_test.mp4

[ivan-aksamentov]

This is intentional and the default behavior of serde and schemars packages.

Unknown fields are allowed to ensure forward compatibility as new features (and new keys) are introduced. Tightening it would break all old Nextclade deployments every time a new field is added in the format.

Currently, schemas are produced as is from rust structs - this covers the basics. But perhaps they could be further improved.

I don't quite have a solution for this problem immediately though. Let's keep this issue open and discuss how this can be organized better.

[ivan-aksamentov]

Possible local workaround:

1. Create a wrapper schema with "unevaluatedProperties": false (docs):

   .vscode/strict-pathogen.schema.json:

   {
     "$schema": "https://json-schema.org/draft/2020-12/schema",
     "allOf": [
       { "$ref": "https://raw.githubusercontent.com/nextstrain/nextclade/refs/heads/release/packages/nextclade-schemas/input-pathogen-json.schema.json" }
     ],
     "unevaluatedProperties": false
   }

2. Tell vscode which schema file to use for which files (docs):

   .vscode/settings.json:

   {
     "json.schemas": [
       {
         "fileMatch": [
           "data/**/pathogen.json"
         ],
         "url": ".vscode/strict-pathogen.schema.json"
       }
     ]
   }

3. Optionally also validate with ajv CLI:

   npx ajv-cli validate \
     -s .vscode/strict-pathogen.schema.json \
     -d path/to/pathogen.json \
     --strict=true \
     --all-errors

   or jsonschema python CLI:

   pip install jsonschema
   jsonschema -i path/to/pathogen.json .vscode/strict-pathogen.schema.json
   

4. Not sure if "unevaluatedProperties": false is supported by vscode fully. If not, .vscode/strict-pathogen.schema.json needs to be a full copy of the original schema with "additionalProperties": false (docs) added in each object's definition recursively.

[ivan-aksamentov]

If we'd go for strict checking everywhere, each rust struct needs to be annotated with #[serde(deny_unknown_fields)]. This will reject unknown fields when Nextclade runs. This will also result in "additionalProperties": false automatically added to schemas by schemars.

[ivan-aksamentov]

Quick example of what I mean by forward compatibility:

If this is added: #1700, say in version 3.x.y, and we have strict checks, this dataset nextstrain/nextclade_data#367 will be rejected by all versions below 3.x.y - they will report an unknown field error.

This is a design choice to make. Current choice is permissive schemas.

[nneune]

Ahh I see, thanks for looking into this!
I'll test the workaround and let you know.

[emmahodcroft]

Thank you for the detailed information and digging into this this Ivan!