Resources not matching schema should be an error, and provide more info

[technicalpickles]

Given this config:

apiVersion: scope.github.com/v1alpha
kind: ScopeDoctorGroup
metadata:
  name: kafka
  description: Kafka
spec:
  actions:
    - name: port-available
      description: Port available
      needs:
        - tes
      check:
        commands:
          - script/wait-for-it.sh localhost:9092 --timeout=1 --quiet
      fix:
        commands:
          - docker-compose up -d kafka
          - script/wait-for-it.sh localhost:9092 --timeout=45

I get a warning that it doesn't match the schema, and it reports as successfully:

❯ scope doctor run --only server                                                                                                                                                                                                                                       
 WARN Resource 'ScopeDoctorGroup/kafka' didn't match the schema for ScopeDoctorGroup. Additional properties are not allowed ('needs' was unexpected)
Summary: 1 groups succeeded

This sends some mixed messaging. The succeeded language suggests it is run despite it matching the schema.

I think as someone writing and implementing scope checks, I'd want to see this be an error and not run anything else.

Some things I think I'd want to see when that happens:

• the path to the file
• what part didn't match, bonus points for line numbers
• link to the supported schema

[ethankhall]

I don't think that erroring is correct, because you may a future version of the model, and wouldn't be able to run because a new (valid) field isn't known yet.

I think what you're getting at is a lint or validate step that would give you the information you're looking for. Then you could add it to CI and ensure that your resources are valid.

[noizwaves]

@technicalpickles something that might help your experience here is configuring IDE based YAML schema validation against https://raw.githubusercontent.com/oscope-dev/scope/main/scope/schema/merged.json. For Gusto's repo, I have this config committed, but only for VS Code.