Support Protobuf Editions

[spenczar]

Editions are a fancy new way that protobuf can expand the feature set gradually. Plugin authors are supposed to describe their minimum and maximum supported edition in the CodeGeneratorResponse they produce.

It would be nice if the protogen Options class supported these. I'd propose a type signature like this on init_:

    def __init__(
        self,
        *,
        ...
        supported_editions: Optional[Tuple[descriptor_pb2.Edition, descriptor_pb2.Edition]] = None
    ):

If we get a non-None value for supported_editions, then we add the "supports editions" feature to the supported feature set, and mark the appropriate spots in the CodeGeneratorResponse.

I've heard this will get increasingly important, as code generators will simply not work with proto files using Editions unless they opt in in this way.

[fischor]

In general yes, I think the library should support that going forward. I am happy to discuss how to implement that.

See https://protobuf.dev/editions/implementation/#generators under "Explicit support":

Generators must specify exactly which editions they support. This allows you to safely add support for an edition after it’s been released, on your own schedule. Protoc will reject any editions protos sent to generators that don’t include FEATURE_SUPPORTS_EDITIONS in the supported_features field of their CodeGeneratorResponse. Additionally, we have minimum_edition and maximum_edition fields for specifying your precise support window. Once you’ve defined all of the code and feature changes for a new edition, you can bump maximum_edition to advertise this support.
Codegen Tests

These fields have been added to the CodeGeneratorResponse in plugin.proto:

 // A bitmask of supported features that the code generator supports.
  // This is a bitwise "or" of values from the Feature enum.
  optional uint64 supported_features = 2;

  // Sync with code_generator.h.
  enum Feature {
    FEATURE_NONE = 0;
    FEATURE_PROTO3_OPTIONAL = 1;
    FEATURE_SUPPORTS_EDITIONS = 2;
  }

  // The minimum edition this plugin supports.  This will be treated as an
  // Edition enum, but we want to allow unknown values.  It should be specified
  // according the edition enum value, *not* the edition number.  Only takes
  // effect for plugins that have FEATURE_SUPPORTS_EDITIONS set.
  optional int32 minimum_edition = 3;

  // The maximum edition this plugin supports.  This will be treated as an
  // Edition enum, but we want to allow unknown values.  It should be specified
  // according the edition enum value, *not* the edition number.  Only takes
  // effect for plugins that have FEATURE_SUPPORTS_EDITIONS set.
  optional int32 maximum_edition = 4;

So we might need to also pass the supported features to protoc?

---

But to be able to usefully with features, I think then we need to do the "Feature resolution" in protogen. As there seems to be for each feature a default value, which differs per proto2, proto3 and editionXYZ. And these values might be overwritten at different levels (see https://protobuf.dev/editions/features/ under "Features"):

Feature settings apply at different levels:
File-level: These settings apply to all elements (messages, fields, enums, and so on) that don’t have an overriding setting.
Non-nested: Messages, enums, and services can override settings made at the file level. They apply to everything within them (message fields, enum values) that aren’t overridden, but don’t apply to other parallel messages and enums.
Nested: Oneofs, messages, and enums can override settings from the message they’re nested in.
Lowest-level: Fields, extensions, enum values, extension ranges, and methods are the lowest level at which you can override settings.

We should provide an easy way to access at every possible level (global, file, message, field...) the feature settings that currently applies (this is hopefully just a matter of recursively passing down the feature value that currently applies and overwrite it in case the Feature is set/changes on the current level).

Then the plugin authors can just use e.g.

for message in file.messages:
   for field in message.fields:
      if field.features.field_presence == LEACY_REQUIRED:
          g.P("required ", field.py_name)
      elif ...

I will need to tinker around with this first to figure out how it works and how e.g. setting a feature behaves in the CodeGeneratorRequest that protoc generates..