Revisit strict CommonMark in PHEPs

[jtniehof]

#31 brought up the question of tables in PHEPs. I'm going to try to summarize the situation here.

Markdown tables are a GitHub extension to CommonMark. By the first telecon on PHEPs we had pretty quickly settled on using MarkDown instead of rst, and decided in favor of CommonMark without extensions. Some discussion is in the PHEP1 thread: 1, 2, but eschewing extensions was not extensively discussed. I took that as "accepted right off" but might have been "lost in the noise".

A few options for handling tables:

1. Do not use tables and present in some other form
2. Use an HTML table instead (I haven't checked that this renders properly in, say, pandoc!)
3. Update / replace PHEP1 and switch to GitHub markdown (and update the Content-Type header)
4. Don't update PHEP1 and just proceed with using GitHub markdown anyhow.
5. Allow only tables but not other extensions?!?
6. Reconsider MarkDown and particularly CommonMark.

Regarding breadth of support:

• reText doesn't support tables.
• pandoc with -f commonmark doesn't. With -f markdown or -f gfm it does. (Although with gfm the table cells don't wrap, and it runs off the page).
• vscodium previews tables fine.
• Google Docs imports tables okay, too.

(This is essentially a subtopic of #27 but seems relatively urgent to resolve soon, and also a fairly meaty topic by itself.)

[jtniehof]

I mostly see this as a tradeoff between:

• flexibility is great
• features are great
• tabular presentation of information is great

vs.

• standards are great
• interoperability is great

We are a standards and interoperability focused group, so I do lean that way.

[rebeccaringuette]

Tables are a great concise way to present information. There should be a way to support this.

[nabobalis]

I will say tables suck no matter the language.

E.g. for this to render correctly in the sunpy readme this is the source code for an rst table,

+-----------------------------------+-----------------------------------+-----------------------------------+
|           Release                 |           Development             |           Community               |
+===================================+===================================+===================================+
|       |Latest PyPi Version|       |        |Python Versions|          |         |Matrix Chat Room|        |
+-----------------------------------+-----------------------------------+-----------------------------------+
|     |Latest Conda Version|        |     |Project Status: Active|      |     |OpenAstronomy Discourse|     |
+-----------------------------------+-----------------------------------+-----------------------------------+
|      |Zenodo - Latest DOI|        |  |Continuous Integration Status|  |    |Google Groups Mailing List|   |
+-----------------------------------+-----------------------------------+-----------------------------------+
|    |sunpy stable documentation|   |     |CodeCov Code Coverage|       |       |Powered by NumFOCUS|       |
+-----------------------------------+-----------------------------------+-----------------------------------+
|         |sunpy citation|          |                                   |            |pyOpenSci|            |
+-----------------------------------+-----------------------------------+-----------------------------------+

But we should pick a language that supports them.

Whether that be RST or GitHub Markdown or even https://mystmd.org/

[jtniehof]

I will say tables suck no matter the language.

+1 to that! One of the least annoying ways to write them is the rST list-table. Readability of the source is meh, but that's sort of true of any source format for any table that's non-trivial.

[jtniehof]

I did a little poking around because table support Seems Like Something That Should Be Obvious In Markdown, and I'm starting to understand why it's not.

There are several proposals for syntax in the CommonMark discussion, from 2014, 2015, 2022.

Someone has also tried to provide a detailed spec encompassing various existing implementations

Here's the pandoc spec for their four different types of tables, and the PHP spec (which is basically GFM).

None of this is to say "tables are a nonstarter", just "tables aren't as consistenty supported or intuitive as one might like". At the point where somebody made up a Markdown table generator, we're a long ways from "easy-to-read and easy-to-write...without looking like it’s been marked up with tags or formatting instructions."*. The original Markdown syntax description used tables as the canonical example of "just drop to HTML".

[jtniehof]

Throwing in notes from Whatever The Last Meeting We Had Was:

@wtbarnes noted that rST does render nicely in GitHub now.

@namurphy suggested MyST which extends CommonMark (and @nabobalis seconded that suggestion in the meeting, and has noted above).

[jtniehof]

I'm going to throw out a Big Change here. Our PHEPs take a two round vote. And the definition of how we deal with PHEPs (including the two round vote) can be updated just like anything else. So in theory we can override any aspect of the PHEP process by first passing a replacement for PHEP 1, then pass something under that new process. The protection against things going off the rails is that community vote.

Given that it's unlikely we'll get process "perfect" at any point, it seems reasonable to provide a way of streamlining this sort of approach.

Here's my suggestion: any PHEP may violate any approved process PHEP, contingent on:

• The PHEP must include a description of what PHEP it is violating, what portion of that PHEP, the reason for the violation, and any mitigation of the impacts of the violation.
• No PHEP may so override its own approval process (this could use a lot of clarifying) or the Code of Conduct.

Obviously this mechanism would need to be voted on (either establishing a new process, or replacing PHEP 1). But once that was done, PHEP 4 could say: this is violating the CommonMark requirement, it is only using tables based on (e.g.) MyST table extension, the header will be updated to indicate this.

Canadians may recognize this as modeled on the Notwithstanding Clause. Which is not exactly a pedigree suggesting it's a great idea...