PHEP 4: PyHC Package Tiering

pheps/phep-0004.md

@@ -0,0 +1,145 @@
+```
+PHEP: 4
+Title: PyHC Package Tiering
+Author: Julie Barnum <julie.barnum@lasp.colorado.edu> <https://orcid.org/0000-0001-8755-0694>
+Discussions-To: https://github.com/heliophysicsPy/standards/pull/31
+Revision: 1
+Status: Draft
+Type: Process
+Content-Type: text/markdown; charset=UTF-8; variant=myst
+Created: 09-July-2024
+Post-History: 09-July-2024, 26-August-2024, 27-August-2024, 4-September-2024, 11-September-2024, 13-September-2024, 25-April-2025, 05-June-2025, 26-Jun-2025
+```
+
+# Abstract
+<a name="abstract"></a>
+This PHEP establishes a new tiering structure to PyHC projects, which will automatically affect PyHC packages once it goes into effect. Included herein is information on requirements for each of the new three tiers of PyHC projects (Gold, Silver, and Bronze), as well as benefits accrued at each tier.
+
+# Motivation
+<a name="motivation"></a>
+Currently, PyHC is at a crossroads for how to push forward as a community. There are two main schools of thought — originating from bi-annual meeting discussions, telecon chats, and further sidebar conversation — with regards to what PyHC is and should be:
+
+- a basic interpretation where PyHC is a collection, and listing, of open-source Python packages with a relevance to Heliophysics and space physics, or
+- a standards-based interpretation where PyHC strives for compliance with our set standards, package interoperability, and standardization around one or more tools.
+
+There is utility and validity to both approaches. A new PyHC package tiering system is intended to find a "best of both worlds" with the two ideas. Older, out-of-date, unmaintained, or publication-specific code could still have a place for listing and findability, while also allowing nuance between other packages that are more robust, trustworthy, maintained, and work toward the standards-based interpretation of being a PyHC package.
+
+Further, this tiering system also allows users to get a clearer picture on what each PyHC package has to offer, and the state of the package's condition and development. Creation of a PyHC package tiering system also allows for justification for a myriad of benefits, for example, consideration for funding from a community travel fund, or extra help with improving a standards grouping grade.
+
+# Rationale
+<a name="rationale"></a>
+Decisions for tiering levels, requirements for each tier, and benefits accrued at each tier are based on conversations with the community (bi-annual meetings, telecons, etc.), and are listed here as a starting point for more discussion, likely to be refined in the future. Initially, ideas were presented to the community in a pyramid format. To make the differences between tiers more visible and understandable, it has been transformed into a spreadsheet format.
+
+This PHEP intentionally violates the PHEP-1 requirement of being formatted in strict CommonMark markdown. CommonMark does not support tables and tabular presentation is the clearest means of communicating the tier structure. It uses [MyST markdown](https://mystmd.org/) for tables only, and the Content-Type indicates that. MyST is commonly used in the scientific Python community and is a strict superset of CommonMark, so is the most compatible means of supporting tabular information.
+
+# PyHC Package Tiering Specifications
+<a name="specification"></a>
+There are three tiers proposed in this PHEP: Bronze, Silver, and Gold. These would replace the current structure of "Core Packages", "Other Packages", and "Un-evaluated Packages". Extensive updates have been made to the following tiering specifications, based on discussions held at the [PyHC Fall 2024 meeting](https://doi.org/10.5281/zenodo.15080483). See the table below for requirements associated with each tier:
+
+|       | **Gold** | **Silver** | **Bronze** |
+| :-: | :----------: | :-----------: | :------------: |
+| Self-Evaluation | No | No | Yes |
+| PyHC-pyOpenSci Review | Yes, defined in PyHC-pyOpenSci Gold-tier PHEP | Yes, defined in PyHC-pyOpenSci Silver-tier PHEP | No |
+| PyHC Standards Compliance | Packages must comply with all applicable standards (as defined in standards-track PHEPs) |
+
+Descriptions for each heading are as follows:
+   - Self Evaluation: indicates whether a package must complete a self evaluation against PyHC's standards
+   - pyOpenSci Review: indicates whether a package must engage in a PyHC-pyOpenSci review
+   - PyHC Standards Compliance: indicates necessary level of compliance with standards-track PHEPs (set to replace the current PyHC standards document: https://doi.org/10.5281/zenodo.2529131)
+
+The following table shows the benefits that are associated with each tier:
+
+|       | **Gold** | **Silver** | **Bronze** |
+| :-: | :----------: | :-----------: | :------------: |
+| Summer School Inclusion \* | Invited to give a tutorial at the summer school | Invited to give a tutorial at the summer school if space allows\** | Can be brought in by higher-tier package, but not directly invited |
+| PyHC-Chat Bot or PyHC Environment(s) Inclusion \* | Catch-all: higher-tier packages are higher priority for cross-PyHC projects |
+| Standards Compliance Assistance \* | Yes\*** | Yes | Yes |
+| Conference Travel Funding | Yes\* | Yes, if availability after gold-tier packages\*, \** | No |
+
+\*  = can opt-in or opt-out of this
+
+\** = conditional upon justification on importance/use of package (e.g. number of users, level of commitment within PyHC activities, or effort made for those items but gold level not yet achieved)
+
+\*** = conditional upon justification that package is in danger of dropping down to a Silver tier
+
+Descriptions for each heading are as follows:
+   - Summer School Inclusion: indicates whether a package will be included in summer school teaching materials
+   - PyHC-Chat Bot or PyHC Environment(s) Inclusion: indicates whether a packages will have up-to-date information included within the ChatGPT4-powered PyHC-Chat bot or be prioritized for inclusion in future PyHC environment(s)
+   - Standards Compliance Assistance: indicates whether a package will receive extra help and/or funding (dependent on available PyHC Leadership resources) for conforming to the PyHC standards
+   - Conference Travel Funding: indicates whether developers from a package will be considered for PyHC travel funding assistance to relevant science conferences (e.g. SHINE, CEDAR, or GEM). All tiers are conditional upon availability of funds through PyHC Leadership.
+
+Packages are evaluated (as described in 'Implementation Process' below) against compliance with each requirement, as shown in the PyHC tiering chart. To be accepted for a tier, a package must meet **all** the requirements for said tier. Therefore, should a package fall into different tiers depending on the specific requirement, the package will be accepted at the lowest tier of requirements it meets.
+
+For example, if a package meets some requirements for the Silver tier, but other requirements only meet the Bronze tier, the package will be considered a Bronze tier package.
+
+
+# Package Tiering Implementation Process
+
+Following acceptance of this PHEP, the PyHC website will be updated to reflect the new tiers. The process is then as follows:
+- Within six months of PyHC website update (written communication of this provided in email form), packages must submit a PR to [the PyHC website GitHub](https://github.com/heliophysicsPy/heliophysicsPy.github.io) indicating what level they believe their package is
+    - need to include written justification on the PR for chosen tier
+- Within three months of PR publish, members of the to-be-established technical steering committee (TSC)\* review the package and assert if self-evaluated tier matches expectations. If a conflict of interest during a review exists, that TSC member may be replaced during that specific review only. After the TSC review, the process is as follows:
+    - 1) The PR is merged if there's a simple majority of TSC member in agreement with the self-evaluated tier, or
+    - 2) The PR is rejected (and the TSC collectively craft and provide written reasoning through the PyHC Lead for a differing tier level) if a simple majority of TSC members do not agree with the self-evaluated tier.
+
+\* The TSC includes the PyHC Lead, PyHC Tech Lead, as well as volunteer members of the PyHC not involved with the submitted package, with a minimum of 4 total members up to a maximum of 6 total members. Volunteer members of the TSC will serve a one-year tenure, decided upon at the nearest telecon or bi-annual meeting following the written email communication of this PHEP's implementation and at a yearly cadence thereafter. After the year tenture ends, TSC volunteer members can choose to stay or cycle out with a new PyHC member. If a PyHC member is interested in being involved, but cannot attend that telecon or bi-annual meeting where TSC members are chosen, they may email the PyHC Lead with their request for candidacy at least one business day beforehand (to give the PyHC Lead time to see the email). Lastly, future PHEPs may further define the TSC and may supersede this PHEP regarding the TSC without affecting other standards and processes in this PHEP.
+
+General comments:
+
+Packages that do not submit a self-evaluated tier will be automatically assigned a tier after review by the TSC. They may still be eligile later on for a tier re-grade.
+
+Packages are allowed to move between tiers. If a package upgrades their status to match that of Silver, instead of Bronze, tier, for example, they can submit a new PR to have their tier re-graded.
+
+The flip side is also true; should a package become defunct or drop in status, they may be downgraded to a lower tier. The TSC shall meet on an annual basis (likely tied to a bi-annual meeting) to review package tier status and determine if any are in danger of a tier downgrade. A package downgrade requires consensus of the technical steering committee.
+
+Should any package be flagged for downgrade, the follow will occur:
+- Packages will receive six months' notification before a tier downgrade is to take place through written e-mail communication (from the PyHC Lead)
+- Packages then have the opportunity to rectify issues within the six-month period, through changes submitted in a PR
+    - If changes result in continued compliance with current tier (as deemed by the TSC), no tier re-grade happens
+    - If changes do not result in continued compliance with current tier (as deemed by the TSC), a tier re-grade happens; packages may later on apply for a tier re-grade once more.
+
+# Backwards Compatibility
+<a name="backwards-compatibility"></a>
+This PHEP does not propose a direct change to PyHC package code, simply the inclusion or not of packages within the various tiers, thus it introduces no compatibility concerns.
+
+# Security Implications
+<a name="security-implications"></a>
+This PHEP raises no security implications as it does not interact with any executing code.
+
+# How to Teach This
+<a name="how-to-teach-this"></a>
+This PHEP's contents and changes will be presented on, discussed, and hacked at various PyHC bi-weekly telecons and PyHC bi-annual meetings. Additionally, explanations for tiering, the process of obtaining a PyHC tier, etc. will be posted on the new main Projects page, as well as communicated within a blog post under the PyHC Blog page.
+
+Further, links to pages (and potentially recordings) that explain how to satisfy each requirement for each tier will be made and communicated through a PyHC blog post (link then emailed to the community and presented at either a telecon or bi-annual meeting, whichever comes first).
+
+Please note that the PyHC project submission process used thus far will be superseded by this PHEP's PyHC tier request submission. Further, the self evaluations relied upon previously will now only grant a package Bronze-tier status; Technical steering committee evaluation is required to fully verify a package's tier. Also, there are now additional requirement for package tiers that exist apart from the main PyHC standards (as detailed in this PHEP). PyHC leadership commits to updating the submission process to match this PHEP's specifications.
+
+# Rejected Ideas
+<a name="rejected-ideas"></a>
+None yet to note.
+
+# Open Issues
+<a name="open-issues"></a>
+None yet to note.
+
+# Footnotes
+<a name="footnotes"></a>
+None yet to note.
+
+# Revisions
+<a name="revisions"></a>
+Revision 1 (pending): Initial draft.
+
+# Copyright
+<a name="copyright"></a>
+This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. It should be cited as:
+```
+@techreport(phep4,
+  author = {Julie I. Barnum},
+  title  = {PHEP Package Tiering},
+    year = {2024--2025},
+    type = {PHEP},
+  number = {4},
+     doi = {10.5281/zenodo.15747682}
+)
+```