Skip to content

Update process results workflow for new website #919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
rcannood merged 87 commits into from
Aug 19, 2025
Merged

Update process results workflow for new website #919

rcannood merged 87 commits into from
Aug 19, 2025

Conversation

@lazappi
Copy link
Member

@lazappi lazappi commented Feb 3, 2025 •
edited
Loading

Describe your changes

  • Add a new component to combine results output
  • Update the results processing workflow to output combined results
  • Update results format to new JSON schemas

TODO:

  • Output metric execution info from get_results
  • Update generate_qc to new schema
  • Update combine_output to new schema
  • Add schema validation to components
    • Add "additionalProperties": false to schemas
    • Add schemas to input info
  • Add roles to author schema
  • Add report
  • Update workflow to new components
  • Update container image versions
  • Strip quotes from descriptions
  • Add QC check for missing controls
  • Add a script for creating results v4 test resources
  • Update the main repository reference
  • Test reprocesssing existing results

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
    • Documentation

  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

@lazappi
Copy link
Member Author

lazappi commented Feb 3, 2025

This should produce a combined tasks.json file in the format required by the new website. However, parts of that file (datasets etc.) will be in the v2 format which the website may not expect.

@lazappi lazappi marked this pull request as draft February 10, 2025 09:29
lazappi and others added 18 commits July 17, 2025 08:40
@lazappi
Add metric resources to results component/schema
d318832
@lazappi
Update generate_qc to match new schema
d5d1b8f 
Add some new QC checks
@lazappi
Update combine_output to new schema
11d6d9d
@lazappi
Update viash version and reference
6a91673
@lazappi
Update process_results workflow to new components
5f70ab9
@lazappi
Add render_report component
99a4a50
@lazappi
Add render_report to process results workflow
264790e
@lazappi
Handle missing values in generate_qc()
272bba8 
This can happen when a method is disabled/skipped so there are no
results to check
@lazappi
Handle missing controls in results report
8ab4f60 
Skip scaling/results when there are no controls or give a warning when
some controls failed
@rcannood
update common submodule
d8fdfd9
@rcannood
Merge remote-tracking branch 'origin/main' into feature/no-ref/update…
15a2d18 
…-process-results
@lazappi
Merge remote-tracking branch 'origin' into feature/no-ref/update-proc…
e9e3ef1 
…ess-results

* origin:
  Rename items (#932)
  Rename dataset loader and add compression argument to more components (#930)
  Added op3 loader (#926)
  Patch: Update image version for openproblems/base_* images (#928)
  bump viash version (#925)
  Add email address to CoC (#923)
@lazappi
Merge branch 'feature/no-ref/update-process-results' of github.com:op…
cbf0e50 
…enproblems-bio/openproblems into feature/no-ref/update-process-results

* 'feature/no-ref/update-process-results' of github.com:openproblems-bio/openproblems:
  update common submodule
@lazappi
Strip quotes from descriptions/summaries
c1198e5
@lazappi
Add roles to author details
a7b9c6d
@lazappi
Add QC check for number of successful controls
19ecc9d
@lazappi
Handle missing exit codes in report
dc75604
@lazappi
Add schema validation to process_results workflow
382a35d
@lazappi lazappi marked this pull request as ready for review July 30, 2025 13:31
@lazappi lazappi requested a review from rcannood July 30, 2025 13:32
lazappi and others added 25 commits August 7, 2025 08:43
@lazappi
Handle minimal dataset info in old results
f3381a0
@lazappi
Handle missing file size in get_dataset_info
e23473b
@lazappi
Handle empty string in get_references_list()
51e5fa6
@lazappi
Handle method info stored in functionality field
7ce4cc8
@lazappi
Move get_additional_info() to shared functions
2df3b82
@lazappi
Handle missing maximize in get_metric_info
6667812
@lazappi
Handle missing metric values in get_results
73dbf12
@lazappi
Properly handle workflow component in get_results
bb87f2e
@lazappi
Handle metrics stored in functionality field
732a2f2
@lazappi
Give better error when dataset IDs don't map
26189c7
@lazappi
Remove duplicate datasets in get_dataset_info
0634e62
@lazappi
Handle infinite values in generate_qc
eaa1f17
@lazappi
Add check that any valid scores are found
ad5b6b7
@lazappi
Adjust dataset process mapping
a427571
@lazappi
Handle source_urls in render_report
fe3f2e1
@lazappi
Fix additional info in get_method_info
7d39c12
@lazappi
Handle missing file size/date in report
96e8e4e
@lazappi
Use regex to match DOI references
8031e40
@lazappi
Fix empty scores check in report
072a6a3
@lazappi
Fix DOI regex
583fdcf
@lazappi
Handle DOIs without text citations
7e7c11a
@lazappi
Merge remote-tracking branch 'origin/feature/no-ref/update-process-re…
efa00bb 
…sults' into feature/no-ref/update-process-results

* origin/feature/no-ref/update-process-results:
  fix script
@lazappi
Warn about missing values for succeeded metrics
c0dd768
@lazappi
Improve controls check in report
ca4bb8c
@rcannood
update submodule
6ba4f49
@rcannood

This comment was marked as outdated.

Sign in to view
@rcannood @lazappi
Add results filtering (#935)
e8119ce 
* introduce new component

* implement component

* format code

* Update src/reporting/process_task_results/main.nf

* Apply suggestions from code review

Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com>

* add back previously removed arguments

---------

Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com>
rcannood
rcannood approved these changes Aug 19, 2025
View reviewed changes
Copy link
Member

@rcannood rcannood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rcannood rcannood merged commit 6729bf6 into main Aug 19, 2025
10 checks passed
@rcannood rcannood deleted the feature/no-ref/update-process-results branch August 19, 2025 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rcannood rcannood rcannood approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lazappi @rcannood