Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a061fb4
tdd initial tests
jaymedina Sep 29, 2025
d85a9d7
initial intro of dataclasses
jaymedina Sep 29, 2025
50498b7
expose api services for submission object
jaymedina Sep 30, 2025
94cedc6
style and update docstring
jaymedina Sep 30, 2025
cd19e0c
add submission and submissionstatus models
jaymedina Oct 10, 2025
99cba20
add submission status retrieval and update methods; remove empty subm…
jaymedina Oct 10, 2025
9ca2448
pipe query params directly into restAPI httpx requests
jaymedina Oct 14, 2025
f0d7ecc
new dataclass object submission_bundle
jaymedina Oct 14, 2025
91b947b
move submission services functions to evaluation_services.py
jaymedina Nov 5, 2025
a6b46d9
renaming imports, to_synapse_request, request body refactor
jaymedina Nov 10, 2025
b68bd8e
patching up store method signature
jaymedina Nov 10, 2025
5a44f07
update docs
jaymedina Nov 10, 2025
65664d1
new suite of tests
jaymedina Nov 10, 2025
d214673
submissionstatus rework as a mutable object
jaymedina Nov 10, 2025
2781797
bug fix for Statuses: updated to_synapse_request to follow same patte…
jaymedina Nov 11, 2025
3230b1f
replace != with is not for full object comparison (not just keys)
jaymedina Nov 11, 2025
2d43e5d
expose the is_private arg for to_submission_status_annotations ONLY F…
jaymedina Nov 11, 2025
d4d63ba
fixed submission status/submission annotations
jaymedina Nov 12, 2025
8b9f9c9
add support for legacy annotations
jaymedina Nov 13, 2025
6aa914a
remove debug prints
jaymedina Nov 13, 2025
2d67fcc
get_all_submission_statuses now returns a list of substat objects
jaymedina Nov 13, 2025
c368e49
docstring updates
jaymedina Nov 13, 2025
fbd43f6
update submissionbundle docstrings, add more examples
jaymedina Nov 14, 2025
d5fac7e
initial sync test for status. moved evaluation_id retrieval to fill_f…
jaymedina Nov 14, 2025
47797c0
update submissionBundle submissionstatus with evaluation_id
jaymedina Nov 14, 2025
e8e5361
patch sync substatus integ tests. style.
jaymedina Nov 14, 2025
58ffe42
fix submissionStatus integ tests and has_changed attribute
jaymedina Nov 17, 2025
f34667d
new substatus async integ tests. can_cancel can now be modified by an…
jaymedina Nov 20, 2025
5e762f0
new test class for submission cancel functionality
jaymedina Nov 20, 2025
1bd7f4b
substatus async unit tests
jaymedina Nov 20, 2025
4c8904d
remove compare=false for some attributes. update sync unit tests
jaymedina Nov 20, 2025
a223320
add submissionBundle integration tests
jaymedina Nov 20, 2025
f059751
add submissionBundle unit tests
jaymedina Nov 20, 2025
3cb1915
remove unnecessary imports and add style
jaymedina Nov 20, 2025
1fb0fbf
get_evaluation_submissions returns generator object
jaymedina Nov 21, 2025
281ed3d
get_user_submissions returns generator object
jaymedina Nov 21, 2025
1168cae
submissionBundle methods return generators
jaymedina Nov 21, 2025
a9977b2
address final todo: implement docker_tag, and note in docs that versi…
jaymedina Nov 21, 2025
689fcb1
add async page in api references
jaymedina Sep 30, 2025
c54609b
add initial submission, status, bundle docs
jaymedina Nov 21, 2025
e0a79cc
updated tutorial purpose. add api reference to navbar
jaymedina Nov 24, 2025
35efed7
new tutorial scripts
jaymedina Nov 24, 2025
e07206d
remove try/excepts. add line references. add resources and source cod…
jaymedina Dec 1, 2025
e193753
add reference to File model
jaymedina Dec 1, 2025
b710da0
deprecate old submission model
jaymedina Dec 1, 2025
7917011
style
jaymedina Dec 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/reference/experimental/async/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Submission

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.Submission
options:
inherited_members: true
members:
- store_async
- get_async
- delete_async
- cancel_async
- get_evaluation_submissions_async
- get_user_submissions_async
- get_submission_count_async
14 changes: 14 additions & 0 deletions docs/reference/experimental/async/submission_bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Submission Bundle

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionBundle
options:
inherited_members: true
members:
- get_evaluation_submission_bundles_async
- get_user_submission_bundles_async
16 changes: 16 additions & 0 deletions docs/reference/experimental/async/submission_status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Submission Status

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionStatus
options:
inherited_members: true
members:
- get_async
- store_async
- get_all_submission_statuses_async
- batch_update_submission_statuses_async
19 changes: 19 additions & 0 deletions docs/reference/experimental/sync/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Submission

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.Submission
options:
inherited_members: true
members:
- store
- get
- delete
- cancel
- get_evaluation_submissions
- get_user_submissions
- get_submission_count
14 changes: 14 additions & 0 deletions docs/reference/experimental/sync/submission_bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Submission Bundle

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionBundle
options:
inherited_members: true
members:
- get_evaluation_submission_bundles
- get_user_submission_bundles
16 changes: 16 additions & 0 deletions docs/reference/experimental/sync/submission_status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Submission Status

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionStatus
options:
inherited_members: true
members:
- get
- store
- get_all_submission_statuses
- batch_update_submission_statuses
131 changes: 131 additions & 0 deletions docs/tutorials/python/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Submissions, SubmissionStatuses, SubmissionBundles
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not seeing this in the index page of tutorial:

Screenshot 2025-12-02 at 1 57 43 PM

Users can work with Submissions on the python client, since these objects are part of the Evaluation API data model. A user in a Synapse Evaluation can submit a Synapse Entity as a Submission to that Evaluation. Submission data is owned by the parent Evaluation, and is immutable.

The data model includes additional objects to support scoring of Submissions and convenient data access:

- **SubmissionStatus**: An object used to track scoring information for a single Submission. This object is intended to be modified by the users (or test harnesses) managing the Evaluation.
- **SubmissionBundle**: A convenience object to transport a Submission and its accompanying SubmissionStatus in a single web service call.

This tutorial will demonstrate how to work with all 3 object types using the python client for 2 different use-cases:

1. Participating in a Synapse challenge
1. Organizing a Synapse challenge


## Tutorial Purpose
In this tutorial:

As a participant of a Synapse challenge, you will

1. Make a submission to an existing evaluation queue on Synapse
1. Fetch your existing submission
1. Count your submissions
1. Fetch all of your submissions from an existing evaluation queue on Synapse
1. Check the status of your submission
1. Cancel your submission

As an organizer of a Synapse challenge, you will

1. Annotate a submission to score it
1. Batch-update submission statuses
1. Fetch the submission bundle for a given submission
1. Allow cancellation of submissions
1. Delete submissions

## Prerequisites
* You have completed the [Evaluation](./evaluation.md) tutorial, or have an existing Evaluation on Synapse to work from
* You have an existing entity with which to make a submission (can be a [File](./file.md) or Docker Repository)
* You have the correct permissions on the Evaluation queue for your desired tutorial section (participant or organizer)

## 1. Participating in a Synapse challenge

### 1. Make a submission to an existing evaluation queue on Synapse

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=30-51}
```

### 2. Fetch your existing submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=56-71}
```

### 3. Count your submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=72-92}
```

### 4. Fetch all of your submissions from an existing evaluation queue on Synapse

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=93-107}
```

### 5. Check the status of your submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=108-130}
```

### 6. Cancel your submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=131-157}
```

## 2. Organizing a Synapse challenge

### 1. Annotate a submission to score it

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=29-57}
```

### 2. Batch-update submission statuses

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=58-101}
```

### 3. Fetch the submission bundle for a given submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=102-142}
```

### 4. Allow cancellation of submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=143-195}
```

### 5. Delete submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=196-229}
```

## Source code for this tutorial

<details class="quote">
<summary>Click to show me (source code for Participant)</summary>

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!}
```
</details>

<details class="quote">
<summary>Click to show me (source code for Organizer)</summary>

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!}
```
</details>

## References
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing submission?

- [Evaluation][synapseclient.models.Evaluation]
- [File][synapseclient.models.File]
- [syn.login][synapseclient.Synapse.login]
Loading
Loading