Skip to content

A/B test report #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
crusaderky wants to merge 6 commits into from
Closed

A/B test report #307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
80a68f1
WIP barchart for comparing different commits rather than a timeseries.
Sep 1, 2022
b03c3f1
Add PyCharm files to .gitignore
crusaderky Sep 2, 2022
87fd490
Merge branch 'main' into barchart
crusaderky Sep 2, 2022
80c6461
Merge branch 'main' into barchart
crusaderky Sep 2, 2022
689f39a
WIP
crusaderky Sep 6, 2022
273a0c0
dynamic barchart height
crusaderky Sep 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,19 @@ jobs:
if [[ "$COMMIT" == *"test-upstream"* || ${{ github.event_name }} == "schedule" ]]
then
export TEST_UPSTREAM="true"
export AB_BASELINE="coiled-upstream-py3.9 coiled-0.1.0-py3.9"
else
export TEST_UPSTREAM="false"
export AB_BASELINE="coiled-0.1.0-py3.9"
fi

# Put TEST_UPSTREAM into $GITHUB_ENV so it can be used in subsequent workflow steps
echo $TEST_UPSTREAM
echo TEST_UPSTREAM=$TEST_UPSTREAM >> $GITHUB_ENV

# Put TEST_UPSTREAM into a file so it can be downloaded in subsequent workflow jobs
# Put env variables into files so it can be downloaded in subsequent workflow jobs
echo $TEST_UPSTREAM > test_upstream.txt
echo $AB_BASELINE > ab_baseline.txt

- name: Build Coiled Software Environment
env:
Expand Down Expand Up @@ -104,6 +107,7 @@ jobs:
latest.yaml
software_name.txt
test_upstream.txt
ab_baseline.txt

runtime:
name: Runtime - ${{ matrix.os }}, Python ${{ matrix.python-version }}, Runtime ${{ matrix.runtime-version }}
Expand Down Expand Up @@ -457,9 +461,14 @@ jobs:
python-version: "3.9"
environment-file: ci/environment-dashboard.yml

- name: Download software environment assets
uses: actions/download-artifact@v3
with:
name: software-environment-py3.9

- name: Generate dashboards
run: |
python dashboard.py
python dashboard.py -d benchmark.db -o static -b <(cat ab_baseline.txt)

- name: Deploy 🚀
uses: JamesIves/github-pages-deploy-action@4.1.7
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,9 @@ venv.bak/
# Rope project settings
.ropeproject

# PyCharm project settings
.idea

# mkdocs documentation
/site

Expand Down
1 change: 1 addition & 0 deletions ci/environment-dashboard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ dependencies:
- dask
- dask-ml
- distributed
- xarray
- xgboost
- pandas
- tabulate
Loading