Skip to content

Commit 0ba8d6f

Browse files
Clarify readme re parallelism and @testset (#167)
* Add section on parallelism * Clarify `at-testset` usage
1 parent f25832e commit 0ba8d6f

File tree

1 file changed

+26
-4
lines changed

1 file changed

+26
-4
lines changed

README.md

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,28 @@ Filtering by `name` and `tags` can be combined to run only test-items that match
9595
julia> runtests("test/Database/"; tags=:regression, name=r"^issue")
9696
```
9797

98+
99+
#### Running tests in parallel
100+
101+
You can run tests in parallel on multiple worker processes using the `nworkers` keyword.
102+
103+
The `nworkers` keyword specifies the number of worker processes to use for running tests in parallel:
104+
- `nworkers=0` (the default), runs tests sequentially on the current process.
105+
- `nworkers=1` runs tests sequentially in a new process.
106+
- `nworkers=2` runs tests in parallel on 2 new processes (and so on for `nworkers=3`,
107+
`nworkers=4`, ...)
108+
109+
All new workers processes are started before any tests are run. Each worker runs the `worker_init_expr` (if given), and then selects a test-item to run from a global queue in a work-stealing manner until all test-items have been run.
110+
111+
The number of threads each worker processes should have is specified by the `nworker_threads` keyword.
112+
For example, `nworker_threads=2` will start each worker process with 2 threads, and
113+
`nworker_threads="2,1"` with start each worker with 2 default threads and 1 interactive
114+
thread
115+
(see [the Julia manual on
116+
threadpools](https://docs.julialang.org/en/v1/manual/multi-threading/#man-threadpools)).
117+
118+
Note ReTestItems.jl uses distributed parallelism, not multi-threading, to run test-items in parallel.
119+
98120
## Writing tests
99121

100122
Tests must be wrapped in a [`@testitem`](https://docs.juliahub.com/General/ReTestItems/stable/autodocs/#ReTestItems.@testitem-Tuple{Any,%20Vararg{Any}}).
@@ -205,14 +227,14 @@ runtests("frobble_tests.jl"; nworkers, worker_init_expr)
205227
1. Write tests inside of an `@testitem` block.
206228
- These are like an `@testset`, except that they must contain all the code they need to run;
207229
any imports or definitions required for the tests must be inside the `@testitem`.
208-
- A `@testset` can still be used to add structure to your tests, but all `@testset`s must be inside an `@testitem`.
209-
These nested `@testset`s can add structure to the reporting, but serve no other purpose.
230+
- A `@testset` can still be used, but all `@testset`s must be inside an `@testitem`.
231+
These nested `@testset`s can add structure to your test code and to the summary of results printed out at the end of the tests, but serve no other functional purpose.
210232
- Tests that might previously have had imports and `struct` or `function` definitions outside of an `@testset` should instead now declare these inside of a `@testitem`.
211-
- `@testitem` will be run in parallel (using whatever threads or workers are available to the current Julia process).
233+
- `@testitem` can be run in parallel by setting the `nworkers` keyword.
212234
2. Write shared/re-used testing code in a `@testsetup module`
213235
- If you want to split tests up into multiple `@testitem` (so they can run in parallel), but also want to share common helper functions, types, or constants,
214236
then put the shared helper code in a module marked with `@testsetup`.
215-
- Each `@testsetup` will only be evaluated once per Julia process.
237+
- Each `@testsetup` will only be evaluated once per worker process that requires it.
216238
- A `@testsetup module` is recommended to be used for sharing helper definitions or shared immutable data;
217239
not for initializing shared global state that is meant to be mutated (like starting a server).
218240
For example, a server should be explicitly started and stopped as needed in a `@testitem`, not started within a `@testsetup module`.

0 commit comments

Comments
 (0)