Skip to content

add test for determinism (#32) #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
j143 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

add test for determinism (#32) #33

j143 wants to merge 1 commit into from
+86 −0

Conversation

@j143
Copy link
Owner

@j143 j143 commented Sep 11, 2025 •
edited
Loading 

======================================================================
FAIL: test_parallel_execution_is_non_deterministic (__main__.TestDeterminism.test_parallel_execution_is_non_deterministic)
Tests that two identical runs produce different event traces
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspaces/ooc/tests/test_determinism.py", line 77, in test_parallel_execution_is_non_deterministic
    self.assertNotEqual(
AssertionError: [('MISS', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 0)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/A.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 1024)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('HIT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 2048, 0)), ('EVICT', ('test_determinism_data/A.bin', 1024, 1024)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('EVICT', ('test_determinism_data/A.bin', 1024, 2048)), ('MISS', ('test_determinism_data/A.bin', 2048, 1024)), ('EVICT', ('test_determinism_data/A.bin', 1024, 3072)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 2048, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('MISS', ('test_determinism_data/A.bin', 2048, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 3072, 0)), ('EVICT', ('test_determinism_data/A.bin', 2048, 1024)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('EVICT', ('test_determinism_data/A.bin', 2048, 2048)), ('MISS', ('test_determinism_data/A.bin', 3072, 1024)), ('EVICT', ('test_determinism_data/A.bin', 2048, 3072)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 3072, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('MISS', ('test_determinism_data/A.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/B.bin', 3072, 0))] == [('MISS', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 0)), ('MISS', ('test_determinism_data/B.bin', 0, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 0)), ('HIT', ('test_determinism_data/A.bin', 0, 1024)), ('MISS', ('test_determinism_data/B.bin', 1024, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 0, 2048)), ('MISS', ('test_determinism_data/B.bin', 2048, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 0, 3072)), ('MISS', ('test_determinism_data/A.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 1024)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('HIT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 1024, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 1024, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 1024, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 1024, 0)), ('MISS', ('test_determinism_data/A.bin', 2048, 0)), ('EVICT', ('test_determinism_data/A.bin', 1024, 1024)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('EVICT', ('test_determinism_data/A.bin', 1024, 2048)), ('MISS', ('test_determinism_data/A.bin', 2048, 1024)), ('EVICT', ('test_determinism_data/A.bin', 1024, 3072)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 2048, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('MISS', ('test_determinism_data/A.bin', 2048, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 2048, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/A.bin', 2048, 0)), ('MISS', ('test_determinism_data/A.bin', 3072, 0)), ('EVICT', ('test_determinism_data/A.bin', 2048, 1024)), ('MISS', ('test_determinism_data/B.bin', 0, 0)), ('EVICT', ('test_determinism_data/A.bin', 2048, 2048)), ('MISS', ('test_determinism_data/A.bin', 3072, 1024)), ('EVICT', ('test_determinism_data/A.bin', 2048, 3072)), ('MISS', ('test_determinism_data/B.bin', 1024, 0)), ('EVICT', ('test_determinism_data/B.bin', 0, 0)), ('MISS', ('test_determinism_data/A.bin', 3072, 2048)), ('EVICT', ('test_determinism_data/B.bin', 1024, 0)), ('MISS', ('test_determinism_data/B.bin', 2048, 0)), ('EVICT', ('test_determinism_data/B.bin', 3072, 3072)), ('MISS', ('test_determinism_data/A.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/B.bin', 2048, 0)), ('HIT', ('test_determinism_data/B.bin', 3072, 0)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 1024)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('HIT', ('test_determinism_data/B.bin', 3072, 2048)), ('HIT', ('test_determinism_data/A.bin', 3072, 0)), ('HIT', ('test_determinism_data/B.bin', 0, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 1024)), ('HIT', ('test_determinism_data/B.bin', 1024, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 2048)), ('HIT', ('test_determinism_data/B.bin', 2048, 3072)), ('HIT', ('test_determinism_data/A.bin', 3072, 3072)), ('MISS', ('test_determinism_data/B.bin', 3072, 3072)), ('EVICT', ('test_determinism_data/B.bin', 3072, 0))] : Execution was deterministic, which is unexpected with the current parallel backend.

----------------------------------------------------------------------
Ran 1 test in 8.014s

FAILED (failures=1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@j143