Skip to content

Conversation

@yeya24
Copy link
Contributor

@yeya24 yeya24 commented Dec 8, 2025

What this PR does:

Try to fix the flaky query fuzz tests due to sample comparison. We can treat +Inf, -Inf to be the same as NaN in our fuzz tests . Althought they are not the same, it doesn't really impact the user experience in dashboards. So we can ignore the slight difference here.

Example failed test https://github.com/cortexproject/cortex/actions/runs/20012089043/job/57389986650?pr=7142

02:34:24 Ports for container: e2e-cortex-test-cortex-2 Mapping: map[80:32836 9095:32837]
    query_fuzz_test.go:1795: case 1231 results mismatch.
        range query: avg by (__name__, series, status_code) (
          avg without (series, job, status_code) (
            quantile by (__name__) (
              scalar({__name__="test_series_a",series="1"} offset 3m46s),
              (
                  {__name__="test_series_b",series!~".*5"} offset 4m17s
                -
                  {__name__="test_series_b",status_code=~"4.*"} offset 3m25s
              )
            )
          )
        )
        res1 len: 1 data: {} =>
        +Inf @[1765154365.501]
        +Inf @[1765154425.501]
        +Inf @[1765154485.501]
        +Inf @[1765154545.501]
        +Inf @[1765154605.501]
        +Inf @[1765154665.501]
        +Inf @[1765154725.501]
        +Inf @[1765154785.501]
        +Inf @[1765154845.501]
        +Inf @[1765154905.501]
        +Inf @[1765154965.501]
        +Inf @[1765155025.501]
        +Inf @[1765155085.501]
        +Inf @[1765155145.501]
        +Inf @[1765155205.501]
        +Inf @[1765155265.501]
        +Inf @[1765155325.501]
        +Inf @[1765155385.501]
        +Inf @[1765155445.501]
        +Inf @[1765155505.501]
        +Inf @[1765155565.501]
        +Inf @[1765155625.501]
        +Inf @[1765155685.501]
        +Inf @[1765155745.501]
        +Inf @[1765155805.501]
        +Inf @[1765155865.501]
        +Inf @[1765155925.501]
        +Inf @[1765155985.501]
        +Inf @[1765156045.501]
        +Inf @[1765156105.501]
        +Inf @[1765156165.501]
        +Inf @[1765156225.501]
        +Inf @[1765156285.501]
        +Inf @[1765156345.501]
        +Inf @[1765156405.501]
        +Inf @[1765156465.501]
        +Inf @[1765156525.501]
        +Inf @[1765156585.501]
        +Inf @[1765156645.501]
        +Inf @[1765156705.501]
        +Inf @[1765156765.501]
        +Inf @[1765156825.501]
        +Inf @[1765156885.501]
        +Inf @[1765156945.501]
        +Inf @[1765157005.501]
        +Inf @[1765157065.501]
        +Inf @[1765157125.501]
        +Inf @[1765157185.501]
        +Inf @[1765157245.501]
        +Inf @[1765157305.501]
        +Inf @[1765157365.501]
        +Inf @[1765157425.501]
        +Inf @[1765157485.501]
        +Inf @[1765157545.501]
        +Inf @[1765157605.501]
        +Inf @[1765157665.501]
        res2 len: 1 data: {} =>
        NaN @[1765154365.501]
        NaN @[1765154425.501]
        NaN @[1765154485.501]
        NaN @[1765154545.501]
        NaN @[1765154605.501]
        NaN @[1765154665.501]
        NaN @[1765154725.501]
        NaN @[1765154785.501]
        NaN @[1765154845.501]
        NaN @[1765154905.501]
        NaN @[1765154965.501]
        NaN @[1765155025.501]
        NaN @[1765155085.501]
        NaN @[1765155145.501]
        NaN @[1765155205.501]
        NaN @[1765155265.501]
        NaN @[1765155325.501]
        NaN @[1765155385.501]
        NaN @[1765155445.501]
        NaN @[1765155505.501]
        NaN @[1765155565.501]
        NaN @[1765155625.501]
        NaN @[1765155685.501]
        NaN @[1765155745.501]
        NaN @[1765155805.501]
        NaN @[1765155865.501]
        NaN @[1765155925.501]
        NaN @[1765155985.501]
        NaN @[1765156045.501]
        NaN @[1765156105.501]
        NaN @[1765156165.501]
        NaN @[1765156225.501]
        NaN @[1765156285.501]
        NaN @[1765156345.501]
        NaN @[1765156405.501]
        NaN @[1765156465.501]
        NaN @[1765156525.501]
        NaN @[1765156585.501]
        NaN @[1765156645.501]
        NaN @[1765156705.501]
        NaN @[1765156765.501]
        NaN @[1765156825.501]
        NaN @[1765156885.501]
        NaN @[1765156945.501]
        NaN @[1765157005.501]
        NaN @[1765157065.501]
        NaN @[1765157125.501]
        NaN @[1765157185.501]
        NaN @[1765157245.501]
        NaN @[1765157305.501]
        NaN @[1765157365.501]
        NaN @[1765157425.501]
        NaN @[1765157485.501]
        NaN @[1765157545.501]
        NaN @[1765157605.501]
        NaN @[1765157665.501]
    query_fuzz_test.go:1800: 
        	Error Trace:	/home/runner/work/cortex/cortex/integration/query_fuzz_test.go:1800
        	            				/home/runner/work/cortex/cortex/integration/query_fuzz_test.go:1583
        	Error:      	finished query fuzzing tests
        	Test:       	TestBackwardCompatibilityQueryFuzz
        	Messages:   	1 test cases failed

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: yeya24 <benye@amazon.com>
Copy link
Member

@SungJin1212 SungJin1212 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 8, 2025
@yeya24 yeya24 merged commit f57a7c9 into cortexproject:master Dec 8, 2025
50 of 52 checks passed
@yeya24 yeya24 deleted the flaky-thanos-engine-tests branch December 8, 2025 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size/S type/flaky-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants