Skip to content

Have mock_uss retry queries with connection reset by peer #1306

@BenjaminPelletier

Description

@BenjaminPelletier

Is your feature request related to a problem? Please describe.
mock_uss was observed to cause a test run to fail because a DELETE call to the DSS had its connection reset by peer; see log_messages reported by mock_uss upon the failure to delete the flight:

item_1: [2025-12-17 18:59:25.868 DEBUG monitoring.mock_uss.scd_injection.routes_injection:log:278] [delete_flight/24:13304d0d-b0ec-4ffc-a7c9-d1ae17154020] Acquiring and deleting flight
item_2: [2025-12-17 18:59:25.874 DEBUG monitoring.mock_uss.scd_injection.routes_injection:log:278] [delete_flight/24:13304d0d-b0ec-4ffc-a7c9-d1ae17154020] deleting operational intent c3eedd02-e4c2-41e7-9225-e241f6500fc6 with OVN B.OGW7Gr6fm9cMATw7d1TR-tuIR8Oi-7TOvl.njsXso_ from DSS
item_3: [2025-12-17 18:59:26.241 WARNING monitoring.monitorlib.fetch:query_and_describe:690] query_and_describe attempt 1 from PID 24 to DELETE /dss/v1/operational_intent_references/c3eedd02-e4c2-41e7-9225-e241f6500fc6/B.OGW7Gr6fm9cMATw7d1TR-tuIR8Oi-7TOvl.njsXso_ failed with non-retryable ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
At File "/app/monitoring/monitorlib/clients/scd.py", line 130, in delete_operational_intent_reference
item_4: [2025-12-17 18:59:26.242 DEBUG monitoring.mock_uss.scd_injection.routes_injection:log:278] [delete_flight/24:13304d0d-b0ec-4ffc-a7c9-d1ae17154020] Unexpected response from remote server while deleting operational intent c3eedd02-e4c2-41e7-9225-e241f6500fc6 with OVN B.OGW7Gr6fm9cMATw7d1TR-tuIR8Oi-7TOvl.njsXso_ from DSS for flight 13304d0d-b0ec-4ffc-a7c9-d1ae17154020: deleteOperationalIntentReference from /dss/v1/operational_intent_references/c3eedd02-e4c2-41e7-9225-e241f6500fc6/B.OGW7Gr6fm9cMATw7d1TR-tuIR8Oi-7TOvl.njsXso_ failed 999:

Describe the solution you'd like
Include ConnectionResetError in the set of conditions that indicate a retryable query. With this change, mock_uss would have retried the DELETE operation, that retry hopefully would have succeeded, and the test hopefully would have not have failed due to the failed interaction between mock_uss and the DSS.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priorityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions