Skip to content

Flaky unit test: TestReaderReconnector.test_reconnect_on_repeatable_error (timeout on 3.12) #850

Description

@vgvoleg

Summary

ydb/_topic_reader/topic_reader_asyncio_test.py::TestReaderReconnector::test_reconnect_on_repeatable_error fails intermittently in CI with a timeout. Recurring flake — has been seen on several unrelated PRs.

Failure

ydb._topic_common.test_helpers.WaitConditionError: Bad condition in test
ydb/_topic_common/test_helpers.py:67: WaitConditionError

The test drives ReaderReconnector through one repeatable error (Overloaded) and then a healthy stream, and waits via:

await wait_for_fast(reconnector.wait_message())   # topic_reader_asyncio_test.py:1561

wait_for_fastwait_condition budgets 1s wall / 1000 loop iterations (test_helpers.py:46-67). When reconnector.wait_message() does not resolve within that budget the helper raises WaitConditionError. So this is a timing-sensitive timeout, not an assertion about reconnect behavior.

A teardown side-effect also shows up after the failure (likely secondary, from the aborted coroutine):

PytestUnraisableExceptionWarning: Exception ignored in: <coroutine object Queue.get ...>
RuntimeError: Event loop is closed

Environment / occurrence

Hypothesis / directions to investigate

  • The reconnect path (error stream → recreate → first message) occasionally needs more than the 1s / 1000-iteration budget under CI load, especially on 3.12's event-loop scheduling. Bumping the budget would only mask it.
  • Worth checking whether ReaderReconnector does extra await asyncio.sleep(0) hops on the reconnect path on 3.12, or whether a backoff/retry delay leaks into the loop and eats the budget.
  • Confirm the Event loop is closed teardown warning is purely secondary (GC of the parked wait_forever/Queue.get coroutine) and not contributing to the hang.

Workaround for now

Re-run the job; failure is intermittent and not a regression from the triggering PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions