Skip to content

Conversation

@insipx
Copy link
Contributor

@insipx insipx commented Dec 11, 2025

fixes #2924

bunch of tests were failing for d14n ordered streams caused by this

@insipx
Copy link
Contributor Author

insipx commented Dec 11, 2025

This PR is part of a stack of 1 bookmark:

  1. trunk()
  2. insipx/fake-streams ← this PR

Created with jj-stack

@claude
Copy link

claude bot commented Dec 11, 2025

Claude finished @insipx's task —— View job


Code Review - December 11, 2025 19:41 UTC

I've reviewed the PR that introduces fake streams to avoid opening tonic connections when subscribing with empty topics/installations. The implementation is solid overall, but I have one concern about the FakeEmptyStream behavior.


Critical Issue: FakeEmptyStream Semantics

Location: xmtp_api_grpc/src/streams/fake_empty.rs:39-42

The FakeEmptyStream has contradictory FusedStream behavior:

impl<E: Error> FusedStream for FakeEmptyStream<E> {
    fn is_terminated(&self) -> bool {
        true  // ← Claims to be terminated
    }
}

impl<E: Error> Stream for FakeEmptyStream<E> {
    fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
        Poll::Pending  // ← But returns Pending forever
    }
}

Problem: A stream that returns Pending indefinitely should NOT report is_terminated() = true. According to Rust's FusedStream contract, is_terminated() should only return true if the stream has yielded Poll::Ready(None).

Impact: This could cause subtle bugs with stream combinators that check is_terminated() (like futures::select! with select_biased!, FuturesUnordered, etc.). They may incorrectly assume the stream is done and skip polling it.

Recommendation: Either:

  1. Remove the FusedStream impl entirely (safest option)
  2. Make poll_next return Poll::Ready(None) to match the terminated state
  3. Make is_terminated() return false to match the pending behavior

I recommend option 1 (removing FusedStream) since this is a "fake pending" stream and doesn't need to be fused.


Minor Issues

  1. Missing documentation on Client::fake_stream (xmtp_proto/src/traits.rs:109-110)

  2. No direct tests for fake_stream behavior

    • While existing tests like test_stream_all_messages_* will exercise this code path when called with no groups, there are no explicit unit tests verifying that:
      • fake_stream() creates a stream that stays pending
      • Empty topic lists correctly trigger the fake stream path
      • The fake stream behaves correctly with stream combinators
    • Consider adding targeted tests in xmtp_api_d14n/src/queries/d14n/streams.rs

Positive Observations

Good performance optimization - Avoiding unnecessary network connections when there are no topics to subscribe to is the right approach

Consistent implementation - The pattern is applied consistently across all middleware layers (auth, multi-node, read-write, readonly)

Proper short-circuiting - All three subscription methods (subscribe_group_messages, subscribe_group_messages_with_cursors, subscribe_welcome_messages) correctly check for empty inputs

Clean trait design - Adding fake_stream() to the Client and QueryStream traits maintains good abstraction boundaries


Summary

The PR successfully solves the problem of avoiding tonic stream creation for empty topic lists. The only blocking concern is the FusedStream implementation semantics which could cause subtle bugs. Once that's addressed, this is ready to merge.


@macroscopeapp
Copy link
Contributor

macroscopeapp bot commented Dec 11, 2025

Return a non-networking fake stream and avoid starting tonic streams when SubscribeEnvelopes topics are empty across d14n query paths

Introduce Client::fake_stream and use it in d14n subscription paths to short‑circuit streaming when inputs are empty; update GrpcClient to construct a pending FakeEmptyStream; default SubscribeEnvelopes builder fields to empty topics and None last_seen; and wire fake stream passthroughs through client wrappers.

📍Where to Start

Start with the d14n subscription shortcuts in D14nQueryStream methods in xmtp_api_d14n/src/queries/d14n/streams.rs, then review Client::fake_stream in xmtp_proto/src/traits.rs and its GrpcClient implementation in xmtp_api_grpc/src/grpc_client/client.rs.


Macroscope summarized a618e13.

@insipx insipx force-pushed the insipx/fake-streams branch 2 times, most recently from a288874 to debcf6c Compare December 11, 2025 19:24
@insipx insipx changed the base branch from insipx/paged-stream to main December 11, 2025 19:25
@insipx insipx enabled auto-merge (squash) December 11, 2025 19:25
@insipx insipx force-pushed the insipx/fake-streams branch from debcf6c to a618e13 Compare December 11, 2025 19:39
@insipx insipx moved this to In Progress in Decentralization Dec 11, 2025
@codecov
Copy link

codecov bot commented Dec 11, 2025

Codecov Report

❌ Patch coverage is 59.70149% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.74%. Comparing base (418c978) to head (a618e13).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
xmtp_api_d14n/src/middleware/auth.rs 0.00% 8 Missing ⚠️
xmtp_api_grpc/src/streams/non_blocking_stream.rs 0.00% 7 Missing ⚠️
xmtp_proto/src/traits.rs 33.33% 6 Missing ⚠️
xmtp_api_d14n/src/middleware/readonly_client.rs 0.00% 3 Missing ⚠️
xmtp_api_grpc/src/streams/fake_empty.rs 72.72% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2941      +/-   ##
==========================================
- Coverage   73.78%   73.74%   -0.04%     
==========================================
  Files         401      402       +1     
  Lines       51265    51332      +67     
==========================================
+ Hits        37828    37857      +29     
- Misses      13437    13475      +38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@insipx insipx merged commit ddba3a6 into main Dec 11, 2025
23 checks passed
@insipx insipx deleted the insipx/fake-streams branch December 11, 2025 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dont start a stream connection on empty topic lists in d14n

3 participants