Skip to content

Conversation

@neekolas
Copy link
Contributor

@neekolas neekolas commented Dec 4, 2025

Putting together a draft PR for RC's

insipx and others added 6 commits December 4, 2025 11:13
- moves the `VectorClock` trait & implementation into `xmtp_proto`.
debating whether the trait should be removed altogether and methods
simply moved onto `GlobalCursor`, since it doesn't seem like we want
multiple `VectorClock` types anyway. I mostly like the trait for the
separation between more normal `HashMap` operations and formal
`VectorClock` operations.
- added an `lcc` and `gcc` function to `TopicCursor` which computes lcc
& gcc with a iterative fold operation on all VectorClocks in
`TopicCursor`.
smallvec backs a vec first with a fixed-size `Copy`-able Array (in this
case `[u8;33]`) and if it grows larger "spills" the contents onto the
heap.

`Topics` should not grow beyond 33 bytes, since the largest topic
identifier is an `InstallationId` at 32 bytes, so this should reduce
cloning wherever `Topic`s are used

moves Ids into their own files (`GroupId` and `InstallationId`)
<!-- Macroscope's pull request summary starts here -->
<!-- Macroscope will only edit the content between these invisible
markers, and the markers themselves will not be visible in the GitHub
rendered markdown. -->
<!-- If you delete either of the start / end markers from your PR's
description, Macroscope will post its summary as a comment. -->
### Make `Conversations.stream_message_deletions` in
[conversations.rs](https://github.com/xmtp/libxmtp/pull/2890/files#diff-305d8a9df912ee15a450791cf1af33e26798a0a4877f6a4dc457d291914bc126)
async to add on_close support for the message deletion stream
Change the N-API method `Conversations.stream_message_deletions` to
`async`, returning a `Future` that yields `Result<StreamCloser>` after
invoking `RustXmtpClient::stream_message_deletions_with_callback`.

#### 📍Where to Start
Start with the `Conversations::stream_message_deletions` N-API method in
[conversations.rs](https://github.com/xmtp/libxmtp/pull/2890/files#diff-305d8a9df912ee15a450791cf1af33e26798a0a4877f6a4dc457d291914bc126).

----
<!-- Macroscope's review summary starts here -->

<a href="https://app.macroscope.com">Macroscope</a> summarized abda21c.
<!-- Macroscope's review summary ends here -->

<!-- macroscope-ui-refresh -->
<!-- Macroscope's pull request summary ends here -->
Dev flag was doing nothing. This fixes it.
@claude
Copy link

claude bot commented Dec 4, 2025

Claude finished @neekolas's task —— View job


Code Review Complete

Review Date: December 4, 2025, 11:14 PM UTC

This PR represents a well-structured refactoring that improves code organization and prepares for future work. The changes are generally sound, but there are a few considerations to note:

Positive Aspects

  1. Good architectural decision: Moving VectorClock to xmtp_proto improves layer separation and makes the trait available across the codebase without circular dependencies.

  2. Performance optimization: The SmallVec<[u8; 33]> change for Topic is smart - it avoids heap allocations for the common case (33 bytes fits inline), while supporting larger topics if needed. This is a good use of stack optimization.

  3. ID module organization: Splitting InstallationId and GroupId into separate files (xmtp_proto/src/types/ids/installation_id.rs:201, xmtp_proto/src/types/ids/group_id.rs:115) improves maintainability.

  4. Test coverage: The PR includes new test coverage in xmtp_mls/src/subscriptions/stream_all/tests.rs:41 for streaming functionality with changing group lists.

Concerns & Recommendations

1. Potential Performance Issue with cloned_vec()

The new Topic::cloned_vec() method (xmtp_proto/src/types/topic.rs:121-123) clones the SmallVec to create a Vec. This is called in hot paths:

  • xmtp_api_d14n/src/queries/d14n/identity.rs:80
  • xmtp_api_d14n/src/endpoints/d14n/subscribe_envelopes.rs:39
pub fn cloned_vec(&self) -> Vec<u8> {
    self.inner.clone().to_vec()  // Double copy: clone SmallVec, then convert to Vec
}

Recommendation: Consider adding pub fn as_vec(&self) -> &[u8] or implementing AsRef<Vec<u8>> to avoid the clone when possible. The gRPC calls likely need owned Vec<u8>, but it's worth verifying if zero-copy approaches are viable.

2. Missing Documentation

The lcc() method in xmtp_proto/src/types/topic_cursor.rs:43-48 computes the "Lowest Common Cursor" but this critical streaming logic could use more explanation about why the minimum (not maximum) is used and the implications for message ordering.

3. Test Coverage Gap

While there's a test for dominates_empty() (xmtp_proto/src/types/global_cursor.rs:329), the VectorClock implementation lacks comprehensive tests for:

  • compare() edge cases (especially Concurrent ordering)
  • missing() correctness
  • merge_least() behavior

4. Topic Byte Layout Assumption

The code in xmtp_proto/src/types/topic.rs:109-112 uses an expect():

pub fn kind(&self) -> TopicKind {
    self.inner[0]
        .try_into()
        .expect("A topic must always be built with a valid `TopicKind`")
}

This is safe given the constructors, but could panic if Topic::from_bytes() (test-only) is misused. Consider adding a debug assertion in the test-only constructor.

5. Minor Code Quality

In xmtp_proto/src/types/global_cursor.rs:52, there's a typo in the doc comment:

/// get the full [`super::Cursor`] that belongs to this [`OriginatorId``

Missing closing backtick.

Security & Correctness

No security issues identified. The VectorClock implementation appears correct for causal ordering, and the SmallVec optimization maintains the same byte semantics.

Summary

This is solid refactoring work. The main concern is the performance impact of cloned_vec() in streaming paths - consider profiling or benchmarking if this becomes a bottleneck. The architectural improvements outweigh the minor issues noted above.


@macroscopeapp
Copy link
Contributor

macroscopeapp bot commented Dec 4, 2025

Release v1.6.6 and move VectorClock to xmtp_proto while updating topic storage to SmallVec<[u8; 33]> across API and MLS streams

Bumps versions to 1.6.6-rc1, relocates VectorClock into xmtp_proto and implements it for GlobalCursor, switches topic bytes to SmallVec<[u8; 33]> with new Topic constructors and cloned_vec(), updates d14n endpoints and queries to the new topic APIs, seeds MLS stream initialization with persisted cursors, and adds TopicCursor::lcc/gcc.

📍Where to Start

Start with the VectorClock trait and its GlobalCursor implementation in xmtp_proto/src/traits/vector_clock.rs and xmtp_proto/src/types/global_cursor.rs, then review topic storage changes in xmtp_proto/src/types/topic.rs and callsite updates in xmtp_mls/src/subscriptions/stream_messages.rs and d14n endpoints.


Macroscope summarized b2d6897.

@neekolas neekolas marked this pull request as ready for review December 5, 2025 22:36
@neekolas neekolas requested a review from a team as a code owner December 5, 2025 22:36
@neekolas neekolas enabled auto-merge (squash) December 5, 2025 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants