docs(firestore): Introducing "Watch" as a concept

Author: rafikhan

packages/firestore/devdocs/architecture.md

@@ -26,7 +26,10 @@ The SDK is composed of several key components that work together to provide the
     *   **Query Engine**: Determines the most efficient strategy (Index vs. Scan) to identify documents matching a query in the local cache.
     *   **Overlays**: A performance-optimizing cache that stores the calculated effect of pending mutations from the Mutation Queue on documents. Instead of re-applying mutations every time a document is read, the SDK computes this "overlay" once and caches it, allowing the Local View to be constructed more efficiently.
     * For a detailed breakdown of the IndexedDB structure and tables, see [Persistence Schema](./persistence-schema.md).
-*   **Remote Store**: The component responsible for all network communication with the Firestore backend. It manages the gRPC streams for reading and writing data, and it abstracts away the complexities of the network protocol from the rest of the SDK.
+*   **Remote Store**: The component responsible for all network communication with the Firestore backend.
+    *   It manages the **Watch Stream** (see **[The Watch System](./watch.md)**) for reading and listening to data.
+    *   It manages the gRPC streams for writing data.
+    *   It abstracts away the complexities of the network protocol from the rest of the SDK.
 *   **Persistence Layer**: The underlying storage mechanism used by the Local Store to persist data on the client. In the browser, this is implemented using IndexedDB.
 
 The architecture and systems within the SDK map closely to the directory structure, which helps developers navigate the codebase. Here is a mapping of the core components to their corresponding directories.

packages/firestore/devdocs/query-execution.md

@@ -32,7 +32,12 @@ To support efficient querying without blocking the main thread, the SDK utilizes
 
 ## Composite Queries (OR / IN)
 
-Queries using `OR` or `IN` are not executed as a single monolithic scan. The SDK transforms these into **Disjunctive Normal Form (DNF)**—essentially breaking them into multiple sub-queries.
+Queries using `OR` or `IN` are not executed as a single monolithic scan.
+
+> [!NOTE]
+> **Scalability & Watch**: While functionality exists to run these queries against the backend, the SDK implements **Disjunctive Normal Form (DNF)** transformation primarily to enable efficient **local** execution using simpler indexes (as seen in `IndexedDbIndexManager`). This allows the SDK to support complex queries offline or against the cache without requiring full table scans. [See Watch System](./watch.md) for more on the backend interaction.
+
+The SDK transforms these into **Disjunctive Normal Form (DNF)**—essentially breaking them into multiple sub-queries.
 
 *   **Execution**: Each sub-query is executed independently using the strategies above (Index vs. Scan).
 *   **Union**: The resulting sets of document keys are unioned together in memory to produce the final result.

packages/firestore/devdocs/transactions.md

@@ -62,6 +62,9 @@ Transactions utilize a dedicated pathway in the `RemoteStore`/`Datastore` layer.
     *   Standard writes use `CommitStream` (requires `mutation_queue` persistence).
     *   **Transactions** use direct Unary RPCs (`BatchGetDocuments` and `Commit`) via the underlying `Datastore` helper.
 
+> [!WARNING]
+> **Consistency Warning**: Transactions use a different endpoint (`runTransaction`) than the standard `Listen` (Watch) system. As a result, they **do not** guarantee consistency with the Watch stream. A write committed via Watch might not be immediately visible to a transaction, and vice versa, due to the distributed nature of the backend.
+
 ## Constraints
 
 *   **Online Only**: Because they bypass the local cache and require server-side verification, transactions require an active network connection. They will fail if the client is offline.

packages/firestore/devdocs/watch.md

@@ -0,0 +1,41 @@
+# The Watch System
+
+This document explains the "Watch" backend system, which powers the real-time capabilities and standard read/write operations of the Firestore SDKs.
+
+## Overview
+
+"Watch" is the internal name for the high-scale system that the SDKs interact with. It serves two primary purposes:
+
+1.  **Reads & Writes**: It is the main entry point for standard document operations.
+2.  **Real-Time Listeners**: It powers the `onSnapshot` live updates by tracking database changes and pushing them to clients.
+
+## Architecture: The Reverse Proxy
+
+Watch functions as a massive **Reverse Proxy**.
+
+1.  **Connection**: Massive numbers of end-user devices (phones, browsers) connect directly to Watch.
+2.  **Routing**: Watch takes the incoming query or write and forwards it to the appropriate underlying storage backend (partition).
+3.  **Observation**: For queries, Watch doesn't just fetch data once. It "watches" the backend for any writes that would affect the query results and pushes those changes to the subscribed client.
+
+This architecture allows Firestore to handle millions of concurrent connections while abstracting the complexity of sharding and storage from the client.
+
+## Consistency Guarantees
+
+The Watch system (exposed via the `Listen` endpoint) enables strong consistency between reads and writes.
+
+*   **Consistency**: All reads and writes performed through this system are consistent with each other. If you write a document and then immediately read it (or listen to it) via Watch, you will see the latest version.
+*   **Authentication**: Watch interacts directly with Firebase Auth to identify the user and enforces Firestore Security Rules for every operation.
+
+> [!IMPORTANT]
+> **Transactions** use a different endpoint (`runTransaction`) and **do not** guarantee consistency with the Watch stream. See [Transactions](./transactions.md) for details.
+
+### Scalability and Client-Side Logic
+The Watch system operates at a massive scale. To maintain performance, the backend may rely on the SDK to handle certain query complexities, particularly for **local** execution and consistency.
+
+> [!NOTE]
+> **Composite Queries (OR/IN)**
+> While the Watch system supports complex queries (including `OR` and `IN`), the SDK performs significant client-side logic to support them efficiently **locally**.
+> *   **Local Indexing**: For local cache execution, the SDK transforms composite queries into **Disjunctive Normal Form (DNF)** (breaking them into simpler sub-queries) to utilize simple field indexes.
+> *   **consistency**: The SDK merges results from the Watch stream and local cache to ensure a consistent view.
+
+This architectural decision explains why you see complex logic like **Composite Query** execution in the [Query Engine](./query-execution.md). The SDK implements this logic to bridge the gap between user intent and Watch's scalability constraints.