docs: Add write lifecycle documentation and update schema details

Author: rafikhan

packages/firestore/devdocs/overview.md

@@ -42,6 +42,7 @@ To navigate the internals of the SDK, use the following guide:
 ### Core Concepts
 *   **[Architecture](./architecture.md)**: The high-level block diagram of the system (API -> Core -> Local -> Remote).
 *   **[Query Lifecycle](./query-lifecycle.md)**: The state machine of a query. **Read this** to understand how querying and offline capabilities work.
+*   **[Write Lifecycle](./write-lifecycle.md)**: How writes work (Mutations, Batches, Overlays).
 
 ### Subsystem Deep Dives
 *   **[Persistence Schema](./persistence-schema.md)**: A reference guide for the IndexedDB tables.

packages/firestore/devdocs/persistence-schema.md

@@ -12,16 +12,20 @@ While the Android/iOS SDKs use SQLite, the JS SDK uses IndexedDB Object Stores.
 *   **Value**:
     *   **Document Data**: The serialized Protobuf of the document.
     *   **ReadTime**: The snapshot version at which this document was read.
-*   **Note**: This store **never** contains local, unacknowledged writes. It only contains data confirmed by the server. To see what the developer sees, we overlay the `mutation_queues` on top of this.
+*   **Note**: This store **never** contains local, unacknowledged writes. It only contains data confirmed by the server. To see what the developer sees, we overlay the `mutation_queue` on top of this.
 
-### `mutation_queues`
-*   **Concept**: The "Pending Writes" queue.
-*   **Key**: `BatchID` (Integer, auto-incrementing).
-*   **Grouping**: Queues are partitioned by **UID**. When a developer logs out, the SDK switches to a different queue.
-*   **Value**:
-    *   **Mutation**: The serialized operation (Set, Patch, Delete).
-    *   **Metadata**: Timestamp, offsets.
-*   **Behavior**: When the network is available, the `RemoteStore` reads from this queue to send write batches to the backend. Once acknowledged, entries are removed.
+### `mutation_queue`
+*   **Concept**: The "Pending Writes" queue. An ordered log of all local writes that have not yet been acknowledged by the server.
+*   **Key**: `(user_id, batch_id)`. Segregating by User ID ensures that if a user logs out, their pending writes do not leak to the next user.
+*   **Value**: A serialized `MutationBatch` containing one or more mutations (Set, Patch, Delete).
+*   **Behavior**: This is the "Single Source of Truth" for local changes. If the app restarts, the SDK replays these mutations to rebuild the Overlays. When the network is available, the `RemoteStore` reads from this queue to send write batches to the backend. Once acknowledged, entries are removed.
+
+### `document_overlays`
+*   **Concept**: A cache of the *result* of applying pending mutations.
+*   **Key**: `(user_id, document_key)`.
+*   **Purpose**: Read Performance. Without this table, the SDK would have to read the Remote Document and re-apply every pending mutation from the queue every time a query ran.
+*   **Lifecycle**: Created immediately when a user writes. Deleted immediately when the backend acknowledges the write (or rejects it).
+*   **Priority**: When the `LocalStore` reads a document, it checks this table first. If an entry exists, it takes precedence over `remote_documents`.
 
 ### `targets`
 *   **Concept**: Metadata about active and cached queries.
@@ -54,5 +58,5 @@ While the Android/iOS SDKs use SQLite, the JS SDK uses IndexedDB Object Stores.
 
 ## Data Relationships
 
-1.  **The "View"**: To construct a document for the developer, the SDK reads `remote_documents[key]` and applies any mutations found in `mutation_queues` matching that key.
+1.  **The "View"**: To construct a document for the developer, the SDK reads `remote_documents[key]` and applies any mutations found in `mutation_queue` matching that key.
 2.  **Garbage Collection**: The `LruGarbageCollector` uses `target_globals.last_sequence_number` and `targets.last_sequence_number` to determine which targets are old and can be evicted. It then uses `target_documents` to find which documents are no longer referenced by *any* target and deletes them from `remote_documents`.

packages/firestore/devdocs/query-lifecycle.md

@@ -41,7 +41,7 @@ When a query runs (initially or after a remote update), the **Local Store** perf
     *   *Deep Dive*: For details on Index Scans, Full Scans, and Optimization strategies, see [Query Execution & Indexing](./query-execution.md).
 2.  **Base State**: The store retrieves the confirmed server state from the `remote_documents` table.
 3.  **Overlay Application**:
-    *   The store checks the `mutation_queues` for any pending writes associated with these keys.
+    *   The store checks the `mutation_queue` for any pending writes associated with these keys.
     *   These mutations are converted into **Overlays**.
     *   The Overlay is applied on top of the Remote Document.
 4.  **Projection**: The final composed documents are sent to the Event Manager.
@@ -94,5 +94,5 @@ For a detailed walkthrough of the algorithm, Sequence Numbers, and Orphaned Docu
 If you are debugging a **"Zombie Document"** (data appearing that should be gone) or **"Missing Data"**:
 
 1.  **Check `targets`**: Is there an active target (valid `resumeToken`) covering that document?
-2.  **Check `mutation_queues`**: Is there a pending mutation (BatchID) that hasn't been acknowledged? This creates an Overlay that persists even if the remote doc is deleted.
+2.  **Check `mutation_queue`**: Is there a pending mutation (BatchID) that hasn't been acknowledged? This creates an Overlay that persists even if the remote doc is deleted.
 3.  **Check `target_documents`**: Is the document explicitly linked to a TargetID that you thought was closed?

packages/firestore/devdocs/write-lifecycle.md

@@ -0,0 +1,54 @@
+# Write Lifecycle & Latency Compensation
+
+This document details the lifecycle of a write operation (Set, Update, Delete) from the moment the API is called to when it is committed to the backend. It focuses on **Mutations**, **Overlays**, and how the SDK achieves instant **Latency Compensation**.
+
+## Key Concepts
+
+*   **Mutation**: An operation that modifies a document (e.g., `SetMutation`, `PatchMutation`, `DeleteMutation`).
+*   **Mutation Batch**: A group of mutations that must be applied atomically. Every user write creates a new Batch with a unique `BatchID`.
+*   **Overlay**: A "Materialized View" of the changes applied to a document. Instead of re-calculating the result of a mutation every time a query runs, the SDK calculates the result *once* at write time and saves it as an Overlay.
+
+## Phase 1: Mutation Creation & Batching
+
+When a user calls `setDoc` or `updateDoc`:
+1.  **Validation**: The SDK validates the data locally.
+2.  **Batching**: The operation is wrapped in a `MutationBatch`.
+3.  **Persistence**: The batch is serialized and saved to the `mutation_queue` table in IndexedDB.
+    *   **Partitioning**: Queues are partitioned by User ID. If the user is offline, these batches accumulate in the queue.
+
+## Phase 2: Overlay Calculation (Optimization)
+
+To ensure queries run fast, the SDK does not apply raw mutations to remote documents during every query execution. Instead, it pre-calculates the result.
+
+1.  **Base State**: The SDK retrieves the current state of the document (from `remote_documents`).
+2.  **Apply**: It applies the new `Mutation` to the base state to determine what the document *should* look like locally.
+3.  **Persist Overlay**: This resulting state is saved to the `document_overlays` table.
+    *   **Field Mask**: The overlay tracks specifically which fields were modified.
+4.  **Latency Compensation**: The `Event Manager` immediately triggers active listeners. The listeners read the `Overlay` instead of the `Remote Document`, giving the user the illusion of instant updates.
+
+> **Formula:** `Local View = Remote Document + Overlay`
+
+## Phase 3: Synchronization (The Write Pipeline)
+
+The `SyncEngine` manages the flow of data to the server:
+
+1.  **Filling the Pipeline**: The `RemoteStore` reads the `mutation_queue` in order of `BatchID` (FIFO).
+2.  **Transmission**: Mutations are sent to the backend via gRPC (or REST in Lite).
+3.  **Atomicity**: If a batch contains multiple writes, the backend guarantees they are applied together or not at all.
+
+## Phase 4: Acknowledgement & Cleanup
+
+When the backend responds:
+
+### Scenario A: Success (Ack)
+1.  **Commit**: The backend commits the change and returns the authoritative version of the document (and transformation results, like server timestamps).
+2.  **Update Remote**: The SDK updates the `remote_documents` table with this new server version.
+3.  **Cleanup**:
+    *   The `MutationBatch` is removed from `mutation_queue`.
+    *   The corresponding `Overlay` is removed from `document_overlays` (since the Remote Document now matches the desired state).
+4.  **Re-Evaluation**: Active queries are re-run. Since the Overlay is gone, they now read the updated Remote Document.
+
+### Scenario B: Rejection
+1.  The `MutationBatch` is removed.
+2.  The `Overlay` is removed.
+3.  The Local View reverts to the `Remote Document` state (rolling back the optimistic update).