You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: packages/firestore/devdocs/garbage-collection.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,15 +35,15 @@ A document is only eligible for collection if it is **Orphaned**. A document is
35
35
## Key Concepts
36
36
37
37
### Sequence Numbers (The Logical Clock)
38
-
To determine "recency," the SDK maintains a global `last_sequence_number` in the `target_globals` table.
38
+
To determine "recency," the SDK maintains a global `lastListenSequenceNumber` in the **Target Globals** store (`targetGlobal`).
39
39
***Tick**: Every transaction (write, query listen, remote update) increments this number.
40
-
***Tagging**: When a Target is actively listened to or updated, its `last_listen_sequence_number` is updated to the current global tick.
40
+
***Tagging**: When a Target is actively listened to or updated, its `lastListenSequenceNumber` is updated to the current global tick.
41
41
***Effect**: Higher numbers = More recently used.
42
42
43
-
### The Reference Map (`target_documents`)
44
-
This table acts as a reference counter linking Documents to Targets.
45
-
***Active Association**: If `target_id: 2` matches `doc_key: A`, a row exists.
46
-
***Sentinel Rows (`target_id: 0`)**: If a document exists in the cache but is not matched by *any* specific target (perhaps previously downloaded, or part of a target that was deleted), it may have a row with `target_id: 0`. This marks the document as present but potentially orphaned.
43
+
### The Reference Map (`targetDocuments`)
44
+
The **Target-Document Index** (`targetDocuments`) acts as a reference counter linking Documents to Targets.
45
+
***Active Association**: If `targetId: 2` matches `documentKey: A`, a row exists.
46
+
***Sentinel Rows (`targetId: 0`)**: If a document exists in the cache but is not matched by *any* specific target (perhaps previously downloaded, or part of a target that was deleted), it may have a row with `targetId: 0`. This marks the document as present but potentially orphaned.
47
47
48
48
## The Collection Algorithm
49
49
@@ -52,11 +52,11 @@ The `LruGarbageCollector` runs periodically (e.g., every few minutes).
52
52
1.**Threshold Check**: It calculates the byte size of the current cache. If `CurrentSize < CacheSizeBytes` (default 100MB), the process aborts.
53
53
2.**Calculate Cutoff**:
54
54
* The GC decides how many items to cull (e.g., 10%).
55
-
* It queries the `target_documents` table, ordered by `sequence_number` ASC.
55
+
* It queries the **Target-Document Index** (`targetDocuments`) table, ordered by `sequenceNumber` ASC.
56
56
* It finds the sequence number at the 10th percentile. This becomes the **Upper Bound**.
57
57
3.**Sweep Targets**:
58
-
* Any Target in the `targets` table with a `last_listen_sequence_number` <= **Upper Bound** is deleted.
58
+
* Any Target in the **Targets** table (`targets`) with a `lastListenSequenceNumber` <= **Upper Bound** is deleted.
59
59
* This removes the "Active" link for any documents associated with that target.
60
60
4.**Sweep Documents**:
61
-
* The GC scans for documents that have *no* rows in `target_documents` (or only sentinel rows) AND have a sequence number <= **Upper Bound**.
62
-
* These "Orphaned" documents are deleted from the `remote_documents` table.
61
+
* The GC scans for documents that have *no* rows in **Target-Document Index** (or only sentinel rows) AND have a sequence number <= **Upper Bound**.
62
+
* These "Orphaned" documents are deleted from the **Remote Document Cache** (`remoteDocuments`).
Copy file name to clipboardExpand all lines: packages/firestore/devdocs/overview.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -58,9 +58,9 @@ To navigate the internals of the SDK, use the following guide:
58
58
***[Write Lifecycle](./write-lifecycle.md)**: How writes work (Mutations, Batches, Overlays).
59
59
60
60
### Subsystem Deep Dives
61
-
***[Persistence Schema](./persistence-schema.md)**: A reference guide for the IndexedDB tables.
62
-
***[Garbage Collection](./garbage-collection.md)**: Details the LRU algorithm, Sequence Numbers, and how the SDK manages cache size.
63
61
***[Query Execution](./query-execution.md)**: Details on the algorithms used by the Local Store to execute queries (Index Scans vs. Full Collection Scans).
62
+
***[Garbage Collection](./garbage-collection.md)**: Details the LRU algorithm, Sequence Numbers, and how the SDK manages cache size.
63
+
***[Persistence Schema](./persistence-schema.md)**: A reference guide for the IndexedDB tables.
64
64
***[Transactions](./transactions.md)**: How the SDK implements Optimistic Concurrency Control, retries, and the online-only write pipeline.
65
65
***[Limbo Resolution](./limbo-resolution.md)**: How the SDK detects and cleans up stale data using Existence Filters and Bloom Filters.
66
66
***[Bundles](./bundles.md)**: How the SDK loads and processes data bundles.
Copy file name to clipboardExpand all lines: packages/firestore/devdocs/persistence-schema.md
+28-22Lines changed: 28 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,57 +6,63 @@ While the Android/iOS SDKs use SQLite, the JS SDK uses IndexedDB Object Stores.
6
6
7
7
## Core Object Stores
8
8
9
-
### `remote_documents`
9
+
### Remote Document Cache
10
10
***Concept**: The client's cache of the backend's "Source of Truth."
11
+
***Implementation**: Stored in `remoteDocuments` (legacy: `remoteDocumentsV14`).
11
12
***Key**: `DocumentKey` (Path to the document).
12
13
***Value**:
13
14
***Document Data**: The serialized Protobuf of the document.
14
15
***ReadTime**: The snapshot version at which this document was read.
15
-
***Note**: This store **never** contains local, unacknowledged writes. It only contains data confirmed by the server. To see what the developer sees, we overlay the `mutation_queue` on top of this.
16
+
***Note**: This store **never** contains local, unacknowledged writes. It only contains data confirmed by the server. To see what the developer sees, we overlay the **Mutation Queue** on top of this.
16
17
17
-
### `mutation_queue`
18
+
### Mutation Queue
18
19
***Concept**: The "Pending Writes" queue. An ordered log of all local writes that have not yet been acknowledged by the server.
19
-
***Key**: `(user_id, batch_id)`. Segregating by User ID ensures that if a user logs out, their pending writes do not leak to the next user.
20
-
***Value**: A serialized `MutationBatch` containing one or more mutations (Set, Patch, Delete).
21
-
***Behavior**: This is the "Single Source of Truth" for local changes. If the app restarts, the SDK replays these mutations to rebuild the Overlays. When the network is available, the `RemoteStore` reads from this queue to send write batches to the backend. Once acknowledged, entries are removed.
20
+
***Implementation**: Split across `mutationQueues` (User Metadata) and `mutations` (Batch Data).
21
+
***Key**: `(userId, batchId)`. Segregating by User ID ensures that if a user logs out, their pending writes do not leak to the next user.
22
+
***Value**: A serialized `DbMutationBatch` containing one or more mutations (Set, Patch, Delete).
23
+
***Behavior**: This is the "Single Source of Truth" for local changes. If the app restarts, the SDK replays these mutations to rebuild the **Document Overlays**. When the network is available, the `RemoteStore` reads from this queue to send write batches to the backend. Once acknowledged, entries are removed.
22
24
23
-
### `document_overlays`
25
+
### Document Overlays
24
26
***Concept**: A cache of the *result* of applying pending mutations.
25
-
***Key**: `(user_id, document_key)`.
27
+
***Implementation**: Stored in `documentOverlays`.
28
+
***Key**: `(userId, documentKey)`.
26
29
***Purpose**: Read Performance. Without this table, the SDK would have to read the Remote Document and re-apply every pending mutation from the queue every time a query ran.
27
30
***Lifecycle**: Created immediately when a user writes. Deleted immediately when the backend acknowledges the write (or rejects it).
28
-
***Priority**: When the `LocalStore` reads a document, it checks this table first. If an entry exists, it takes precedence over `remote_documents`.
31
+
***Priority**: When the `LocalStore` reads a document, it checks this table first. If an entry exists, it takes precedence over **Remote Document Cache**.
29
32
30
-
### `targets`
33
+
### Targets
31
34
***Concept**: Metadata about active and cached queries.
32
-
***Key**: `TargetID` (Internal Integer allocated by `SyncEngine`).
35
+
***Implementation**: Stored in `targets`.
36
+
***Key**: `TargetId` (Internal Integer allocated by `SyncEngine`).
33
37
***Value**:
34
38
***Canonical ID**: A hash string representing the query (filters, sort order). Used for deduplication.
35
39
***Resume Token**: An opaque token from the backend used to resume a stream without re-downloading all data.
36
40
***Last Sequence Number**: Used for Garbage Collection (LRU).
37
41
38
-
### `target_documents` (The Index)
39
-
***Concept**: A reverse index mapping `TargetID` $\leftrightarrow$ `DocumentKey`.
42
+
### Target-Document Index
43
+
***Concept**: A reverse index mapping `TargetId` $\leftrightarrow$ `DocumentKey`.
44
+
***Implementation**: Stored in `targetDocuments`.
40
45
***Purpose**:
41
46
1.**Query Execution**: Quickly identify documents for a query.
42
-
2.**Garbage Collection**: Acts as a reference counter. If a document has entries here with active TargetIDs, it cannot be collected.
43
-
***Sentinel Rows**: A row with `TargetID = 0` indicates the document exists in the cache but may not be attached to any active listener. These are primary candidates for Garbage Collection.
47
+
2.**Garbage Collection**: Acts as a reference counter. If a document has entries here with active `TargetId`s, it cannot be collected.
48
+
***Sentinel Rows**: A row with `TargetId = 0` indicates the document exists in the cache but may not be attached to any active listener. These are primary candidates for Garbage Collection.
44
49
***Maintenance**: This is updated whenever a remote snapshot adds/removes a document from a query view.
45
50
46
51
## Metadata & Garbage Collection Stores
47
52
48
-
### `target_globals`
53
+
### Target Globals
49
54
***Concept**: A singleton store for global system state.
55
+
***Implementation**: Stored in `targetGlobal`.
50
56
***Key**: Fixed singleton key.
51
57
***Value**:
52
-
***`last_sequence_number`**: A global integer counter incremented on every transaction.
53
-
***`target_count`**: Number of targets currently tracked.
58
+
***`lastListenSequenceNumber`**: A global integer counter incremented on every transaction.
59
+
***`targetCount`**: Number of targets currently tracked.
54
60
55
-
### `remote_document_changes` (Ephemeral)
61
+
### Remote Document Changes (Ephemeral)
56
62
***Concept**: A temporary staging area used during `SyncEngine` processing.
57
-
***Purpose**: Used to track read-time updates for documents during a remote event application before they are committed to the main `remote_documents` store.
63
+
***Purpose**: Used to track read-time updates for documents during a remote event application before they are committed to the main **Remote Document Cache**.
58
64
59
65
## Data Relationships
60
66
61
-
1.**The "View"**: To construct a document for the developer, the SDK reads `remote_documents[key]`and applies any mutations found in `mutation_queue` matching that key.
62
-
2.**Garbage Collection**: The `LruGarbageCollector` uses `target_globals.last_sequence_number` and `targets.last_sequence_number` to determine which targets are old and can be evicted. It then uses `target_documents`to find which documents are no longer referenced by *any* target and deletes them from `remote_documents`.
67
+
1.**The "View"**: To construct a document for the developer, the SDK reads **Remote Document Cache**and applies any mutations found in **Mutation Queue** matching that key.
68
+
2.**Garbage Collection**: The `LruGarbageCollector` uses `TargetGlobal.lastListenSequenceNumber` and `Target.lastListenSequenceNumber` to determine which targets are old and can be evicted. It then uses **Target-Document Index**to find which documents are no longer referenced by *any* target and deletes them from **Remote Document Cache**.
0 commit comments