Skip to content

Commit d3f8973

Browse files
committed
docs(firestore): add spec testing documentation
1 parent d485ae7 commit d3f8973

File tree

3 files changed

+109
-12
lines changed

3 files changed

+109
-12
lines changed

packages/firestore/devdocs/overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,4 +56,5 @@ To navigate the internals of the SDK, use the following guide:
5656
* **[Code Layout](./code-layout.md)**: Maps the architectural components to specific source files and directories.
5757
* **[Build Process](./build.md)**: How to build the artifacts.
5858
* **[Testing](./testing.md)**: How to run unit and integration tests.
59+
* **[Spec Tests](./spec-tests.md)**: Deep dive into the cross-platform JSON test suite.
5960
* **[Contributing](../CONTRIBUTING.md)**: How to run unit and integration tests.
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Spec Tests (Cross-Platform Verification)
2+
3+
Spec Tests are the backbone of the Firestore SDK's correctness strategy. They are a suite of deterministic, data-driven tests that verify the behavior of the **Sync Engine**, **Local Store**, and **Event Manager** without connecting to a real backend.
4+
5+
## The "Sandwich" Architecture
6+
7+
Spec tests operate by mocking the edges of the SDK while running the core logic for real.
8+
9+
1. **Mocked Top (API Layer)**: instead of user code calling `doc.set()`, the test runner injects "User Actions" directly into the Event Manager/Sync Engine.
10+
2. **Real Middle (Core)**: The Sync Engine, Local Store, Query Engine, and Garbage Collector run exactly as they do in production.
11+
3. **Mocked Bottom (Remote Store)**: The network layer (gRPC/REST) is mocked out. The test runner intercepts outgoing writes and injects incoming Watch Stream events (snapshots, acks).
12+
13+
This allows us to simulate complex race conditions—such as a Write Acknowledgment arriving exactly when a Watch Snapshot is processing—that are impossible to reproduce reliably in a real integration test.
14+
15+
## Cross-Platform Consistency
16+
17+
A unique feature of the Firestore SDKs is that they share logic tests across platforms (JavaScript, Android, iOS).
18+
19+
* **Source of Truth**: The tests are written in **TypeScript** within the JavaScript SDK repository.
20+
* **Compilation**: A build script runs the TS tests in a special mode that exports the steps and expectations into **JSON files**.
21+
* **Consumption**: The Android and iOS SDKs ingest these JSON files and run them using their own platform-specific Test Runners.
22+
23+
This ensures that if a query behaves a certain way on the Web, it behaves *exactly* the same on an iPhone or Android device.
24+
25+
## Anatomy of a Spec Test
26+
27+
A spec test consists of a sequence of **Steps**. Each step performs an action or asserts a state.
28+
29+
```typescript
30+
specTest('Local writes are visible immediately', [], () => {
31+
// Step 1: User sets a document
32+
userSets('collection/key', { foo: 'bar' });
33+
34+
// Step 2: Expect an event (Optimistic update)
35+
expectEvents({
36+
acknowledgedDocs: ['collection/key'],
37+
events: [{ type: 'added', doc: 'collection/key' }]
38+
});
39+
40+
// Step 3: Network acknowledges the write
41+
writeAcks(1); // Ack version 1
42+
43+
// Step 4: Watch stream sends the confirmed data
44+
watchSends({ affects: ['collection/key'] }, ...);
45+
});
46+
47+
### Key Helpers
48+
* `userListens(query)`: Simulates a user calling `onSnapshot`.
49+
* `userSets(key, val)`: Simulates a user write.
50+
* `watchSends(snapshot)`: Simulates the backend sending data over the Watch stream.
51+
* `expectEvents(events)`: Asserts that the Event Manager emitted specific snapshots to the user.
52+
53+
## Configuration & Tags
54+
55+
Spec tests can be configured to run only in specific environments using **Tags**.
56+
57+
* `multi-client`: Runs the test in a multi-tab environment (simulating IndexedDB concurrency).
58+
* `eager-gc`: Runs only when Garbage Collection is set to Eager (Memory persistence).
59+
* `durable-persistence`: Runs only when using IndexedDB/SQLite.
60+
* `exclusive`: **Debug Tool**. If you add this tag to a test, the runner will skip all other tests and only run this one. This is critical for debugging because the sheer volume of spec tests makes logs unreadable otherwise.
61+
62+
## Debugging Spec Tests
63+
64+
Debugging spec tests can be challenging because the code you are stepping through is often the *Test Runner* interpreting the JSON, rather than the test logic itself.
65+
66+
1. **Use `exclusive`**: Always isolate the failing test.
67+
2. **Trace the Helper**: If `userListens` fails, set a breakpoint in the `spec_test_runner.ts` implementation of that step to see how it interacts with the Sync Engine.
68+
3. **Check Persistence**: Remember that spec tests usually run twice: once with Memory Persistence and once with IndexedDB. A failure might only happen in one mode.
69+

packages/firestore/devdocs/testing.md

Lines changed: 39 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,43 @@ This document provides a detailed explanation of the Firestore JavaScript SDK te
55
# Tech Stack
66
- karma, mocha, chai
77

8-
# Strategy
9-
- Firebase emulator for local development
10-
- Integration testing with the backend
8+
# Testing Strategy
9+
10+
The Firestore JS SDK employs a three-tiered testing strategy to ensure reliability, correctness, and cross-platform consistency.
11+
12+
## 1. Unit Tests
13+
* **Scope**: Individual classes and functions.
14+
* **Location**: Co-located with source files (e.g., `src/core/query.test.ts`).
15+
* **Purpose**: Validating low-level logic, util functions, and individual component behavior in isolation.
16+
17+
## 2. Spec Tests (The Core Logic)
18+
* **Scope**: The interaction between the Sync Engine, Local Store, and Event Manager.
19+
* **Location**: `src/core/test/spec_test.ts` and `src/specs/*.ts`.
20+
* **Purpose**: Validating the complex state machine of Firestore without the flakiness of a real network. These tests mock the network layer to simulate specific protocol events (e.g., Watch stream updates, write handshakes).
21+
* **Cross-Platform**: These tests are exported as JSON and run by the Android and iOS SDKs to ensure consistent behavior across all platforms.
22+
* **Deep Dive**: See **[Spec Tests](./spec-tests.md)** for details on how to write and debug these.
23+
24+
## 3. Integration Tests
25+
* **Scope**: End-to-End verification against a real Firestore backend (prod or emulator).
26+
* **Location**: `test/integration/`.
27+
* **Purpose**: Verifying that the client protocol actually matches what the real backend server expects.
28+
* **Behavior**: These tests create real writes and listeners. They are slower and subject to network timing, but essential for catching protocol drifts.
29+
30+
## Running Tests
31+
32+
### Unit & Spec Tests
33+
Run via Karma.
34+
```bash
35+
yarn test
36+
```
37+
38+
### Integration Tests
39+
Requires the Firebase Emulator Suite running.
40+
```bash
41+
# Start emulators
42+
yarn emulators:start
43+
44+
# In another terminal, run integration tests
45+
yarn test:integration
46+
```
1147

12-
## Component Testing
13-
* **Query Engine Tests**: We rely on specific spec tests to ensure the Query Engine picks the correct strategy (e.g., verifying that it uses an Index when available rather than scanning).
14-
* **Integration Tests**: Use the Firebase Emulator to verify that "Limbo Resolution" correctly fetches documents when the local cache drifts from the server state.
15-
16-
17-
# Patterns and Practices
18-
19-
20-
# Spec Tests

0 commit comments

Comments
 (0)