Skip to content

Commit a438152

Browse files
cristianoccristianoc
committed
Update ARCHITECTURE.md to reflect current reactive system
- Remove lookup combinator (was deleted) - Add source and Batch delta type - Update scheduler description to accumulate-then-propagate design - Replace ASCII diagram with reference to Mermaid diagram - Add Stats Tracking section documenting -timing output - Update Reactive module description
1 parent 87c6dff commit a438152

File tree

1 file changed

+43
-86
lines changed

1 file changed

+43
-86
lines changed

‎analysis/reanalyze/ARCHITECTURE.md‎

Lines changed: 43 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -151,40 +151,40 @@ The reactive layer (`analysis/reactive/`) provides delta-based incremental updat
151151
| `subscribe` | Register for delta notifications |
152152
| `iter` | Iterate current entries |
153153
| `get` | Lookup by key |
154-
| `delta` | Change notification: `Set (key, value)` or `Remove key` |
154+
| `delta` | Change notification: `Set (k, v)`, `Remove k`, or `Batch [(k, v option); ...]` |
155+
| `source` | Create a mutable source collection with emit function |
155156
| `flatMap` | Transform collection, optionally merge same-key values |
156157
| `join` | Hash join two collections (left join behavior) |
157158
| `union` | Combine two collections, optionally merge same-key values |
158159
| `fixpoint` | Transitive closure: `init + edges → reachable` |
159-
| `lookup` | Single-key subscription |
160160
| `ReactiveFileCollection` | File-backed collection with change detection |
161161

162162
### Glitch-Free Semantics via Topological Scheduling
163163

164-
The reactive system implements **glitch-free propagation** using a topological scheduler. This ensures derived collections always see consistent parent states, similar to SKStore's approach.
164+
The reactive system implements **glitch-free propagation** using an accumulate-then-propagate scheduler. This ensures derived collections always see consistent parent states, similar to SKStore's approach.
165165

166166
**How it works:**
167-
1. Each collection has a `level` (topological order):
168-
- Source collections (e.g., `ReactiveFileCollection`) have `level = 0`
169-
- Derived collections have `level = max(source levels) + 1`
170-
2. When a delta propagates, emissions are **scheduled by level**
171-
3. The scheduler processes all pending updates in level order
172-
4. This ensures: sources complete → level 1 → level 2 → ... → observers
167+
1. Each node has a `level` (topological order):
168+
- Source collections have `level = 0`
169+
- Derived collections have `level = max(parent levels) + 1`
170+
2. Each combinator **accumulates** incoming deltas in pending buffers
171+
3. The scheduler visits dirty nodes in level order and calls `process()`
172+
4. Each node processes **once per wave** with complete input from all parents
173173

174174
**Example ordering:**
175175
```
176-
Files (level 0) → file_data (level 1) → decls (level 2) → live (level 3) → issues (level 4)
176+
file_collection (L0) → file_data (L1) → decls (L2) → live (L14) → dead_decls (L15)
177177
```
178178

179179
When a batch of file changes arrives:
180-
1. All level 1 collections update first
181-
2. Then level 2, etc.
182-
3. A join never sees one parent updated while the other is stale
180+
1. Deltas accumulate in pending buffers (no immediate processing)
181+
2. Scheduler processes level 0, then level 1, etc.
182+
3. A join processes only after **both** parents have updated
183183

184-
The `Reactive.Scheduler` module provides:
185-
- `schedule ~level ~f` - Queue a thunk at a given level
186-
- `is_propagating ()` - Check if in propagation phase
187-
- Automatic propagation when scheduling outside of propagation
184+
The `Reactive.Registry` and `Reactive.Scheduler` modules provide:
185+
- Named nodes with stats tracking (use `-timing` flag to see stats)
186+
- `to_mermaid()` - Generate pipeline diagram (use `-mermaid` flag)
187+
- `print_stats()` - Show per-node timing and delta counts
188188

189189
### Fully Reactive Analysis Pipeline
190190

@@ -235,74 +235,16 @@ Files → file_data → decls, annotations, refs → live (fixpoint) → dead/li
235235
236236
![Reactive Pipeline](diagrams/reactive-pipeline.svg)
237237

238-
```
239-
┌───────────────────────────────────────────────────────────────────────────────────┐
240-
│ REACTIVE ANALYSIS PIPELINE │
241-
│ │
242-
│ ┌──────────┐ │
243-
│ │ .cmt │ │
244-
│ │ files │ │
245-
│ └────┬─────┘ │
246-
│ │ │
247-
│ ▼ │
248-
│ ┌──────────────────────┐ │
249-
│ │ ReactiveFileCollection│ File change detection + caching │
250-
│ │ file_data │ │
251-
│ └────┬─────────────────┘ │
252-
│ │ flatMap │
253-
│ ▼ │
254-
│ ┌──────────────────────┐ │
255-
│ │ ReactiveMerge │ Derives collections from file_data │
256-
│ │ ┌──────┐ ┌────────┐ │ │
257-
│ │ │decls │ │ refs │ │ │
258-
│ │ └──┬───┘ └───┬────┘ │ │
259-
│ │ │ ┌──────┴─────┐ │ │
260-
│ │ │ │annotations │ │ │
261-
│ │ │ └──────┬─────┘ │ │
262-
│ └────┼─────────┼───────┘ │
263-
│ │ │ │
264-
│ │ ▼ │
265-
│ │ ┌─────────────────────┐ │
266-
│ │ │ ReactiveLiveness │ roots + edges → live (fixpoint) │
267-
│ │ │ ┌──────┐ ┌──────┐ │ │
268-
│ │ │ │roots │→│ live │ │ │
269-
│ │ │ └──────┘ └──┬───┘ │ │
270-
│ │ └──────────────┼──────┘ │
271-
│ │ │ │
272-
│ ▼ ▼ │
273-
│ ┌─────────────────────────────────────────────────────────────────────────────┐ │
274-
│ │ ReactiveSolver │ │
275-
│ │ │ │
276-
│ │ decls ──┬──► dead_decls ──┬──► dead_decls_by_file ──► issues_by_file │ │
277-
│ │ │ │ │ │ │ │ │
278-
│ │ live ───┤ │ │ │ ▼ │ │
279-
│ │ │ ▼ │ │ modules_with_reported │ │
280-
│ │ │ dead_modules ─┼────────────┼──────────────┬─────────┘ │ │
281-
│ │ │ ↑ │ │ │ │ │
282-
│ │ └──► live_decls ──┼────────────┘ ▼ │ │
283-
│ │ │ │ dead_module_issues │ │
284-
│ │ │ │ │ │
285-
│ │ annotations ─────┼────────┴──► incorrect_dead_decls │ │
286-
│ │ │ │ │
287-
│ │ value_refs_from ─┴──► refs_by_file (used by issues_by_file for hasRefBelow)│ │
288-
│ │ │ │
289-
│ │ (Optional args: TODO - not yet reactive, ~8-14ms) │ │
290-
│ └─────────────────────────────────────────────────────────────────────────────┘ │
291-
│ │ │
292-
│ ▼ │
293-
│ ┌─────────────────────────────────────────────────────────────────────────────┐ │
294-
│ │ REPORT │ │
295-
│ │ │ │
296-
│ │ collect_issues iterates pre-computed reactive collections: │ │
297-
│ │ - incorrect_dead_decls (~0.04ms) │ │
298-
│ │ - issues_by_file (~0.6ms) │ │
299-
│ │ - dead_module_issues (~0.06ms) │ │
300-
│ │ │ │
301-
│ │ Total dead_code solving: ~0.7ms on cache hit │ │
302-
│ └─────────────────────────────────────────────────────────────────────────────┘ │
303-
│ │
304-
└───────────────────────────────────────────────────────────────────────────────────┘
305-
```
238+
The Mermaid diagram above shows all 44 reactive nodes. Key stages:
239+
240+
1. **File Layer**: `file_collection` → `file_data` → extracted collections
241+
2. **TypeDeps**: `decl_by_path` → interface/implementation refs → `all_type_refs`
242+
3. **ExceptionRefs**: `cross_file` → `resolved_refs` → `resolved_from`
243+
4. **DeclRefs**: Combines value/type refs → `combined` edges
244+
5. **Liveness**: `annotated_roots` + `externally_referenced` → `all_roots` + `edges` → `live` (fixpoint)
245+
6. **Solver**: `decls` + `live` → `dead_decls`/`live_decls` → per-file issues → module issues
246+
247+
Use `-mermaid` flag to generate the current pipeline diagram from code.
306248

307249
### Delta Propagation
308250

@@ -334,7 +276,7 @@ No joins are recomputed, no fixpoints are re-run - the reactive collections are
334276

335277
| Module | Responsibility |
336278
|--------|---------------|
337-
| `Reactive` | Core primitives: `flatMap`, `join`, `union`, `fixpoint`, delta types |
279+
| `Reactive` | Core primitives: `source`, `flatMap`, `join`, `union`, `fixpoint`, `Scheduler`, `Registry` |
338280
| `ReactiveFileCollection` | File-backed collection with change detection |
339281
| `ReactiveAnalysis` | CMT processing with file caching |
340282
| `ReactiveMerge` | Derives decls, annotations, refs from file_data |
@@ -344,6 +286,21 @@ No joins are recomputed, no fixpoints are re-run - the reactive collections are
344286
| `ReactiveLiveness` | Computes live positions via reactive fixpoint |
345287
| `ReactiveSolver` | Computes dead_decls and issues via reactive joins |
346288

289+
### Stats Tracking
290+
291+
Use `-timing` flag to see per-node statistics:
292+
293+
| Stat | Description |
294+
|------|-------------|
295+
| `d_recv` | Deltas received (Set/Remove/Batch messages) |
296+
| `e_recv` | Entries received (after batch expansion) |
297+
| `+in` / `-in` | Adds/removes received from upstream |
298+
| `d_emit` | Deltas emitted downstream |
299+
| `e_emit` | Entries in emitted deltas |
300+
| `+out` / `-out` | Adds/removes emitted (non-zero `-out` indicates churn) |
301+
| `runs` | Times the node's `process()` was called |
302+
| `time_ms` | Cumulative processing time |
303+
347304
---
348305

349306
## Testing

0 commit comments

Comments
 (0)