feat: Milestone 5 - Scoring & Reputation #163
+12,878
−252
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| name: db-tests | ||
| on: | ||
| # Manual trigger remains available | ||
| workflow_dispatch: | ||
| # Run weekly for drift detection | ||
| schedule: | ||
| - cron: '0 6 * * 1' # weekly Monday 06:00 UTC | ||
| # Run automatically on PRs when DB‑related files change | ||
| pull_request: | ||
| paths: | ||
| - 'db/**' | ||
| - 'server/rpc.js' | ||
| - 'server/journal.js' | ||
| - 'server/watcher.js' | ||
| - 'server/schemas.js' | ||
| - 'bin/db8.js' | ||
| - 'server/test/**' | ||
| types: [opened, synchronize, reopened] | ||
| jobs: | ||
| db-integration: | ||
| # Avoid duplicate concurrent runs for the same PR/branch | ||
| concurrency: | ||
| group: db-tests-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: true | ||
| runs-on: ubuntu-latest | ||
| services: | ||
| postgres: | ||
| image: postgres:16 | ||
| ports: ['5432:5432'] | ||
| env: | ||
| POSTGRES_PASSWORD: test | ||
| POSTGRES_USER: postgres | ||
| POSTGRES_DB: postgres | ||
| # NOTE: Do not include shell line-continuation backslashes here; the | ||
| # options string is passed directly to `docker create`. | ||
| options: >- | ||
| --health-cmd="pg_isready -U postgres" | ||
| --health-interval=10s | ||
| --health-timeout=5s | ||
| --health-retries=5 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '20' | ||
| - run: npm ci | ||
| - name: Install web dependencies | ||
| run: npm ci --prefix web | ||
| - name: Create test database | ||
| env: | ||
| PGPASSWORD: test | ||
| run: | | ||
| until pg_isready -h localhost -p 5432 -U postgres; do sleep 2; done | ||
| psql -h localhost -U postgres -c 'CREATE DATABASE db8_test;' | ||
| - name: Prepare schema/RPC/RLS | ||
| env: | ||
| DATABASE_URL: postgresql://postgres:test@localhost:5432/db8_test | ||
| DB8_TEST_OUTPUT: quiet | ||
| run: node scripts/prepare-db.js | ||
|
Comment on lines
+49
to
+59
There was a problem hiding this comment. 🧹 Nitpick | 🔵 Trivial Hardcoded test credentials are acceptable for ephemeral CI containers. The Checkov warning (CKV_SECRET_4) on lines 57-58 is a false positive. These credentials exist solely for a throwaway Postgres container that lives only during the CI run. The However, if you want to silence the static analysis noise and follow the principle of least surprise, you could reference 🔎 Optional: DRY the password reference - name: Prepare schema/RPC/RLS
env:
- DATABASE_URL: postgresql://postgres:test@localhost:5432/db8_test
+ DATABASE_URL: postgresql://postgres:${{ env.PGPASSWORD }}@localhost:5432/db8_test
DB8_TEST_OUTPUT: quiet
run: node scripts/prepare-db.jsNote: This would require setting
🧰 Tools🪛 Checkov (3.2.334)[medium] 57-58: Basic Auth Credentials (CKV_SECRET_4) 🤖 Prompt for AI Agents |
||
| - name: Run lint (server/db only) | ||
| run: | | ||
| # Limit lint to non-web paths for this DB-focused job to avoid resolver noise | ||
| npx eslint server bin scripts eslint.config.js vitest.config.js | ||
| - name: Run DB-gated tests (glob) | ||
| env: | ||
| DB8_TEST_PG: '1' | ||
| DB8_TEST_DATABASE_URL: postgresql://postgres:test@localhost:5432/db8_test | ||
| run: | | ||
| npx vitest run "server/test/**/*.db.*.js" server/test/journal.byidx.test.js --reporter verbose | ||
|
Comment on lines
+64
to
+69
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Explore server/test directory structure and file naming
echo "=== Directory structure of server/test ==="
find server/test -type f -name "*.js" 2>/dev/null | sort | head -30
echo ""
echo "=== Files matching *.db.*.js pattern ==="
find server/test -type f -name "*.db.*.js" 2>/dev/null | sort
echo ""
echo "=== Checking for journal.byidx.test.js ==="
find server/test -type f -name "*journal.byidx*" 2>/dev/null
echo ""
echo "=== All .test.js files in server/test ==="
find server/test -type f -name "*.test.js" 2>/dev/null | sort | head -30Repository: flyingrobots/db8 Length of output: 2657 Fix the naming convention violation or justify the explicit inclusion. The glob 🤖 Prompt for AI Agents |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,3 +6,4 @@ web/.next/ | |
|
|
||
| # generated assets or external | ||
| public/ | ||
| docs/debates/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| --- | ||
| lastUpdated: 2025-10-08 | ||
| --- | ||
|
|
||
| # AGENTS.md | ||
|
|
@@ -147,6 +147,66 @@ Working style | |
| the worker. | ||
| - Deterministic behavior: prefer stable hashing, canonical JSON, advisory locks. | ||
|
|
||
| Neo4j Shared Memory (Context & Notes) | ||
|
|
||
| When to use (simple rules) | ||
|
|
||
| - At session start: query memory for James’s profile/interests and active topics. | ||
| - On topic switch: append a short “insight” with what changed and why. | ||
| - After major events: PRs opened/merged, CI status changes, architectural decisions. | ||
| - Before answering complex or longitudinal questions: skim recent links around “James” to maintain continuity. | ||
|
|
||
| How to use (quick commands) | ||
|
|
||
| - Connection (local dev): | ||
| - Host: <http://localhost:7474> | ||
| - User/Pass: neo4j / password123 (override via env if available) | ||
| - DB: neo4j (default) | ||
|
|
||
| - Read (curl examples): | ||
|
|
||
| ```bash | ||
| # Interests | ||
| curl -s -u neo4j:password123 -H 'Content-Type: application/json' \ | ||
| -X POST http://localhost:7474/db/neo4j/query/v2 \ | ||
| -d '{"statement":"MATCH (j:User {name: \"James\"})-[:INTERESTED_IN]->(i) RETURN i.name,i.category"}' | ||
|
|
||
| # Active topics | ||
| curl -s -u neo4j:password123 -H 'Content-Type: application/json' \ | ||
| -X POST http://localhost:7474/db/neo4j/query/v2 \ | ||
| -d '{"statement":"MATCH (t:Topic {status: \"active\"}) RETURN t.name,t.description"}' | ||
|
|
||
| # Local context around James | ||
| curl -s -u neo4j:password123 -H 'Content-Type: application/json' \ | ||
| -X POST http://localhost:7474/db/neo4j/query/v2 \ | ||
| -d '{"statement":"MATCH (n)-[r]-(m) WHERE n.name=\"James\" OR m.name=\"James\" RETURN n.name,type(r),m.name LIMIT 10"}' | ||
| ``` | ||
|
|
||
| - Write (append an insight): | ||
|
|
||
| ```bash | ||
| INSIGHT='Short insight about the session (what changed / decisions / PR links)' | ||
| curl -s -u neo4j:password123 -H 'Content-Type: application/json' \ | ||
| -X POST http://localhost:7474/db/neo4j/query/v2 \ | ||
| -d "{\"statement\": \"MATCH (j:User {name: \\\"James\\\"}) CREATE (x:Insight {content: \\\"${INSIGHT//\"/\\\\\"}\\\", added_by: \\\"Codex\\\", confidence: 0.9, timestamp: datetime()}) CREATE (j)-[:HAS_INSIGHT]->(x) RETURN x\"}" | ||
| ``` | ||
|
|
||
| - Tip: JSONL flow (bulk): write one JSON object per line to /tmp and POST; or prefer the agent-collab CLI in `/Users/james/git/agent-collab/` for cleaner UX. | ||
|
|
||
| Private session notes (~/Codex) | ||
|
|
||
| - Also keep a parallel Markdown note per session/day: | ||
| - Path: `~/Codex/YYYY-MM-DD-<topic>.md` | ||
| - Frontmatter: `lastUpdated: YYYY-MM-DD` (ISO date only) | ||
| - Include: summary, links (Issues/PRs), CI status, Mermaid diagrams for flows, and “Next”. | ||
|
|
||
| Style & guardrails | ||
|
|
||
| - Keep insights short and factual; no sensitive tokens. | ||
| - Prefer links to Issues/PRs/Commits for traceability. | ||
| - Use Mermaid/SVG in ~/Codex notes for visual learners. | ||
| - This memory is additive: never delete; append new context as it evolves. | ||
|
|
||
| Guardrails (enforced by repo config) | ||
|
|
||
| - Node 20+. See .nvmrc. | ||
|
|
@@ -960,3 +1020,56 @@ On each change: bump docs `lastUpdated`, update Agent Log, and sync the Project | |
| - [M6: Research Tools](https://github.com/flyingrobots/db8/milestone/7) | ||
| - [M7: Hardening & Ops](https://github.com/flyingrobots/db8/milestone/8) | ||
| - [M2: Provenance](https://github.com/flyingrobots/db8/milestone/16) | ||
|
|
||
| --- | ||
|
|
||
| ### Event — 2025-10-07 | M2 closed, README roadmap, DB tests workflow | ||
|
|
||
| #### Summary | ||
|
|
||
| - Closed both M2 milestones and verified acceptance with green tests. Added CLI journal verify tests, corrected error labels, cleaned temp ignores, and hardened SSH parsing. Rewrote README with a weighted milestone progress bar and added milestone focus descriptions. Introduced a manual/weekly GitHub Actions workflow to run DB‑gated integration suites; ensured lint runs before tests. | ||
|
|
||
| #### References | ||
|
|
||
| - Issues: closed/moved — #67, #68, #70, #30, #117, #121, #9, #10 (closed); #11, #12, #29, #7 (→ M3); #31, #15 (→ M6); #32, #13, #14 (→ M7) | ||
| - PRs: #144 (CLI SSH verify + docs), #145/#146/#142 (deps alignment), #148 (db‑tests workflow + README milestone focus) | ||
| - Files: `server/test/cli.journal.verify.test.js`, `docs/Provenance.md`, `.gitignore`, `server/rpc.js`, `.github/workflows/db-tests.yml`, `README.md` | ||
|
There was a problem hiding this comment. Trivial: Use "GitHub" with capital H. Static analysis correctly flags this. Fix it. 🔎 One-character fix- - Files: `server/test/cli.journal.verify.test.js`, `docs/Provenance.md`, `.gitignore`, `server/rpc.js`, `.github/workflows/db-tests.yml`, `README.md`
+ - Files: `server/test/cli.journal.verify.test.js`, `docs/Provenance.md`, `.gitignore`, `server/rpc.js`, `.github/workflows/db-tests.yml`, `README.md`
🧰 Tools🪛 LanguageTool[uncategorized] ~1036-~1036: The official name of this software platform is spelled with a capital “H”. (GITHUB) 🤖 Prompt for AI Agents |
||
|
|
||
| #### Key Decisions | ||
|
|
||
| - M2 is done; provenance/journals shipped with tests and docs. | ||
| - Keep DB‑gated suites behind a dedicated workflow (manual + weekly); lint must run first in that job. | ||
| - README carries a simple, weighted progress bar plus a concise “Milestone Focus” section. | ||
| - No force‑push; resolve forward with additive commits. | ||
|
|
||
| #### Action Items | ||
|
|
||
| - Monitor the new db‑tests workflow; stabilize if any flakes appear. | ||
| - Kick off M3 (Verification): open issues, define schema/RPCs, add tests and endpoints (see next plan). | ||
| - Keep board hygiene: set new M3 issues to Status=Todo/Workflow=Todo and link them to the project. | ||
|
|
||
| #### Notes | ||
|
|
||
| - Added `/.tmp*` to `.gitignore` and removed tracked temp files. | ||
| - Corrected docs to use `unsupported_signature_kind`; pinned JCS in SSH tests. | ||
|
|
||
| #### Next Moves (Plan — M3 Verification) | ||
|
|
||
| - Schema/RPC (DB) | ||
| - `verification_verdicts` (id, round_id, submission_id/claim_id, verdict enum, rationale, reporter_id, created_at) + indexes + RLS; secure read views. | ||
| - RPCs: `verify_submit(...)`, `verify_aggregate(...)` with idempotency + bounds. | ||
| - pgTAP invariants for tables/uniques/RLS and RPC contracts. | ||
| - Server/CLI/UI | ||
| - Server endpoints: `POST /rpc/verify.submit`, `GET /verify/summary`. | ||
| - CLI: `db8 verify submit` and `db8 verify summary`. | ||
| - Web: minimal verification view on the room page. | ||
| - Tests/CI | ||
| - Unit tests for endpoints/CLI; DB‑gated integration for RPCs end‑to‑end; keep lint first in all jobs. | ||
| - Docs/Board | ||
| - `docs/Verification.md` guide; README link; track under milestone “M3: Verification”. | ||
| {"date":"2025-10-08","time":"19:14","summary":"Shipped M3 Verification: added verification verdicts across DB/Server/CLI/Web, made pgTAP + Docker DB suite green, and opened a draft PR.","topics":[{"topic":"Verification DB & RLS","what":"Added verification_verdicts table, RLS policies, and views","why":"M3 requires recording per-claim/per-submission verdicts","context":"Existing M1/M2 schema with submissions/votes and RLS groundwork","issue":"Design idempotency and enforce role/membership for reporters","resolution":"Unique on (round,reporter,submission,claim); verify_submit enforces judge/host and round phase","future_work":"Consider richer claim structure and cross-round carryover","time_percent":25},{"topic":"Server & CLI endpoints","what":"POST /rpc/verify.submit, GET /verify/summary; CLI verify submit/summary","why":"Expose verdict write/read paths to clients","context":"Express RPCs with Zod validation and in-memory fallback patterns","issue":"Consistent validation + idempotency and friendly CLI UX","resolution":"Zod schema + RPC upsert; CLI flags validated; helpful errors","future_work":"Add --json rich summary and grouping in CLI","time_percent":20},{"topic":"pgTAP + Docker DB suite","what":"Installed pgTAP, added invariants, fixed tests for portability","why":"Gate DB invariants and RPC contracts in CI and locally","context":"Manual/weekly db-tests workflow; local docker compose on :54329","issue":"RLS tests under superuser; pgtap version differences; missing seeds","resolution":"Used reader role, relrowsecurity checks, seeded rows; corrected plans; all green","future_work":"Promote more DB-gated tests and stabilize timings","time_percent":30},{"topic":"Flags view pre-publish leakage","what":"Adjusted submissions_with_flags_view to restrict to published","why":"Ensure zero flags appear before publish even with base-table access","context":"submission_flags RLS + aggregated view consumed by server/web","issue":"Pre-publish aggregate showed 1 due to join behavior","resolution":"Join flags through submissions/rounds and filter rr.phase='published'","future_work":"Revisit if we add moderator preview paths","time_percent":10},{"topic":"Repo hygiene & PR","what":"Merged origin/main, created branch, opened Draft PR, created Issue","why":"Follow AGENTS.md discipline (issues, milestones, project, draft PRs)","context":"Project 'db8 Roadmap', milestone 'M3: Verification'","issue":"Ensure board fields, labels, and milestone are set","resolution":"Issue #149, Draft PR #150 with labels/milestone; project updated","future_work":"Kick off db-tests workflow and request reviews","time_percent":15}],"key_decisions":["Use judge/host roles for verify_submit and require published/final rounds","Keep verdict visibility reporter-only until publish; aggregate via view","Adopt JSONL debrief entries appended to AGENTS.md","Open Draft PR and track via Project/Milestone before merge"],"action_items":[{"task":"Run GitHub 'db-tests' workflow and attach results to PR #150","owner":"james"},{"task":"Request reviewers and iterate on feedback for PR #150","owner":"james"},{"task":"Enhance UI with per-claim verdict badges in transcript","owner":"james"}]} | ||
|
|
||
|
|
||
| --- | ||
|
|
||
| {"date":"2025-10-08","time":"21:35","summary":"Merged PR #151 feedback to tighten verification UPSERT keys, clean ESLint resolver config, harden room poller abort handling, and refine commit hook guardrails.","topics":[{"topic":"Verification UPSERT","what":"Removed client_nonce from verification_verdicts conflict target","why":"Deduplicate on substantive identifiers and keep nonce as metadata","resolution":"Conflict now keys on (round,reporter,submission,coalesce(claim,'')) while updating verdict/rationale","time_percent":25},{"topic":"ESLint Resolver","what":"Dropped import/core-modules bypass and fixed resolver paths","why":"Ensure import/no-unresolved runs against actual node_modules","resolution":"Expanded node resolver moduleDirectory and reran lint successfully","time_percent":20},{"topic":"Web Verify Poller","what":"Abortable fetch loop prevents setState after unmount","why":"Avoid memory leaks and React warnings during navigation","resolution":"Added AbortController per-iteration and guarded error handling","time_percent":30},{"topic":"Repo Guardrails","what":"Hardened commit-msg hook and in-memory verify summary parsing","why":"Enforce Conventional Commits precisely and skip malformed cache keys","resolution":"Hook now matches merge message patterns; mem aggregation ignores short keys","time_percent":25}],"key_decisions":["Use message pattern to allow auto merge commits instead of MERGE_HEAD bypass","Abort summary polling fetches on cleanup to prevent stale updates"],"action_items":[{"task":"Monitor room verify summary polling after deployment","owner":"james"}]} | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Incomplete merge pattern: tag merges will be incorrectly rejected.
The pattern covers branch, pull request, and remote-tracking branch merges, but omits tag merges. Git generates messages like
Merge tag 'v1.0.0'which will fail the conventional commits check.🔎 Fix to include tag merges
Note:
tags?handles both "Merge tag" (singular) and "Merge tags" (multiple tags).📝 Committable suggestion
🤖 Prompt for AI Agents