Skip to content

Conversation

@vcarl
Copy link
Member

@vcarl vcarl commented Dec 12, 2025

There are some drawbacks to the current devops/CI+CD setup:

  • E2E tests run off a service instance that isn't human-interactable
  • We don't have a good way of seeding test data
  • Live deployments result in ~90s of downtime as a new pod stands up

To summarize the changes in this PR briefly:

  • Preview environments - Per-PR previews at .euno-staging.reactiflux.com with automatic cleanup on PR close
  • E2E tests run after deployments - Playwright tests now run against the deployed preview/production URL after each successful deploy, with results posted as a PR comment
  • CI/CD reworked - CI handles lint/typecheck/vitest on PR branches. CD handles build, deploy, and E2E for both preview and production
  • Fixture generation - New scripts/fixtures/ system for seeding staging/local databases with test data
  • Health probes, automatic rollback on failed deploys (this part remains to be seen whether it works)

vcarl and others added 5 commits December 11, 2025 20:24
- Add PodDisruptionBudget to protect against voluntary node disruptions
- Add startup/liveness/readiness probes using /healthcheck endpoint
- Add resource requests and limits (256Mi/512Mi memory, 100m/500m CPU)
- Replace destructive delete/create secret pattern with atomic kubectl apply
- Add rollout status verification with automatic rollback on failure
- Update CONTRIBUTING.md with correct workflow paths and new kubectl commands

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace 80+ lines of individual secretKeyRef entries with a single envFrom
block that injects all keys from modbot-env secret automatically. Adding
new secrets no longer requires editing deployment.yaml.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add preview.yml workflow that deploys on PR open/sync/ready_for_review
- Skip deployment for draft PRs and PRs with 'no-preview' label
- Clean up resources when PR closes or 'no-preview' label added
- Add K8s template for preview Deployment, Service, and Ingress
- Document manual setup steps (DNS, TLS, namespace, secrets)

Previews will be available at https://<pr-number>.euno-staging.reactiflux.com
once manual infrastructure setup is complete.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
vcarl and others added 2 commits December 12, 2025 14:52
- Use CI environment secrets to create modbot-staging-env K8s secret
- Add cert-manager annotation to preview ingress for automatic TLS
- Each PR gets its own cert via HTTP-01 (no wildcard/DNS-01 needed)
- Add DNS-01 solver to cluster-issuer for future use
- Add step to cd.yml to ensure digitalocean-dns-token exists

First PR deploy takes ~1-2 min extra for cert issuance;
subsequent deploys are fast since cert already exists.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Text changed when I added a free trial; it has a `data-test-id` though so we can use that for more robust selection
@github-actions

This comment was marked as outdated.

      Debugging 429 rate limit errors during kubectl setup.

      🤖 Generated with [Claude Code](https://claude.com/claude-code)

      Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions

This comment was marked as outdated.

Switch to new scoped token for kubernetes access.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions

This comment was marked as outdated.

@github-actions
Copy link

vcarl and others added 2 commits December 15, 2025 16:16
Replace 4 separate notes with single consolidated infrastructure-state.md.
Removed outdated issues that were fixed (Dockerfile scripts, tsx removal,
secret naming, hardcoded branch). Kept ci-cd-architecture-review.md as
reference for proposed staging/release model.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Inline E2E job in cd.yml that runs after preview or production deploy
- E2E tests against deployed URL (preview or production)
- Updates single PR comment with test results instead of creating new ones
- Simplified ci.yml to just lint/typecheck/vitest (E2E moved to CD)
- Deploy test reports to GitHub Pages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GitHub runners have Playwright deps pre-installed. The --with-deps flag
was running apt-get and taking 9 minutes. Should be ~seconds without it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vcarl
Copy link
Member Author

vcarl commented Dec 15, 2025

This is messy as heck, but it all works!! Ish. There are still some test isolation issues, i.e. the e2e tests don't reset the test database being used in between runs, but there are enough working changes here that I'd like to get this shipped ✨

Ensures preview environments start fresh on every push by deleting the
sqlite database before restarting the pod.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Should allow for some dead time to be removed from the system
@vcarl vcarl merged commit 31ce1a9 into main Dec 16, 2025
6 checks passed
@vcarl vcarl deleted the vc-devops branch December 16, 2025 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants