feat: add automatic worker restart via cron job and worker health tracking (#517)

# Add automatic worker restart system for edge functions

This PR implements a comprehensive worker management system that automatically restarts edge functions when they stop running. Key features include:

- Added `ensure_workers()` cron job that monitors and restarts edge functions
- Created `worker_functions` table to track registered edge functions and their health status
- Added `stopped_at` column to workers table for detecting graceful shutdowns
- Integrated `trackWorkerFunction` and `markWorkerStopped` functions into edge worker lifecycle
- Consolidated temporary migrations into a single migration file
- Removed manual `drop function` statements from SQL schema files
- Removed redundant test setup code as worker restart is now automatic

The system works by having edge functions register themselves on startup, then monitoring their health via heartbeats. When a worker stops (either gracefully or by crashing), the cron job automatically restarts it. This improves reliability by ensuring workers stay running without manual intervention.

Author: jumski

.changeset/worker-management.md

@@ -0,0 +1,6 @@
+---
+'@pgflow/core': minor
+'@pgflow/edge-worker': minor
+---
+
+Add automatic worker restart via `ensure_workers()` cron job that keeps edge functions running. Add `worker_functions` table for tracking registered edge functions and their health status. Add `stopped_at` column to workers table for graceful shutdown detection. Integrate `trackWorkerFunction` and `markWorkerStopped` into edge worker lifecycle for automatic registration and shutdown signaling.

pkgs/core/schemas/0057_function_track_worker_function.sql

@@ -1,8 +1,6 @@
 -- Track Worker Function
 -- Registers an edge function for monitoring by ensure_workers() cron
 
-drop function if exists pgflow.track_worker_function(text);
-
 create or replace function pgflow.track_worker_function(
   function_name text
 ) returns void

pkgs/core/schemas/0058_function_mark_worker_stopped.sql

@@ -1,8 +1,6 @@
 -- Mark Worker Stopped
 -- Sets stopped_at timestamp on a worker row for graceful shutdown signaling
 
-drop function if exists pgflow.mark_worker_stopped(uuid);
-
 create or replace function pgflow.mark_worker_stopped(
   worker_id uuid
 ) returns void

pkgs/core/schemas/0059_function_ensure_workers.sql

@@ -2,8 +2,6 @@
 -- Returns which worker functions should be invoked, makes HTTP requests, and updates last_invoked_at
 -- Called by cron job to keep workers running
 
-drop function if exists pgflow.ensure_workers();
-
 create or replace function pgflow.ensure_workers()
 returns table (function_name text, invoked boolean, request_id bigint)
 language sql

pkgs/core/schemas/0060_function_cleanup_ensure_workers_logs.sql

@@ -2,8 +2,6 @@
 -- Cleans up old cron job run details to prevent the table from growing indefinitely.
 -- Note: net._http_response is automatically cleaned by pg_net (6 hour TTL), so we only clean cron logs.
 
-drop function if exists pgflow.cleanup_ensure_workers_logs(integer);
-
 create or replace function pgflow.cleanup_ensure_workers_logs(
   retention_hours integer default 24
 )

pkgs/core/schemas/0061_function_setup_ensure_workers_cron.sql

@@ -1,8 +1,6 @@
 -- Setup Ensure Workers Cron
 -- Sets up cron jobs for worker management: ensure_workers (configurable interval) and cleanup_logs (hourly)
 
-drop function if exists pgflow.setup_ensure_workers_cron(text);
-
 create or replace function pgflow.setup_ensure_workers_cron(
   cron_interval text default '1 second'
 )