Skip to content

Conversation

@pyramation
Copy link
Collaborator

@pyramation pyramation commented Dec 31, 2025

feat(plpgsql-deparser): add hydratePlpgsqlAst for parsing embedded SQL expressions

Summary

Adds a new hydratePlpgsqlAst() function that walks PL/pgSQL ASTs and parses the embedded SQL expression strings (PLpgSQL_expr.query) into full SQL AST nodes. This enables downstream tooling to work with structured SQL ASTs instead of opaque strings.

The implementation uses a discriminated union type system with four expression kinds:

  • raw - fallback for unparseable expressions
  • sql-expr - expressions parsed via SELECT <expr> wrapper (parseMode 2)
  • sql-stmt - full SQL statements (parseMode 0)
  • assign - PL/pgSQL assignments split into target/value with parsed ASTs (parseMode 3)

Key features:

  • Uses the tokenizer (scanSync) to safely split assignments on := while respecting nesting
  • Best-effort parsing with error capture - unparseable expressions fall back to raw kind
  • Returns hydration stats for debugging (total/parsed/failed counts)
  • Utility functions: isHydratedExpr() type guard and getOriginalQuery() helper

Updates since last revision

  • Deparser pretty printing for CREATE FUNCTION - Added context.isPretty() checks in CreateFunctionStmt handler to format function parameters and RETURNS TABLE columns on separate lines in pretty mode. This is a change to the core pgsql-deparser package.

  • Workspace dependency - plpgsql-deparser now uses workspace:* for pgsql-deparser dependency to ensure it uses the local workspace version during development.

  • Creative lowercase naming - Demo uses realistic name transformations instead of _MODIFIED suffixes:

    • Function: big_kitchen_sinkorder_rollup_calculator
    • Variables: v_discountv_rebate, v_taxv_levy
    • Default values: 042
  • Full CREATE FUNCTION support - The demo test parses the full SQL statement and outputs the complete CREATE OR REPLACE FUNCTION declaration including RETURNS TABLE clause, parameters, and language specification.

  • dehydratePlpgsqlAst() - Converts hydrated AST back to canonical string format for deparsing. For assign kind, reconstructs the string from target and value fields, enabling AST-level modifications to be reflected in the output.

Review & Testing Checklist for Human

  • Verify deparser pretty printing behavior - The CreateFunctionStmt changes affect ALL functions in pretty mode. Review the updated snapshots in packages/deparser/__tests__/pretty/__snapshots__/ to ensure the new multiline format is acceptable.
  • Check workspace dependency for publishing - The workspace:* dependency in plpgsql-deparser/package.json should be transformed to a version number during publish. Verify this works correctly with your publish workflow.
  • Review PL/pgSQL body formatting - The snapshot shows formatting quirks (e.g., END IF; placement, inconsistent whitespace). The PL/pgSQL body formatting is not as clean as the SQL wrapper.
  • Verify partial renaming is intentional - Only the first occurrences of v_discount and v_tax assignments are renamed. Other references still use original names.

Recommended test plan:

  1. Run npm test in packages/deparser - verify 7 pre-existing failures, no new failures
  2. Run npm test in packages/plpgsql-deparser - all 23 tests should pass with 9 snapshots
  3. Review the snapshot files to verify the pretty-printed CREATE FUNCTION output looks correct

Notes

  • This is a standalone utility - it doesn't modify the deparser behavior yet
  • The hydrated AST is a separate tree from the canonical AST (preserves original for round-trip)
  • Dehydration is required before deparsing since the deparser expects string queries
  • Requested by Dan Lynch (@pyramation)
  • Link to Devin run: https://app.devin.ai/sessions/50192c7bb0454b26845853867bd795ce

…L expressions

- Add hydrate-types.ts with discriminated union types for hydrated expressions:
  - HydratedExprRaw: fallback for unparseable expressions
  - HydratedExprSqlExpr: for SQL expressions parsed via SELECT wrapper
  - HydratedExprSqlStmt: for full SQL statements
  - HydratedExprAssign: for PL/pgSQL assignments with parsed target/value

- Add hydrate.ts with hydratePlpgsqlAst() function that:
  - Walks the PL/pgSQL AST and parses each PLpgSQL_expr.query string
  - Handles parseMode 2 (expressions) by wrapping with SELECT
  - Handles parseMode 3 (assignments) by splitting on := using tokenizer
  - Returns enriched AST with parsed SQL nodes where possible
  - Tracks hydration stats and errors for debugging

- Add utility functions:
  - isHydratedExpr(): type guard for hydrated expressions
  - getOriginalQuery(): extract original query string from hydrated or raw

- Export new types and functions from package index

- Add comprehensive tests for hydration functionality
@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- Add dehydratePlpgsqlAst() function that converts hydrated AST back to string format
- For 'assign' kind, reconstructs string from target/value fields (enables AST-level modifications)
- For other kinds, uses original string

- Add hydrate-demo.test.ts demonstrating full pipeline with big-function.sql:
  - Parse: 68 expressions parsed (20 assignments, 48 SQL expressions, 0 failures)
  - Hydrate: Convert query strings to structured objects
  - Modify: Change assignment targets (v_discount -> v_discount_MODIFIED, v_tax -> v_tax_MODIFIED)
           and default values (0 -> 888)
  - Dehydrate: Convert back to string format
  - Deparse: Output modified PL/pgSQL code

This proves the full parse -> hydrate -> modify -> dehydrate -> deparse pipeline works.
…ydrate-demo

- Remove all console.log statements from hydrate-demo test
- Add snapshot test for modified big-function.sql output
- Verify exact hydration stats (68 total, 20 assignments, 48 SQL expressions)
- Parse full SQL statement to get CREATE FUNCTION wrapper
- Modify function name: big_kitchen_sink -> big_kitchen_sink_MODIFIED
- Include RETURNS TABLE clause in snapshot
- Demonstrate full pipeline: SQL parse -> PL/pgSQL hydrate -> modify -> dehydrate -> SQL deparse
…ames

- Enable pretty print option for SQL deparser
- Rename function: big_kitchen_sink -> order_rollup_calculator
- Rename variables: v_discount -> v_rebate, v_tax -> v_levy
- Change default values: 0 -> 42
…TURNS TABLE

- Add context.isPretty() check in CreateFunctionStmt handler
- Format function parameters on separate lines in pretty mode
- Format RETURNS TABLE columns on separate lines in pretty mode
- Update plpgsql-deparser to use workspace pgsql-deparser dependency
- Update snapshots to reflect new pretty printing format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants