Skip to content

Commit 08519bb

Browse files
author
Nicki Nixon
committed
fix(parameters): add missing properties to includes parameter
Add missing name, example, and required properties to includes parameter to fix validation error. Also refactor phase3_sync_parameters.rb to prevent this issue in future runs: - Changed from regex text extraction to YAML parsing - Now syncs ALL properties from source (name, description, in, required, example, schema) - Uses yq commands to add each property individually - Validates required properties after adding This ensures parameters are copied completely from parameters.yaml source, preventing missing property issues that required post-fix patches. Fixes validation error: #/components/parameters/includes must have required property 'name' Root cause: Original phase3 script used regex that didn't capture all properties, especially for parameters with multi-line descriptions. Refs: devx-7466
1 parent 1dd3c59 commit 08519bb

File tree

2 files changed

+326
-0
lines changed

2 files changed

+326
-0
lines changed

openapi/mx_platform_api.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8588,6 +8588,9 @@ components:
85888588
schema:
85898589
type: string
85908590
in: query
8591+
name: includes
8592+
example: repeating_transactions,merchants,classifications,geolocations
8593+
required: false
85918594
holdingGuid:
85928595
description: The unique id for a `holding`.
85938596
example: HOL-d65683e8-9eab-26bb-bcfd-ced159c9abe2

tmp/phase3_sync_parameters.rb

Lines changed: 323 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,323 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
#
4+
# Parameter Synchronization Script
5+
# =================================
6+
# Synchronizes parameters between source files and consolidated OpenAPI spec
7+
#
8+
# USAGE:
9+
# ruby tmp/sync_parameters.rb [params_file] [api_file] [comparison_file]
10+
#
11+
# ARGUMENTS:
12+
# params_file - Source parameters file (default: openapi/parameters.yaml)
13+
# api_file - Target OpenAPI file (default: openapi/mx_platform_api.yml)
14+
# comparison_file - JSON diff file (default: tmp/comparison_diff.json)
15+
#
16+
# EXAMPLES:
17+
# # Current version (uses defaults)
18+
# ruby tmp/sync_parameters.rb
19+
#
20+
# # Future version v20250224
21+
# ruby tmp/sync_parameters.rb \
22+
# openapi/parameters.yaml \
23+
# openapi/mx_platform_api_v20250224.yml \
24+
# tmp/comparison_diff_v20250224.json
25+
#
26+
# PREREQUISITES:
27+
# - comparison_diff.json must exist (run compare_openapi_specs.rb first)
28+
# - Source parameters.yaml must exist
29+
# - Target API file must have 'components:' and 'securitySchemes:' or 'paths:' sections
30+
#
31+
# OUTPUT:
32+
# - Creates components.parameters section if missing (before securitySchemes)
33+
# - Adds missing parameters from comparison
34+
# - Removes extra parameters from comparison
35+
# - Converts inline parameter definitions to $ref
36+
# - Modifies api_file in place
37+
#
38+
# NOTES:
39+
# - Phase 3a: Adds parameters to components.parameters library
40+
# - Phase 3b: Converts ~352 inline parameters to $ref (atomic operation)
41+
# - Typos fixed before Phase 3: records_per_age → records_per_page, microdeposit_guid → micro_deposit_guid
42+
# - Unmatchable parameters from extra paths (e.g., tax_document_guid) removed in Phase 6
43+
# - Net effect: Typically reduces file size by 1000-2000 lines
44+
45+
require 'json'
46+
require 'yaml'
47+
require 'set'
48+
49+
# ============================================================================
50+
# CONFIGURATION
51+
# ============================================================================
52+
53+
comparison_file = ARGV[2] || 'tmp/comparison_diff.json'
54+
params_file = ARGV[0] || 'openapi/parameters.yaml'
55+
api_file = ARGV[1] || 'openapi/mx_platform_api.yml'
56+
57+
# ============================================================================
58+
# LOAD FILES
59+
# ============================================================================
60+
61+
puts "Reading comparison data from: #{comparison_file}"
62+
comparison_data = JSON.parse(File.read(comparison_file))
63+
64+
puts "Loading parameters from: #{params_file}"
65+
params_content = File.read(params_file)
66+
params_source = YAML.unsafe_load_file(params_file) # Also parse for property access
67+
68+
puts "Loading API file: #{api_file}"
69+
api_content = File.read(api_file)
70+
71+
# ============================================================================
72+
# EXTRACT DIFFERENCES
73+
# ============================================================================
74+
75+
missing_params = comparison_data['missing_parameters'] || []
76+
extra_params = comparison_data['extra_parameters_in_mx'] || []
77+
78+
puts "\nFound:"
79+
puts " - #{missing_params.length} parameters to add"
80+
puts " - #{extra_params.length} parameters to remove"
81+
82+
# Track modifications
83+
modifications = {
84+
added: [],
85+
removed: [],
86+
skipped: []
87+
}
88+
89+
# ============================================================================
90+
# PART 1: ADD MISSING PARAMETERS
91+
# ============================================================================
92+
93+
if missing_params.any?
94+
puts "\nAdding #{missing_params.length} missing parameters..."
95+
96+
# Check if parameters section exists
97+
has_params_section = api_content =~ /^ parameters:\s*\n/
98+
99+
# If no parameters section, create it before securitySchemes (or before paths if no securitySchemes)
100+
unless has_params_section
101+
puts " Creating parameters section..."
102+
103+
# Try to find securitySchemes first
104+
security_match = api_content.match(/^ (securitySchemes:)/)
105+
106+
if security_match
107+
insert_pos = security_match.begin(0)
108+
api_content.insert(insert_pos, " parameters:\n")
109+
puts " ✅ Created parameters section before securitySchemes"
110+
else
111+
# Fallback: insert before paths
112+
paths_match = api_content.match(/^(paths:)/)
113+
114+
if paths_match
115+
insert_pos = paths_match.begin(0)
116+
api_content.insert(insert_pos, " parameters:\n")
117+
puts " ✅ Created parameters section before paths"
118+
else
119+
puts " ⚠️ Could not find securitySchemes or paths section"
120+
puts "Aborting."
121+
exit 1
122+
end
123+
end
124+
end
125+
126+
# Now add each parameter
127+
missing_params.each do |param_info|
128+
param_name = param_info['name']
129+
130+
# Load the parameter definition from source using YAML parser
131+
# This ensures we get the complete, parsed structure
132+
unless params_source[param_name]
133+
puts " ⚠️ Could not find parameter definition in parameters.yaml: #{param_name}"
134+
modifications[:skipped] << param_name
135+
next
136+
end
137+
138+
source_param = params_source[param_name]
139+
140+
# Build parameter definition using yq commands for each property
141+
# This ensures all properties are captured, including multi-line descriptions
142+
properties_to_add = ['name', 'description', 'in', 'required', 'example']
143+
144+
puts " Adding parameter: #{param_name}"
145+
146+
# First, create the parameter key in components.parameters
147+
cmd = "yq -i '.components.parameters.#{param_name} = {}' #{api_file}"
148+
system(cmd)
149+
150+
# Add each property from source
151+
properties_to_add.each do |prop|
152+
next unless source_param[prop]
153+
154+
value = source_param[prop]
155+
156+
case value
157+
when String
158+
# Escape for shell - use printf style to handle special chars
159+
escaped_value = value.gsub("'", "'\\''")
160+
cmd = "yq -i '.components.parameters.#{param_name}.#{prop} = \"#{escaped_value}\"' #{api_file}"
161+
system(cmd)
162+
when TrueClass, FalseClass
163+
cmd = "yq -i '.components.parameters.#{param_name}.#{prop} = #{value}' #{api_file}"
164+
system(cmd)
165+
end
166+
end
167+
168+
# Handle schema property (it's an object)
169+
if source_param['schema']
170+
schema = source_param['schema']
171+
172+
if schema['type']
173+
cmd = "yq -i '.components.parameters.#{param_name}.schema.type = \"#{schema['type']}\"' #{api_file}"
174+
system(cmd)
175+
end
176+
177+
# Handle array items
178+
if schema['items'] && schema['items']['type']
179+
cmd = "yq -i '.components.parameters.#{param_name}.schema.items.type = \"#{schema['items']['type']}\"' #{api_file}"
180+
system(cmd)
181+
end
182+
end
183+
184+
# Validate that all required properties were added
185+
required_props = ['in', 'name', 'schema']
186+
missing_props = required_props.select { |prop| !source_param[prop.to_s] }
187+
188+
if missing_props.any?
189+
puts " ⚠️ Parameter #{param_name} is missing required properties in source: #{missing_props.join(', ')}"
190+
end
191+
192+
modifications[:added] << param_name
193+
puts " ✅ Added: #{param_name} with complete definition"
194+
end
195+
end
196+
197+
# ============================================================================
198+
# PART 2: REMOVE EXTRA PARAMETERS
199+
# ============================================================================
200+
201+
if extra_params.any?
202+
puts "\nRemoving #{extra_params.length} extra parameters..."
203+
204+
extra_params.each do |param_info|
205+
param_name = param_info['name']
206+
207+
# Pattern to match parameter in components.parameters section
208+
# Parameters are at 4-space indent, content at 6-space indent
209+
removal_pattern = /^ #{Regexp.escape(param_name)}:\s*\n((?: .+\n)*)/
210+
211+
if api_content.match(removal_pattern)
212+
api_content.gsub!(removal_pattern, '')
213+
modifications[:removed] << param_name
214+
puts " ✅ Removed parameter: #{param_name}"
215+
else
216+
puts " ⚠️ Could not find parameter to remove: #{param_name}"
217+
end
218+
end
219+
end
220+
221+
# ============================================================================
222+
# PART 3: CONVERT INLINE PARAMETERS TO $REF
223+
# ============================================================================
224+
225+
puts "\nConverting inline parameters to $ref..."
226+
227+
# Build a map of parameter name (from 'name:' field) to parameter key
228+
param_name_to_key = {}
229+
230+
# Extract all parameter keys and their 'name:' values from components.parameters
231+
params_section_match = api_content.match(/^ parameters:\s*\n(.*?)^ \w+:/m)
232+
if params_section_match
233+
params_section_content = params_section_match[1]
234+
235+
# Match each parameter block
236+
params_section_content.scan(/^ (\w+):\s*\n((?: .+\n)*)/) do |param_key, param_content|
237+
name_match = param_content.match(/name:\s+(\S+)/)
238+
if name_match
239+
param_name = name_match[1]
240+
param_name_to_key[param_name] = param_key
241+
end
242+
end
243+
end
244+
245+
puts " Found #{param_name_to_key.size} parameters available for conversion"
246+
247+
# Track conversions
248+
conversions = {
249+
converted: Set.new,
250+
not_found: Set.new,
251+
replacements: 0
252+
}
253+
254+
# Use gsub to replace inline parameter blocks with $ref
255+
# Match: " - " followed by properties (not "$ref:")
256+
# More precise: only match lines that are parameter properties (description, in, name, example, required, schema)
257+
api_content.gsub!(/^ - (description|in|name|example|required|schema):.*?\n((?: .*?\n)*?)(?=^ - |^ \w+:|\z)/m) do |match|
258+
# Extract name field from this parameter block
259+
name_match = match.match(/name:\s+(\S+)/)
260+
261+
if name_match
262+
param_name = name_match[1]
263+
param_key = param_name_to_key[param_name]
264+
265+
if param_key
266+
# Replace with $ref
267+
conversions[:converted] << param_name
268+
conversions[:replacements] += 1
269+
" - $ref: '#/components/parameters/#{param_key}'\n"
270+
else
271+
# Keep inline
272+
conversions[:not_found] << param_name
273+
match
274+
end
275+
else
276+
# No name field, keep as-is
277+
match
278+
end
279+
end
280+
281+
puts " ✅ Converted #{conversions[:converted].size} unique parameters to $ref"
282+
puts " 📊 Total replacements: #{conversions[:replacements]}"
283+
if conversions[:not_found].any?
284+
puts " ⚠️ #{conversions[:not_found].size} parameters not found in components (kept inline)"
285+
puts " Examples: #{conversions[:not_found].to_a.first(5).join(', ')}"
286+
end
287+
288+
modifications[:converted] = conversions[:converted].size
289+
modifications[:not_found] = conversions[:not_found].size
290+
291+
# ============================================================================
292+
# WRITE UPDATED FILE
293+
# ============================================================================
294+
295+
puts "\nWriting changes to: #{api_file}"
296+
File.write(api_file, api_content)
297+
298+
# ============================================================================
299+
# SUMMARY
300+
# ============================================================================
301+
302+
puts "\n" + "="*60
303+
puts "Parameter Synchronization Complete"
304+
puts "="*60
305+
puts "Phase 3a - Library Creation:"
306+
puts " Parameters added: #{modifications[:added].length}"
307+
puts " Parameters removed: #{modifications[:removed].length}"
308+
puts " Parameters skipped: #{modifications[:skipped].length}"
309+
puts "\nPhase 3b - Inline Conversion:"
310+
puts " Converted to $ref: #{modifications[:converted] || 0}"
311+
puts " Not found (kept inline): #{modifications[:not_found] || 0}"
312+
313+
if modifications[:skipped].any?
314+
puts "\nSkipped parameters (not found in source):"
315+
modifications[:skipped].each { |name| puts " - #{name}" }
316+
end
317+
318+
puts "\n✅ Successfully updated #{api_file}"
319+
puts "\nNext steps:"
320+
puts "1. Review changes: git diff #{api_file}"
321+
puts "2. Verify line count: wc -l #{api_file}"
322+
puts "3. Validate: ruby tmp/compare_openapi_specs.rb | grep -i parameter"
323+
puts "4. Test: redocly preview-docs #{api_file}"

0 commit comments

Comments
 (0)