-
-
Notifications
You must be signed in to change notification settings - Fork 158
fix: custom partition validation #1505
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: custom partition validation #1505
Conversation
add check to reject dataset creation / updation when field name contains period (.)
WalkthroughAdds validation to custom partition parsing to reject partition fields containing a dot character. After existing validation logic, the code iterates through comma-separated partition fields and returns a BAD_REQUEST error if any field includes a dot. No function signature changes. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/parseable/mod.rs (1)
1019-1026: LGTM! Validation correctly prevents dots in partition field names.The validation logic correctly iterates through partition fields and rejects any containing a dot character, with a clear error message. This appropriately prevents conflicts with the dot-separated format used in parquet filenames.
Since the validation at lines 1012-1016 already ensures at most 1 partition key, the loop will iterate at most once. If you prefer, this could be slightly simplified:
Optional simplification
- for partition in custom_partition_list.iter() { - if partition.contains('.') { - return Err(CreateStreamError::Custom { - msg: format!("custom partition field {partition} must not contain '.'"), - status: StatusCode::BAD_REQUEST, - }); - } - } + if let Some(partition) = custom_partition_list.first() { + if partition.contains('.') { + return Err(CreateStreamError::Custom { + msg: format!("custom partition field {partition} must not contain '.'"), + status: StatusCode::BAD_REQUEST, + }); + } + }However, the loop approach is also fine and may be more maintainable if the constraint changes in the future.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/parseable/mod.rs
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-18T12:37:47.732Z
Learnt from: nikhilsinhaparseable
Repo: parseablehq/parseable PR: 1405
File: src/parseable/mod.rs:528-533
Timestamp: 2025-08-18T12:37:47.732Z
Learning: In Parseable, the validate_time_partition function in src/utils/json/flatten.rs already provides a default time partition limit of 30 days using `map_or(30, |days| days.get() as i64)` when time_partition_limit is None, so no additional defaulting is needed in the stream creation logic in src/parseable/mod.rs.
Applied to files:
src/parseable/mod.rs
📚 Learning: 2025-08-20T17:01:25.791Z
Learnt from: nikhilsinhaparseable
Repo: parseablehq/parseable PR: 1409
File: src/storage/field_stats.rs:429-456
Timestamp: 2025-08-20T17:01:25.791Z
Learning: In Parseable's field stats calculation (src/storage/field_stats.rs), the extract_datetime_from_parquet_path_regex function correctly works with filename-only parsing because Parseable's server-side filename generation guarantees the dot-separated format date=YYYY-MM-DD.hour=HH.minute=MM pattern in parquet filenames.
Applied to files:
src/parseable/mod.rs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Quest Smoke and Load Tests for Standalone deployments
- GitHub Check: Quest Smoke and Load Tests for Distributed deployments
- GitHub Check: coverage
- GitHub Check: Build Default aarch64-unknown-linux-gnu
- GitHub Check: Build Default x86_64-unknown-linux-gnu
- GitHub Check: Build Default x86_64-pc-windows-msvc
- GitHub Check: Build Default aarch64-apple-darwin
- GitHub Check: Build Kafka x86_64-unknown-linux-gnu
- GitHub Check: Build Kafka aarch64-apple-darwin
add check to reject dataset creation / updation
when field name contains period (.)
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.