-
Notifications
You must be signed in to change notification settings - Fork 94
Implement base_meterPriorityFeePerGas so transactions can outbid resource constraints #201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
niran
wants to merge
14
commits into
main
Choose a base branch
from
node-reth-metered-priority-fee
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
🟡 Heimdall Review Status
|
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
38f238a to
7d9523f
Compare
Add MeteringCache and PriorityFeeEstimator to crates/rpc for resource-aware priority fee estimation in flashblocks. - cache.rs: In-memory cache storing metered transactions by block/flashblock with resource totals (gas, execution time, state root time, data availability bytes) - estimator.rs: Top-down fee estimation algorithm that determines minimum priority fee needed for bundle inclusion based on resource competition
- kafka.rs: KafkaBundleConsumer reads accepted bundle events from Kafka and extracts per-transaction metering data including gas, execution time, and data availability bytes - annotator.rs: ResourceAnnotator correlates Kafka metering data with flashblock inclusion events to populate the metering cache with properly indexed transaction data
Wires priority fee estimation into the RPC layer: - Add MeteredPriorityFeeResponse types with resource-specific estimates - Add metered_priority_fee_per_gas method to MeteringApi trait - Update MeteringApiImpl to accept optional PriorityFeeEstimator - Add MeteringConfig, KafkaConfig, ResourceLimitsConfig to runner - Wire estimator creation in BaseRpcExtension when Kafka is configured - Add 10 CLI args for resource limits, cache size, and Kafka config
Port additional tests from original metering crate: - compute_estimate_empty_transactions: verify uncongested behavior - estimate_for_block_respects_limits: integration test for block estimates - estimate_for_block_propagates_limit_errors: verify error propagation - estimate_rolling_aggregates_across_blocks: verify rolling median calculation
Completes the metering pipeline by: - Creating MeteringRuntime to hold cache, estimator, and channels - Spawning ResourceAnnotator task to correlate transactions with flashblocks - Spawning KafkaBundleConsumer task to consume AcceptedBundle events - Adding CompositeFlashblocksReceiver that forwards to both FlashblocksState and the metering pipeline via FlashblockInclusion events - Adding flashblock_inclusion_from_flashblock helper to extract tx hashes - Supporting Kafka properties file loading The metering cache is now populated when: 1. Kafka consumer receives AcceptedBundle events (transaction data) 2. FlashblocksSubscriber receives flashblocks (inclusion position) 3. ResourceAnnotator correlates both to update the cache
7d9523f to
c381fa9
Compare
- Add Kafka startup warning when metering is enabled but Kafka is not configured - Wire shared OpDAConfig from BaseNodeConfig to OpNode and PriorityFeeEstimator so that miner_setMaxDASize affects priority fee estimation - Add doc comment for metering-da-bytes explaining dynamic override capability - Add reth-optimism-payload-builder dependency to runner and node crates
- Change FlashblockMetrics storage from IndexMap to sorted Vec - Use binary search insertion to maintain descending order by priority fee - Remove redundant sorting in compute_estimate (now expects pre-sorted input) - Rename upsert_transaction to insert_transaction (upserts weren't used) - Remove unused ResourceTotals::subtract method - Keep sort only for aggregate resources (merging multiple flashblocks)
- Change fee fields from String to U256 (matches Ethereum RPC conventions) - Rename RollingPriorityEstimates to RollingPriorityEstimate (singular) - Rename recommended_priority_fee to priority_fee (shorter, clearer) - Rename resource_estimates_to_vec to build_resource_estimate_responses - Update variable names to match singular type
Keep only exceptional/diagnostic metrics: - metering.kafka.errors_total (errors) - metering.kafka.lag_ms (monitoring) - metering.kafka.messages_skipped (data issues) - metering.pending.evicted (capacity issues) - metering.pending.size (queue depth monitoring) - metering.streams.tx_misses_total (correlation failures) Remove normal activity counters: - metering.kafka.messages_total - metering.kafka.tx_events_total - metering.streams.flashblocks_total - metering.streams.tx_matched_total
- Add contains_block() and clear_blocks_from() methods to MeteringCache - Detect reorg when flashblock_index=0 arrives for existing block - Clear affected blocks from cache on reorg detection - Add metering.cache.reorgs_detected metric - Add unit and integration tests for reorg handling
- Remove --metering-kafka-brokers CLI arg (use properties file instead) - Make --metering-kafka-properties-file the only required arg for Kafka - Default --metering-kafka-topic to tips-ingress - Make --metering-kafka-group-id optional (overrides properties file) - Load all rdkafka settings from properties file instead of hardcoding
- Move ResourceFeeEstimateResponse and MeteredPriorityFeeResponse to types.rs - Move build_priority_fee_response helper to meter_rpc.rs - Delete metered_fee_types.rs
- Rename insert_transaction to push_transaction - Remove binary search insertion, just append in sequencer order - Update docs and test to reflect sequencer ordering is preserved
- Add const fn annotations where possible (cache, estimator, runner) - Add Debug impls for ResourceAnnotator and KafkaBundleConsumer - Remove redundant clones in meter_rpc and runner - Change pub mod to pub(crate) mod for internal modules - Replace redundant closure with function reference - Remove unused cache field from MeteringRuntime
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds metered priority fee estimation to the Base node, enabling bundles to receive priority fee recommendations based on real-time resource consumption from recent flashblocks.
base_meteredPriorityFeePerGas- Simulates a bundle and returns a recommended priority fee based on recent flashblock congestion across four resource dimensions:miner_setMaxDASize)The estimator uses a displacement algorithm: for each resource, it finds the minimum priority fee required to displace enough existing transactions to make room for the bundle's resource demand, then takes the maximum across all resources and median across recent blocks.
Architecture
The metering pipeline consists of:
KafkaBundleConsumer) - ConsumesAcceptedBundleevents from TIPS, extracting per-transaction resource usageCLI Configuration
The metering system supports
miner_setMaxDASizeto dynamically update the limit for DA bytes allowed in a block.Test Plan
cargo +nightly fmtandcargo +nightly clippypass