feat(amp-data-store): introduce crate with unified store interface #1517
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Consolidate data storage and caching into a single abstraction to simplify the architecture and eliminate the distinction between cached/uncached stores.
DataStorecombines storage and cache in a single typeNote
Introduces a unified data access layer and refactors the codebase to use it.
amp-data-storecrate providingDataStore(object store + metadata DB) with integrated parquet footer cache (memory-weighted eviction viafoyer)PhyTableUrl,PhyTablePath,PhyTableRevisionPath) andFileNameintoamp-data-store; update error types and usescommon::store(includingCachedStore); replace withDataStoreincommoncatalog/metadata/reader/query_context and related APIsserver,controller,worker,admin-api), CLI (ampd),dump,datasets-derived, and tests to depend on and constructDataStore(passingconfig.parquet.cache_size_mb)parquet_cache_size_mbfromQueryEnv/server config andConfig::make_query_env; caching handled byDataStore::newStreamingQueryHandle::as_stream→into_stream, replacedebug!withtracing::debug!in some pathsWritten by Cursor Bugbot for commit 2f5a5ea. This will update automatically on new commits. Configure here.