feat: new push/pull

[timmilesdw]

Unified Pull/Push Mechanism for Mirror Operations

Summary

This PR introduces a new unified pull/push mechanism activated via NEW_PULL=true environment variable. The key principle is no path transformations - the local OCI layout structure directly mirrors the registry structure.

Changes

New Push Service (internal/mirror/push.go)

• Single push operation for all components (platform, modules, security, plugins)
• Unpacks all bundles into a unified directory
• Walks the directory tree, finds OCI layouts by index.json
• Pushes each layout using its relative path as the registry segment
• Creates modules index (deckhouse/modules:<module-name> tags) for module discovery
• Local folder structure now matches registry structure exactly

Testing

Verified full parity between old and new mechanisms:

• 728 tags in both deckhouse and deckhouse-etalon
• All repositories match including:
  • Platform: install, install-standalone, release-channel
  • Modules: main images, release channels, extra images
  • Security databases: trivy-db, trivy-bdu, etc.
  • Modules index tags

Usage

Pull with new mechanism

NEW_PULL=true d8 mirror pull ...

Push with new mechanism

NEW_PULL=true d8 mirror push ...

Migration

The new mechanism is opt-in via environment variable. Existing behavior is unchanged when NEW_PULL is not set.

[Copilot]

Pull request overview

This PR introduces a new unified pull/push mechanism for mirror operations, activated via the NEW_PULL=true environment variable. The key architectural change is eliminating path transformations - the local OCI layout structure now directly mirrors the registry structure, making the push/pull operations simpler and more reliable.

Changes:

• New unified push service that automatically discovers and pushes all OCI layouts based on their directory structure
• Refactored module extra images to use separate layouts per extra image type instead of a shared layout
• Added --ignore-suspend flag to bypass suspended release channel checks
• Added graceful shutdown handling with signal capture for Ctrl+C interruption

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
internal/mirror/push.go	New push service implementation that walks directory tree and pushes layouts based on relative paths
internal/mirror/pusher/pusher.go	Refactored pusher service with simplified layout pushing logic
internal/mirror/modules/modules.go	Enhanced module pulling to support per-extra-image layouts, internal digest images, and release versions
internal/mirror/modules/layout.go	Changed extra images from single layout to map of layouts per extra image name
internal/mirror/platform/platform.go	Added IgnoreSuspend option and version tags to release-channel
internal/mirror/platform/layout.go	Added version tags to DeckhouseReleaseChannel download list
internal/mirror/puller/puller.go	Added digest reference handling for images with @sha256: format
internal/mirror/puller/types.go	Simplified SplitImageRefByRepoAndTag by removing special digest handling
internal/mirror/security/security.go	Changed layout creation path from nested to root-based
internal/mirror/cmd/push/push.go	Added new push execution path using NEW_PULL environment variable
internal/mirror/cmd/pull/pull.go	Added signal handling and IgnoreSuspend parameter passing
internal/mirror/cmd/pull/flags/flags.go	Added --ignore-suspend flag
internal/mirror/releases/versions.go	Fixed typo in function name and re-enabled suspend check with override
internal/mirror/releases/versions_test.go	Fixed typo in test function names
pkg/libmirror/bundle/bundle.go	Added PackWithPrefix to support prefixed tar paths
pkg/libmirror/layouts/push.go	Refactored push logic to use config struct and improved logging
pkg/libmirror/operations/params/pull.go	Added IgnoreSuspend parameter
pkg/libmirror/util/retry/retry.go	Made RunTaskWithContext private (renamed to runTaskWithContext)
pkg/registry/service/module_service.go	Added ExtraImage method for scoped extra image services
internal/mirror.go	Removed MirrorTypeModulesExtra enum value
internal/layout.go	Updated path mappings and documentation for module layouts

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.