Skip to content

Conversation

@vaibhavdaren
Copy link
Contributor

@vaibhavdaren vaibhavdaren commented Dec 24, 2025

Add simple, example-based integration tests for artifact providers:

  • koji.build: Fetches packages from Koji using build IDs
  • repository-url: Configures external package repositories
  • multi: Demonstrates using multiple providers together

Once the design is approved, additional providers (koji.task, koji.nvr, file, copr.build, brew.*) can be added following the same pattern.

Related to #4420

@vaibhavdaren vaibhavdaren added the ci | skip Do not run any tests (useful for very early drafts). label Dec 24, 2025
@vaibhavdaren vaibhavdaren force-pushed the vaibhav-artifact-install-integration-tests branch 3 times, most recently from dc828a0 to ca4c960 Compare December 29, 2025 18:11
@vaibhavdaren vaibhavdaren added ci | full test Pull request is ready for the full test execution plugin | artifact Related to the `prepare/artifact` plugin. test coverage Improvements or additions to test coverage of tmt itself and removed ci | skip Do not run any tests (useful for very early drafts). labels Jan 5, 2026
@vaibhavdaren vaibhavdaren marked this pull request as ready for review January 5, 2026 12:41
@vaibhavdaren vaibhavdaren force-pushed the vaibhav-artifact-install-integration-tests branch from ca4c960 to a5284c8 Compare January 6, 2026 12:30
@github-project-automation github-project-automation bot moved this to backlog in planning Jan 6, 2026
@happz happz moved this from backlog to review in planning Jan 6, 2026
Add simple, example-based integration tests for artifact providers:
- koji.build: Fetches packages from Koji using build IDs
- repository-url: Configures external package repositories
- multi: Demonstrates using multiple providers together

These tests serve as usage examples and verify basic functionality.
Each test:
- Shows clear usage of the --provide argument
- Runs a complete tmt workflow
- Verifies packages are installed correctly

Once the design is approved, additional providers (koji.task, koji.nvr,
file, copr.build, brew.*) can be added following the same pattern.

Related to #4420
@vaibhavdaren vaibhavdaren self-assigned this Jan 7, 2026
@LecrisUT LecrisUT self-assigned this Jan 7, 2026
Copy link
Contributor

@LecrisUT LecrisUT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the goal for this PR to split the /tests/prepare/artifact into individual provider tests? If so, why not add it to it instead? For example you can add a for loop in the test.sh to loop over each provider if that is the intent.

One concern with the current approach is that the koji list-tagged logic is quite non-trivial and would prefer to be defined in a single place.

rpm -q make

# Install docker-ce-cli from the configured repository
dnf install -y docker-ce-cli
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not be installing from the script since this is uncontrolled by us.

@psss psss added this to the 1.65 milestone Jan 8, 2026
@vaibhavdaren vaibhavdaren force-pushed the vaibhav-artifact-install-integration-tests branch 2 times, most recently from d3d3e13 to 92bad06 Compare January 8, 2026 13:39
Comment on lines +21 to +27
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
rlPhaseEnd

rlPhaseStartTest "Test koji.build provider"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact --provide koji.build:$make_build_id" 0 "Run with koji.build provider"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing error check after calling get_koji_build_id. If the function fails (returns empty string), the test continues with an invalid koji.build: provider (missing build ID) which will cause the tmt run to fail with an unclear error message.

Fix by adding a check:

make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
if [ -z "$make_build_id" ]; then
    rlDie "Failed to get koji build ID for make"
fi
Suggested change
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
rlPhaseEnd
rlPhaseStartTest "Test koji.build provider"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact --provide koji.build:$make_build_id" 0 "Run with koji.build provider"
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
if [ -z "$make_build_id" ]; then
rlDie "Failed to get koji build ID for make"
fi
rlPhaseEnd
rlPhaseStartTest "Test koji.build provider"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact --provide koji.build:$make_build_id" 0 "Run with koji.build provider"

Spotted by Graphite Agent

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

Comment on lines +21 to +29
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
rlPhaseEnd

rlPhaseStartTest "Test multiple providers"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact \
--provide koji.build:$make_build_id \
--provide repository-url:https://download.docker.com/linux/fedora/docker-ce.repo" 0 "Run with multiple providers"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing error check after calling get_koji_build_id. If the function fails (returns empty string), the test continues with an invalid koji.build: provider (missing build ID) which will cause the tmt run to fail with an unclear error message.

Fix by adding a check:

make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
if [ -z "$make_build_id" ]; then
    rlDie "Failed to get koji build ID for make"
fi
Suggested change
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
rlPhaseEnd
rlPhaseStartTest "Test multiple providers"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact \
--provide koji.build:$make_build_id \
--provide repository-url:https://download.docker.com/linux/fedora/docker-ce.repo" 0 "Run with multiple providers"
make_build_id=$(get_koji_build_id "make" "f${fedora_release}")
if [ -z "$make_build_id" ]; then
rlDie "Failed to get koji build ID for make"
fi
rlPhaseEnd
rlPhaseStartTest "Test multiple providers"
rlRun "tmt run -i $run --scratch -vv --all \
provision -h $PROVISION_HOW --image $TEST_IMAGE_PREFIX/fedora/${fedora_release}:latest \
prepare --how artifact \
--provide koji.build:$make_build_id \
--provide repository-url:https://download.docker.com/linux/fedora/docker-ce.repo" 0 "Run with multiple providers"

Spotted by Graphite Agent

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci | full test Pull request is ready for the full test execution plugin | artifact Related to the `prepare/artifact` plugin. test coverage Improvements or additions to test coverage of tmt itself

Projects

Status: review

Development

Successfully merging this pull request may close these issues.

4 participants