summaryrefslogtreecommitdiff
path: root/contrib/cirrus/lib.sh
Commit message (Collapse)AuthorAge
* podman image scp remote support & podman image scp taggingcdoern2022-06-28
| | | | | | | | | | | | | | | | | | | add support for podman-remote image scp as well as direct access via the API. This entailed a full rework of the layering of image scp functions as well as the usual API plugging and type creation also, implemented podman image scp tagging. which makes the syntax much more readable and allows users t tag the new image they are loading to the local/remote machine: allow users to pass a "new name" for the image they are transferring `podman tag` as implemented creates a new image im `image list` when tagging, so this does the same meaning that when transferring images with tags, podman on the remote machine/user will load two images ex: `podman image scp computer1::alpine computer2::foobar` creates alpine:latest and localhost/foobar on the remote host implementing tags means removal of the flexible syntax. In the currently released podman image scp, the user can either specify `podman image scp source::img dest::` or `podman image scp dest:: source::img`. However, with tags this task becomes really hard to check which is the image (src) and which is the new tag (dst). Removal of that streamlines the arg parsing process Signed-off-by: Charlie Doern <cdoern@redhat.com>
* Fix spelling "setup" -> "set up" and similarErik Sjölund2022-06-22
| | | | | | | | | | * Replace "setup", "lookup", "cleanup", "backup" with "set up", "look up", "clean up", "back up" when used as verbs. Replace also variations of those. * Improve language in a few places. Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
* Cirrus: Fix several TODOsChris Evich2022-05-26
| | | | | | | | Most were simply deleted, the main one addressed is in the "pre-testing" `ext_svc_check.sh` script. It will now verify accessibility of several key test images we maintain in `quay.io`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #14179 from cevich/ensure_dest_branchOpenShift Merge Robot2022-05-10
|\ | | | | Cirrus: Guarantee $DEST_BRANCH is passed through
| * Cirrus: Guarantee $DEST_BRANCH is passed throughChris Evich2022-05-10
| | | | | | | | | | | | | | | | | | | | | | | | | | There are several runtime contexts (rootless and container) where `$DEST_BRANCH` is needed but was not supplied. A prior commit (c4865767171b) removed the default value, `main` which was being set incorrectly when CI ran on release branches. Fix this by ensuring the variable is non-empty upon entry to `setup_environment.sh`, then ensure it gets passed through to child environments by way of the `/etc/ci_environment` file. This will maintain compatibility with both CI and `hack/get_ci_vm.sh` use. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Update images + new automation libraryChris Evich2022-05-10
| | | | | | | | | | | | | | | | Add support for new automation library version w/ `$DEBUG` fix (ref: containers/automation_images#128) and added definitions for commonly used Distro/version variables. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Simplify rootless ssh setupChris Evich2022-05-10
|/ | | | | | | | | | | | | The sshd service is guaranteed to be running by the VM image build process - it's required by the packer tool for access. Remove the startup and check on the sshd service. For many tests, man ssh connections to/from $ROOTLESS_USER on the host are needed. To facilitate this, the localhost key is added to `known_hosts` for root and `$ROOTLESS_USER`. Simplify this setup using the `ssh-keyscan` tool. Signed-off-by: Chris Evich <cevich@redhat.com>
* test skipper: check for $DEST_BRANCHEd Santiago2022-05-09
| | | | | | | | | | | | | | The test-skipping optimization is failing as rootless on non-main, because $DEST_BRANCH is not set. Solution: check for envariable, skip test if missing. (This was part of my original PR, but was accidentally removed in #14013) Also: DEST_BRANCH was silently being defaulted to 'main' in lib.sh. Remove that: per @cevich, it is no longer necessary. Fixes: #14131 Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: Fix ownership of repos. to keep git happyChris Evich2022-04-27
| | | | | | | | | | | | | | | | | | Newer versions of git are much more pedantic about who owns the repository files. When setting up to run rootless, prior to this commit, the repo. ownership was changed from root. This causes all subsequent git-operations as root to fail: ``` fatal: unsafe repository ('<$GOSRC>' is owned by someone else) ``` Fix this by re-ordering operations, such that the change in ownership is done immediately before executing as a user. Also disable the git-ownership check on the source repository assuming the CI environment is disposable. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Temporarily update netavark/aardvark-dnsChris Evich2022-04-27
| | | | | | | | | | | | Normally installing/updating packages at test runtime is highly discouraged for reliability and efficiency reasons. However, in this specific case, development work of these packages is still fairly hot. As a compromise to support podman test development, temporarily update these two specific packages at runtime. At a future date, when updates are less frequent, this commit can/should be safely reverted. At that point, the versions installed at VM image build time will persist. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Test w/ netavark/aardvark-dns in F36+Chris Evich2022-04-27
| | | | | | | | | | | | | | | | | | | | Now that netavark and aardvark are packaged and default in F36, support CNI-based testing in F35 and Ubuntu. * Remove the temporary/special `$TEST_ENVIRON=host-netavark` construct. * Remove dedicated/special integration and system testing tasks. * Update test-config setup to properly handle CNI vs netavark/aardvark environments. * Update package-version logging to operate based on installed packages (along with some other minor script cleanups). * Update global environment setup to force `$NETWORK_BACKEND=netavark` in F36 and later. Except when `upgrade_test` task runs. * Discontinue installing netavark and aardvark-dns binaries from upstream build artifacts. * Drop CGV1-vs-2 policy check. Ubuntu VMs now exclusively test CGv1, Fedora VMs test CGv2, with F35 testing CNI and F36 testing Netavark. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add netavark/aardvark system test taskChris Evich2022-02-03
| | | | | | | Also add a system-test that verifies netavark driver is in use when magic env. var. is set. Signed-off-by: Chris Evich <cevich@redhat.com>
* CI: rootless user: also create in some root testsEd Santiago2022-01-17
| | | | | | | | | | | | viz, rootful system tests. The rootless account will be used by image-scp tests. Unfortunately, having ssh available means the system-connection tests will start running, which is very bad because they will fail, because system connection doesn't actually work (long story). Add a few more checks to prevent this test from running. Signed-off-by: Ed Santiago <santiago@redhat.com>
* podman image scp never enter podman user NScdoern2021-12-23
| | | | | | | | | | | Podman image scp should never enter the Podman UserNS unless it needs to. This allows for a sudo exec.Command to transfer images to and from rootful storage. If this command is run using sudo, the simple sudo podman save/load does not work, machinectl/su is necessary here. This modification allows for both rootful and rootless transfers, and an overall change of scp to be more of a wrapper function for different load and save calls as well as the ssh component Signed-off-by: cdoern <cdoern@redhat.com>
* test connection addJhon Honce2021-11-08
| | | | | | | | | | * Fix connection JSON encoding * Add custom ginkgo matchers for connection testing * Cleanup code Fixes #11984 Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Cirrus: Authorize rootless user self-sshChris Evich2021-11-01
| | | | | | | | | | | Future testing needs dictate rootless (in addition to root) users are able to ssh to localhost. Add ssh-key generation commands for the rootless user, and authorize their public key. Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is now done automatically by `{user,group}add` commands. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fixes due to master->main renameChris Evich2021-06-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu images to 21.04Chris Evich2021-04-27
| | | | | | | | | | | | | Also simplify `lib.sh` after supporting changes incorporated into automation library 2.x+ (present in all VM and container images). * No need to force-load `/etc/profile` and handle it's expectation to **not** being in `errexit` mode. * Slightly re-arrange loading of automation library files for clarity. * Update comments. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Make use of shared get_ci_vm containerChris Evich2021-04-05
| | | | | | | | | | | Depends on: https://github.com/containers/automation_images/pull/57 https://github.com/containers/automation/pull/64 https://github.com/containers/automation/pull/66 https://github.com/containers/automation/pull/67 https://github.com/containers/automation/pull/68 Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix running Validate task on branchesChris Evich2021-01-27
| | | | | | | | | | | | | | | | | | | | | | Followup to dbb9943 Despite skipping the `Smoke` check, it was observed on a *new* branch, the `validate` task (specifically `git-validation`) will fail. This is because: * `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch. * `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push * `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`. Fix this by eliminating the `Smoke` task entirely, simplifying all the `make validate` operations into the `validate` cirrus task. Ensure this task does not run when a new branch or tag is pushed. Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not actually used anywhere. It was formerly used for building VM images, but this has moved to another repo entirely. Signed-off-by: Chris Evich <cevich@redhat.com>
* SpellingJosh Soref2020-12-22
| | | | Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* Cirrus: Add support for Ubuntu 20.xChris Evich2020-12-15
| | | | | | | | | | | | | | | | | | | | | | | | | Previously automation always dropped the minor version number for distributions. This was intended for presentation and conditional simplicity. Bash does not support non-integer comparison natively. With the release of version 20.10, supporting testing with it and the LTS release (20.04) requires scripts to consider minor version numbers for Ubuntu VMs. This is necessary because many times in the past, some behaviors needed to be conditional on the release version number. With this commit, the images and embedded scripts/tooling uses an altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially) `$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing in the official version (from `/etc/os-release`) are dropped, and the result is concatenated. For example the current Ubuntu LTS version is `20.04`. Prior to this commit, `$OS_RELEASE_VER` would have been `20`. With this change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10` is shown as `2010`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Remove varlink support from PodmanDaniel J Walsh2020-11-26
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Use F33beta VM imageChris Evich2020-11-04
| | | | | | | | | | | | Includes disk-space increase for all Fedora images to accommodate the static-build job disk space requirements. This job substantially leverages task-cache, which was previously failing to restore early on in the Cirrus-CI task setup, due to disk-space limitations. Also simplify .cirrus.yml slightly by removing an unncessary setup and run directory change step. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #8046 from cevich/simplify_environmentOpenShift Merge Robot2020-10-29
|\ | | | | Cirrus: Simplify setting/passing env. vars.
| * Cirrus: Simplify setting/passing env. vars.Chris Evich2020-10-29
| | | | | | | | | | | | | | | | | | | | | | | | Test VMs by design are to be single-purpose, single-use, and readily disposable. Therefore it's unnecessary to overcomplicate storage of runtime environment variables. This commit makes these points clear, and reorganizes all CI-related env. vars on the system into a single location, `/etc/ci_environment`. This file is then automatically loaded, and variables exported, (by `lib.sh`) from `runner.sh` prior to executing all forms of testing. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Use google mirror for docker.ioChris Evich2020-10-28
|/ | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix obtaining a CI VMChris Evich2020-10-07
| | | | | | | | | Also removed automatic exection of setup_environment.sh since most people using this script are podman developers (not automation/CI folks). If executing the automation scripts is necessary, manual attendance to required variables like `$TEST_FLAVOR` is mandatory. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix running shellcheck locallyChris Evich2020-10-06
| | | | | | Also, check the contents of hack/get_ci_vm.sh Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix branch-validation failureChris Evich2020-10-05
| | | | | | | | | | | When validating code on a branch, determining a starting commit to check from isn't as straightforward as it would seem. Default to using the SHA from last time CI was green. If for some reason that isn't available, use an obviously wrong value to cause an intentional failure. Entirely skip this check on tag-push, since determining a starting point is incredibly difficult to do automatically. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Implement podman automation 2.0Chris Evich2020-10-02
| | | | | | | | | | | | | | | | | | Reimplement CI-automation to remove accumulated technical-debt and optimize workflow. The task-dependency graph designed goal was to shorten it's depth and increase width (i.e. more parallelism). A reduction in redundant building (and 3rd party module download) was also realized by caching `$GOPATH` and `$GOCACHE` early on. This cache is then reused in favor of a fresh clone of the repository (when possible). Note: The system tests typically execute MUCH faster than the integration tests. However, contrary to a fail-fast/fail-early principal, they are executed last. This was implemented due to debug-ability related concerns/preferences of the primary (golang-centric) project developers. Signed-off-by: Chris Evich <cevich@redhat.com>
* Fix up errors found by codespellDaniel J Walsh2020-09-11
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Obsolete CI:IMG process & related filesChris Evich2020-09-09
| | | | | | | | | | | All VM-building functionality has been migrated to https://github.com/containers/automation_images Some container-build functions are still maintained here but are on a very-short list to also be migrated to the repository linked above. Signed-off-by: Chris Evich <cevich@redhat.com>
* Use `bash` binary from env instead of /bin/bash for scriptsSascha Grunert2020-08-17
| | | | | | | | It's not possible to run any of the scripts on distributions which do have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env bash` instead. Signed-off-by: Sascha Grunert <sgrunert@suse.com>
* Remove TEST_REMOTE_CLIENT from RCLIDaniel J Walsh2020-08-10
| | | | | | | | We know these are TEST_, hoping this makes the display in cirrus easier for users to see true|false, since this is the valuable information is. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Switch all references to github.com/containers/libpod -> podmanDaniel J Walsh2020-07-28
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* CI: fix rootless permission errorEd Santiago2020-07-23
| | | | | | | | | | | | | CI runs are failing in special_testing_rootless: mkdir /var/tmp/go/pkg: permission denied Probable cause: #6822, which universally set GOPATH. Solution: in rootless setup, chown -R GOPATH as well as GOSRC (the latter was already being chowned). Signed-off-by: Ed Santiago <santiago@redhat.com>
* Fix systemd pid 1 testBrent Baude2020-07-13
| | | | | | | | | | fedora removed the systemd package from its standard container image causing our systemd pid1 test to fail. Replacing usage of fedora to ubi-init. adding ubi images to the cache for local tests. also, remove installation of test/policy.json to the system wide /etc/containers Signed-off-by: Brent Baude <bbaude@redhat.com>
* Cirrus: Include packages for containers/conmon CIChris Evich2020-06-09
| | | | | | | | | | | | | | This allows the containers/conmon repository to share the same VM images produced by containers/libpod. Included are several packages which are downloaded only since they might otherwise interfere with testing for some repos. This allows stable versions to be at the ready at testing runtime, avoiding any version updates surprising developers. Also, re-enable running the VM-image check test which was not working due to a logic problem in Cirrus-CI configuration. Update the neglected tests so that they pass on all distros. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix image-name hintsChris Evich2020-05-14
| | | | | | | | | This properly prints out image-name hints when executing the hack script without any arguments. It is required due to changes made by Ed for test-name beatification. An identical change was made and reviewed by Ed in the containers/storage repo. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu 18 to 20Chris Evich2020-05-14
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Remove libpod.conf from repoMatthew Heon2020-05-12
| | | | | | | | | | | Now that we're shipping containers.conf, we don't want to provide a libpod.conf anymore. This removes libpod.conf from the repo and as many direct uses as I can find. There are a few more mentions in the documentation, but someone more familiar with containers.conf should make those edits. Signed-off-by: Matthew Heon <mheon@redhat.com>
* Cirrus: Utilize new base imagesChris Evich2020-04-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Update to Fedora 32 properChris Evich2020-04-30
| | | | | | | | Now that it's officially released, update to it from the beta. Also (and significant), adjust the SELinux context of the GCP metadata service. Add a comment to the code explaining why this is necessary. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Unify package installationChris Evich2020-04-24
| | | | | | | Also, test-build critical container images depended upon for CI-purposes. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add support for Fedora 32Chris Evich2020-04-20
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Enable future installing buildah packagesChris Evich2020-03-19
| | | | | | | | | Many of the packages required for CI in buildah overlap with libpod. When building new VM images, attempt to source a package list from the buildah repository. If found, also install the listed packages on the VM. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu base imagesChris Evich2020-03-19
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix gate image & false-positive exitsChris Evich2020-03-02
| | | | | | | | | | | | | | | | | | | A number of scripts relating to tooling used and the gate container image were not exiting upon errors as intended. Coupled with external service unavailability (i.e. downloading golangci-lint) was observed to cause difficult to debug failures. This change corrects the scripts inside/out of the gate container as well as fixes many golang related path consistency problems vs other CI jobs. After this change, all jobs use consistent path names reducing the number of special-case overrides needed. Lastly, I also made a documentation-pass, updating/correcting as needed, including documenting a likely local validation-failure mode, related to `$EPOCH_TEST_COMMIT`. This is dependent on the developers git environment, so documentation is the only possible "fix". Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Force runc use in F30Chris Evich2020-02-28
| | | | | | | | | | | Suspect crun might be sneaking in during VM image build via podman RPM dependency. Add it to the removal list when building, then also force use of runc at runtime in F30. Also quote all true/false vars to force them as strings instead of booleans (which will become capitalized) Signed-off-by: Chris Evich <cevich@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-10-23 20:42:39 +0000