summaryrefslogtreecommitdiff
path: root/contrib/cirrus/lib.sh
Commit message (Collapse)AuthorAge
* CI: rootless user: also create in some root testsEd Santiago2022-01-17
| | | | | | | | | | | | viz, rootful system tests. The rootless account will be used by image-scp tests. Unfortunately, having ssh available means the system-connection tests will start running, which is very bad because they will fail, because system connection doesn't actually work (long story). Add a few more checks to prevent this test from running. Signed-off-by: Ed Santiago <santiago@redhat.com>
* podman image scp never enter podman user NScdoern2021-12-23
| | | | | | | | | | | Podman image scp should never enter the Podman UserNS unless it needs to. This allows for a sudo exec.Command to transfer images to and from rootful storage. If this command is run using sudo, the simple sudo podman save/load does not work, machinectl/su is necessary here. This modification allows for both rootful and rootless transfers, and an overall change of scp to be more of a wrapper function for different load and save calls as well as the ssh component Signed-off-by: cdoern <cdoern@redhat.com>
* test connection addJhon Honce2021-11-08
| | | | | | | | | | * Fix connection JSON encoding * Add custom ginkgo matchers for connection testing * Cleanup code Fixes #11984 Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Cirrus: Authorize rootless user self-sshChris Evich2021-11-01
| | | | | | | | | | | Future testing needs dictate rootless (in addition to root) users are able to ssh to localhost. Add ssh-key generation commands for the rootless user, and authorize their public key. Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is now done automatically by `{user,group}add` commands. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fixes due to master->main renameChris Evich2021-06-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu images to 21.04Chris Evich2021-04-27
| | | | | | | | | | | | | Also simplify `lib.sh` after supporting changes incorporated into automation library 2.x+ (present in all VM and container images). * No need to force-load `/etc/profile` and handle it's expectation to **not** being in `errexit` mode. * Slightly re-arrange loading of automation library files for clarity. * Update comments. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Make use of shared get_ci_vm containerChris Evich2021-04-05
| | | | | | | | | | | Depends on: https://github.com/containers/automation_images/pull/57 https://github.com/containers/automation/pull/64 https://github.com/containers/automation/pull/66 https://github.com/containers/automation/pull/67 https://github.com/containers/automation/pull/68 Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix running Validate task on branchesChris Evich2021-01-27
| | | | | | | | | | | | | | | | | | | | | | Followup to dbb9943 Despite skipping the `Smoke` check, it was observed on a *new* branch, the `validate` task (specifically `git-validation`) will fail. This is because: * `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch. * `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push * `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`. Fix this by eliminating the `Smoke` task entirely, simplifying all the `make validate` operations into the `validate` cirrus task. Ensure this task does not run when a new branch or tag is pushed. Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not actually used anywhere. It was formerly used for building VM images, but this has moved to another repo entirely. Signed-off-by: Chris Evich <cevich@redhat.com>
* SpellingJosh Soref2020-12-22
| | | | Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* Cirrus: Add support for Ubuntu 20.xChris Evich2020-12-15
| | | | | | | | | | | | | | | | | | | | | | | | | Previously automation always dropped the minor version number for distributions. This was intended for presentation and conditional simplicity. Bash does not support non-integer comparison natively. With the release of version 20.10, supporting testing with it and the LTS release (20.04) requires scripts to consider minor version numbers for Ubuntu VMs. This is necessary because many times in the past, some behaviors needed to be conditional on the release version number. With this commit, the images and embedded scripts/tooling uses an altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially) `$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing in the official version (from `/etc/os-release`) are dropped, and the result is concatenated. For example the current Ubuntu LTS version is `20.04`. Prior to this commit, `$OS_RELEASE_VER` would have been `20`. With this change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10` is shown as `2010`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Remove varlink support from PodmanDaniel J Walsh2020-11-26
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Use F33beta VM imageChris Evich2020-11-04
| | | | | | | | | | | | Includes disk-space increase for all Fedora images to accommodate the static-build job disk space requirements. This job substantially leverages task-cache, which was previously failing to restore early on in the Cirrus-CI task setup, due to disk-space limitations. Also simplify .cirrus.yml slightly by removing an unncessary setup and run directory change step. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #8046 from cevich/simplify_environmentOpenShift Merge Robot2020-10-29
|\ | | | | Cirrus: Simplify setting/passing env. vars.
| * Cirrus: Simplify setting/passing env. vars.Chris Evich2020-10-29
| | | | | | | | | | | | | | | | | | | | | | | | Test VMs by design are to be single-purpose, single-use, and readily disposable. Therefore it's unnecessary to overcomplicate storage of runtime environment variables. This commit makes these points clear, and reorganizes all CI-related env. vars on the system into a single location, `/etc/ci_environment`. This file is then automatically loaded, and variables exported, (by `lib.sh`) from `runner.sh` prior to executing all forms of testing. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Use google mirror for docker.ioChris Evich2020-10-28
|/ | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix obtaining a CI VMChris Evich2020-10-07
| | | | | | | | | Also removed automatic exection of setup_environment.sh since most people using this script are podman developers (not automation/CI folks). If executing the automation scripts is necessary, manual attendance to required variables like `$TEST_FLAVOR` is mandatory. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix running shellcheck locallyChris Evich2020-10-06
| | | | | | Also, check the contents of hack/get_ci_vm.sh Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix branch-validation failureChris Evich2020-10-05
| | | | | | | | | | | When validating code on a branch, determining a starting commit to check from isn't as straightforward as it would seem. Default to using the SHA from last time CI was green. If for some reason that isn't available, use an obviously wrong value to cause an intentional failure. Entirely skip this check on tag-push, since determining a starting point is incredibly difficult to do automatically. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Implement podman automation 2.0Chris Evich2020-10-02
| | | | | | | | | | | | | | | | | | Reimplement CI-automation to remove accumulated technical-debt and optimize workflow. The task-dependency graph designed goal was to shorten it's depth and increase width (i.e. more parallelism). A reduction in redundant building (and 3rd party module download) was also realized by caching `$GOPATH` and `$GOCACHE` early on. This cache is then reused in favor of a fresh clone of the repository (when possible). Note: The system tests typically execute MUCH faster than the integration tests. However, contrary to a fail-fast/fail-early principal, they are executed last. This was implemented due to debug-ability related concerns/preferences of the primary (golang-centric) project developers. Signed-off-by: Chris Evich <cevich@redhat.com>
* Fix up errors found by codespellDaniel J Walsh2020-09-11
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Obsolete CI:IMG process & related filesChris Evich2020-09-09
| | | | | | | | | | | All VM-building functionality has been migrated to https://github.com/containers/automation_images Some container-build functions are still maintained here but are on a very-short list to also be migrated to the repository linked above. Signed-off-by: Chris Evich <cevich@redhat.com>
* Use `bash` binary from env instead of /bin/bash for scriptsSascha Grunert2020-08-17
| | | | | | | | It's not possible to run any of the scripts on distributions which do have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env bash` instead. Signed-off-by: Sascha Grunert <sgrunert@suse.com>
* Remove TEST_REMOTE_CLIENT from RCLIDaniel J Walsh2020-08-10
| | | | | | | | We know these are TEST_, hoping this makes the display in cirrus easier for users to see true|false, since this is the valuable information is. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Switch all references to github.com/containers/libpod -> podmanDaniel J Walsh2020-07-28
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* CI: fix rootless permission errorEd Santiago2020-07-23
| | | | | | | | | | | | | CI runs are failing in special_testing_rootless: mkdir /var/tmp/go/pkg: permission denied Probable cause: #6822, which universally set GOPATH. Solution: in rootless setup, chown -R GOPATH as well as GOSRC (the latter was already being chowned). Signed-off-by: Ed Santiago <santiago@redhat.com>
* Fix systemd pid 1 testBrent Baude2020-07-13
| | | | | | | | | | fedora removed the systemd package from its standard container image causing our systemd pid1 test to fail. Replacing usage of fedora to ubi-init. adding ubi images to the cache for local tests. also, remove installation of test/policy.json to the system wide /etc/containers Signed-off-by: Brent Baude <bbaude@redhat.com>
* Cirrus: Include packages for containers/conmon CIChris Evich2020-06-09
| | | | | | | | | | | | | | This allows the containers/conmon repository to share the same VM images produced by containers/libpod. Included are several packages which are downloaded only since they might otherwise interfere with testing for some repos. This allows stable versions to be at the ready at testing runtime, avoiding any version updates surprising developers. Also, re-enable running the VM-image check test which was not working due to a logic problem in Cirrus-CI configuration. Update the neglected tests so that they pass on all distros. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix image-name hintsChris Evich2020-05-14
| | | | | | | | | This properly prints out image-name hints when executing the hack script without any arguments. It is required due to changes made by Ed for test-name beatification. An identical change was made and reviewed by Ed in the containers/storage repo. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu 18 to 20Chris Evich2020-05-14
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Remove libpod.conf from repoMatthew Heon2020-05-12
| | | | | | | | | | | Now that we're shipping containers.conf, we don't want to provide a libpod.conf anymore. This removes libpod.conf from the repo and as many direct uses as I can find. There are a few more mentions in the documentation, but someone more familiar with containers.conf should make those edits. Signed-off-by: Matthew Heon <mheon@redhat.com>
* Cirrus: Utilize new base imagesChris Evich2020-04-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Update to Fedora 32 properChris Evich2020-04-30
| | | | | | | | Now that it's officially released, update to it from the beta. Also (and significant), adjust the SELinux context of the GCP metadata service. Add a comment to the code explaining why this is necessary. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Unify package installationChris Evich2020-04-24
| | | | | | | Also, test-build critical container images depended upon for CI-purposes. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add support for Fedora 32Chris Evich2020-04-20
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Enable future installing buildah packagesChris Evich2020-03-19
| | | | | | | | | Many of the packages required for CI in buildah overlap with libpod. When building new VM images, attempt to source a package list from the buildah repository. If found, also install the listed packages on the VM. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Update Ubuntu base imagesChris Evich2020-03-19
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix gate image & false-positive exitsChris Evich2020-03-02
| | | | | | | | | | | | | | | | | | | A number of scripts relating to tooling used and the gate container image were not exiting upon errors as intended. Coupled with external service unavailability (i.e. downloading golangci-lint) was observed to cause difficult to debug failures. This change corrects the scripts inside/out of the gate container as well as fixes many golang related path consistency problems vs other CI jobs. After this change, all jobs use consistent path names reducing the number of special-case overrides needed. Lastly, I also made a documentation-pass, updating/correcting as needed, including documenting a likely local validation-failure mode, related to `$EPOCH_TEST_COMMIT`. This is dependent on the developers git environment, so documentation is the only possible "fix". Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Force runc use in F30Chris Evich2020-02-28
| | | | | | | | | | | Suspect crun might be sneaking in during VM image build via podman RPM dependency. Add it to the removal list when building, then also force use of runc at runtime in F30. Also quote all true/false vars to force them as strings instead of booleans (which will become capitalized) Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Support testing with F31Chris Evich2020-02-25
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix libpod base images going staleChris Evich2020-01-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | VM Base images are used as a starting point for runtime VM images. The in-use VM base images should never be pruned, which is an operation that potentially occurs periodically from automation running on the master branch of the libpod repo. However the only place which updates timestamps (blocking pruning) of base images, occurs during runtime VM image building. Therefor, if images are not regularly rebuilt, it's possible their base images go stale and are pruned. Changes: * Add freshly-produced base images (old ones got pruned) * Wrap the timestamp update script to include base image names in the update list. Notes: * Regularly updating base image timestamps only needs to happen on the libpod repo's meta task, since all base images live there. * Using a wrapper is needed to maintain compatibility with multiple versions of the imgts container image used by other repos / branchs. Signed-off-by: Chris Evich <cevich@redhat.com>
* codespell: spelling correctionsDmitry Smirnov2019-11-13
| | | | Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
* Cirrus: Fix upload_release_archive on branch or tagChris Evich2019-10-30
| | | | | | | | | | | | | | | | | Cirrus-CI only sets `$CIRRUS_BASE_SHA` when testing PRs since the destination and it's state is easy to discover. However, when post-merge and/or tag-push testing, the previous state is not easily discoverable (changes have already merged). The `upload_release_archive` script incorrectly assumed this variable was always set, causing a constant stream of post-merge testing failures. Tweak the `is_release()` function to properly handle an empty `$CIRRUS_BASE_SHA` whether or not `$CIRRUS_TAG` is also set. Also update the unit-tests to check for this. Also account for a corner case where hack/get_ci_vm.sh is running on a VM w/o git. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Only upload tagged releasesChris Evich2019-10-29
| | | | | | | | | | | | | | | | | | Prior to this commit, every push to master had it's builds packaged and uploaded to google storage. This is a waste, since potential users are only ever concerned about tagged releases. Unfortunately because the release process involves humans with potentially multiple human and automation steps happening in parallel, it's easy for automation to not detect a tagged release, or trigger on development|pre-release tags. Fix this in `upload_release_archive.sh` using a new unit-tested function `is_release()`. This acts as the definitive authority on whether or not a specific commit rage or `$CIRRUS_TAG` value constitutes something worthy of upload. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #4222 from cevich/collect_varlink_logOpenShift Merge Robot2019-10-09
|\ | | | | Cirrus: Produce and collect varlink output
| * Cirrus: Produce and collect varlink outputChris Evich2019-10-08
| | | | | | | | | | | | | | | | | | | | | | | | | | When executing 'make remotesystem' testing, a varlink process is started up but it's stdio is dumped due to the production of excessive data. However, this also means if the process has a problem, any errors will not be accessible. Instead, grab only the last 100 lines and direct them into a file. Also update automation's log collection to retrieve this file when the `$REMOTE_CLIENT` env. var. is `true`. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Install conmon in Fedora VMsChris Evich2019-10-07
|/ | | | | | | | | | This is needed because older versions of podman (1.5.1) do not automatically install the new conmon package. Also, include removal of `/usr/libexec/podman/conmon` when preparing to install and test podman built from source. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Disable boottime Ubuntu package updateChris Evich2019-09-26
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: disable Evil Units in base-imagesChris Evich2019-09-20
| | | | | | Also, minor update to prevent harmless 'Fatal: not a git repo' error. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add latest ubuntuChris Evich2019-09-20
| | | | | | | | | | | | Add the latest Ubuntu version into the testing matrix and image-build workflow. This is also needed to support other containers projects which share use of VM images from this one. Update package lists to include needs for contianers/storage use of images. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Prevent resident pollutionChris Evich2019-09-10
| | | | | | | | | | | | | When constructing VM cache-images, the latest/greatest podman package is installed to ensure all necessary dependencies are met. Prior to testing source-built binaries, most of of the packaged files are removed. However, if the `io.podman` service or socket is enabled/running, it could cause the packaged podman and varlink binaries to be both resident and cached. Since this condition would cause very difficult to diagnose behaviors, add preventative measures to ensure these services are absent prior to removing packaged podman files. Signed-off-by: Chris Evich <cevich@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-06 10:21:43 +0000