summaryrefslogtreecommitdiff
path: root/contrib/cirrus/setup_environment.sh
Commit message (Collapse)AuthorAge
* Cirrus: Add podman-machine integration testChris Evich2022-07-01
| | | | | | | | | | | | | | | | | | | | | | | The podman-machine integration tests are designed to execute on bare-metal, since they perform significant work with virtual-machines. This test is costly to run at scale, so it is limited to being manually triggered by developers (for now). A 'trigger' button will appear in the task status page of the Github WebUI once all test dependencies are met. In the Cirrus-CI WebUI, there is also a 'pre-trigger' button that may be pressed if a developer doesn't wish to wait. Also: * Add a `localmachine` target in the `Makefile` on the off-chance developers wish to execute locally. Update the `ginkgo-run` target to accommodate re-use by the new `localmachine` target. * Exclude `podman_machine` task from `success` dependency verification. This also involves adding an exception to `cirrus_yaml_test.py` otherwise it will complain loudly. * ***NOTE*** Inclusion of `ec2_instance` in *any* task will cause `hack/get_ci_vm.sh` to barf and be non-functional. Future updates will be made to restore functionality. Before then, simply comment out the `ec2_instance` section as a temporarily workaround. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix elevator workaround multi-cloud supportChris Evich2022-07-01
| | | | | | | | | | | | | | | | In order to support execution on various non-GCP cloud environments, the BFQ scheduler workaround needs updating. Previously it assumed the root disk was always `/dev/sda`. With the addition of new clouds (AWS) and different environment types, the assumption is not always valid. Update the workaround to take care in looking up the block device where '/' comes from. Also update the scheduler to 'none', as all modern clouds already have highly optimized underlying storage configurations. There's no reason to complicate I/O paths further by hard-coding specific scheduler(s) for all environment types. Signed-off-by: Chris Evich <cevich@redhat.com>
* podman image scp remote support & podman image scp taggingcdoern2022-06-28
| | | | | | | | | | | | | | | | | | | add support for podman-remote image scp as well as direct access via the API. This entailed a full rework of the layering of image scp functions as well as the usual API plugging and type creation also, implemented podman image scp tagging. which makes the syntax much more readable and allows users t tag the new image they are loading to the local/remote machine: allow users to pass a "new name" for the image they are transferring `podman tag` as implemented creates a new image im `image list` when tagging, so this does the same meaning that when transferring images with tags, podman on the remote machine/user will load two images ex: `podman image scp computer1::alpine computer2::foobar` creates alpine:latest and localhost/foobar on the remote host implementing tags means removal of the flexible syntax. In the currently released podman image scp, the user can either specify `podman image scp source::img dest::` or `podman image scp dest:: source::img`. However, with tags this task becomes really hard to check which is the image (src) and which is the new tag (dst). Removal of that streamlines the arg parsing process Signed-off-by: Charlie Doern <cdoern@redhat.com>
* Merge pull request #14179 from cevich/ensure_dest_branchOpenShift Merge Robot2022-05-10
|\ | | | | Cirrus: Guarantee $DEST_BRANCH is passed through
| * Cirrus: Guarantee $DEST_BRANCH is passed throughChris Evich2022-05-10
| | | | | | | | | | | | | | | | | | | | | | | | | | There are several runtime contexts (rootless and container) where `$DEST_BRANCH` is needed but was not supplied. A prior commit (c4865767171b) removed the default value, `main` which was being set incorrectly when CI ran on release branches. Fix this by ensuring the variable is non-empty upon entry to `setup_environment.sh`, then ensure it gets passed through to child environments by way of the `/etc/ci_environment` file. This will maintain compatibility with both CI and `hack/get_ci_vm.sh` use. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Update images + new automation libraryChris Evich2022-05-10
| | | | | | | | | | | | | | | | Add support for new automation library version w/ `$DEBUG` fix (ref: containers/automation_images#128) and added definitions for commonly used Distro/version variables. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Simplify use of cache in automationChris Evich2022-05-10
|/ | | | | | | | | | | | | | | | | With the increasingly complex `.cirrus.yml` task relationships, build cache wasn't always working as intended. Recently, non-build tasks were observed assuming authority over `gopath_cache`. Ref.: https://github.com/containers/podman/pull/13998#issuecomment-1108834538 Address this by an overall simplification using artifacts instead of cache. Using artifacts allows establishing concrete authorship/authority over cached repo. content. In this way, dependent tasks may simply consume the artifact with `curl` instead of relying on complex caching algorithms. Also/Minor: Add YAML checking to the pre-commit configuration. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix ownership of repos. to keep git happyChris Evich2022-04-27
| | | | | | | | | | | | | | | | | | Newer versions of git are much more pedantic about who owns the repository files. When setting up to run rootless, prior to this commit, the repo. ownership was changed from root. This causes all subsequent git-operations as root to fail: ``` fatal: unsafe repository ('<$GOSRC>' is owned by someone else) ``` Fix this by re-ordering operations, such that the change in ownership is done immediately before executing as a user. Also disable the git-ownership check on the source repository assuming the CI environment is disposable. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Test w/ netavark/aardvark-dns in F36+Chris Evich2022-04-27
| | | | | | | | | | | | | | | | | | | | Now that netavark and aardvark are packaged and default in F36, support CNI-based testing in F35 and Ubuntu. * Remove the temporary/special `$TEST_ENVIRON=host-netavark` construct. * Remove dedicated/special integration and system testing tasks. * Update test-config setup to properly handle CNI vs netavark/aardvark environments. * Update package-version logging to operate based on installed packages (along with some other minor script cleanups). * Update global environment setup to force `$NETWORK_BACKEND=netavark` in F36 and later. Except when `upgrade_test` task runs. * Discontinue installing netavark and aardvark-dns binaries from upstream build artifacts. * Drop CGV1-vs-2 policy check. Ubuntu VMs now exclusively test CGv1, Fedora VMs test CGv2, with F35 testing CNI and F36 testing Netavark. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Re-fix build-cache miss on mainChris Evich2022-04-25
| | | | | | | | | | | | | | After merging #13998 it was observed that the `docker-py` task was still failing with the same error on `main`. The original quick-fix had placed the full-build (`make`) call too late in the process. This commit moves it up to right before the `make install` call which was resulting in an error. Again, a further future commit is planned to re-work and simplify the entire cache setup. This is only a quick fix to make branch-builds pass. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix missing git-enforced runtime identityChris Evich2022-04-21
| | | | | | | | | Newer versions of git (like `2.35`) fail on certain operations (like `rebase` and `am`) without a local identity. Add a fake one from the start, with a clearly identifiable test-value to avoid problems at runtime. Signed-off-by: Chris Evich <cevich@redhat.com>
* Resolves #13629 Add RegistryAuthHeader to manifest pushjason2022-03-26
| | | | Signed-off-by: Jason Montleon <jmontleo@redhat.com>
* Add tests with Docker Compose v2Matthew Heon2022-03-17
| | | | | | | | | | | Add a pair of new Cirrus test suites using Compose v2 instead of Compose v1 (as is currently packaged in Fedora). They work identically, and run the same tests, as the Compose v1 tests, but with the new v2 binary instead. [NO NEW TESTS NEEDED] This adds an entire Cirrus suite... Signed-off-by: Matthew Heon <mheon@redhat.com>
* Move all python tests to pytestJhon Honce2022-03-04
| | | | | | | * Add configuration to add report header for python client used in tests * Move report headers into the individual test runners vs runner.sh Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Cirrus: Use updated VM imagesChris Evich2022-02-17
| | | | | | | | | | | Mainly this is to confirm some changes needed for the podman-py CI setup don't disrupt operations here. Ref: https://github.com/containers/automation_images/pull/111 Also includes a minor steup fix WRT setting up for test-rpm build. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add netavark/aardvark system test taskChris Evich2022-02-03
| | | | | | | Also add a system-test that verifies netavark driver is in use when magic env. var. is set. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #12814 from cevich/netavarkOpenShift Merge Robot2022-02-01
|\ | | | | Cirrus: Add e2e task w/ upstream netavark
| * Cirrus: Also download aardvark-dns binaryChris Evich2022-01-24
| | | | | | | | | | | | | | | | This involves a minor code-change so the download/install can run in a loop for the two different repositories and binaries. Given everything is exactly the same except the URLs and names. Signed-off-by: Chris Evich <cevich@redhat.com>
| * Cirrus: Add e2e task w/ upstream netavarkChris Evich2022-01-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds the CI mechanisms to obtain the latest upstream netavark binary, and set a magic env-var to indicate e2e tests should execute podman with `--network-driver=netavark`. A future commit implement this functionality within the e2e tests. Due to the way the new environment is enabled, the standard task name is too long for github to display without adding ellipsis. Force the custom task name `Netavark Integration` to workaround this. At some future point, when netavark is more mainstream/widely supported, this custom task and upstream binary install can simply be removed - i.e. netavark will simply be used by default in the normal e2e tasks. Signed-off-by: Chris Evich <cevich@redhat.com>
* | CI: fix nightly buildsLokesh Mandvekar2022-01-31
| | | | | | | | | | | | | | | | | | | | | | | | Nightly builds were failing on CI ever since the Makefile change to have install target independent of build targets. See: e4636ebdc84ca28cf378873435cc9a27c81756f8 This commit ensures everything is built before installation. [NO NEW TESTS NEEDED] Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
* | Makefile: install targets independent of buildLokesh Mandvekar2022-01-25
|/ | | | | | | | | | | | | | Building from source would involve separate `make` and `make install` steps. This removes a lot of unnecessary `-nobuild` targets which were otherwise needed for packaging. This commit also removes spec files for unused copr jobs. [NO NEW TESTS NEEDED] Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
* CI: rootless user: also create in some root testsEd Santiago2022-01-17
| | | | | | | | | | | | viz, rootful system tests. The rootless account will be used by image-scp tests. Unfortunately, having ssh available means the system-connection tests will start running, which is very bad because they will fail, because system connection doesn't actually work (long story). Add a few more checks to prevent this test from running. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: Freshen VM imagesChris Evich2022-01-10
| | | | | | | | | Specifically, this brings in `crun 1.4-1` allowing removal of a temporary workaround. Ref: https://github.com/containers/podman/pull/12759 Signed-off-by: Chris Evich <cevich@redhat.com>
* Fix CIDaniel J Walsh2022-01-06
| | | | | | | | crun should be available in f35. [ NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* ci: force scratch build for crunGiuseppe Scrivano2021-12-21
| | | | | | force a version with this fix: https://github.com/containers/crun/pull/819 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Cirrus: Bump Fedora to release 35Chris Evich2021-11-18
| | | | | | | | | | | The Fedora 35 cloud images have switched to UEFI boot with a GPT partition. Formerly, all Fedora images included support for runtime re-partitioning. However, the requirement to test alternate storage has since been dropped/removed. Rather than maintain a disused feature, and supporting scripts, these Fedora VM images have reverted to the default: Automatically resize to 100% on boot. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Partially revert catatonit --force installChris Evich2021-11-18
| | | | | | | | VM Images created as of this commit contain the new/required version. Remove the `--force` install, but retain the hack script's ability to support this in the future. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Workaround log_driver=journald settingChris Evich2021-11-18
| | | | | | | | | | In F35 the hard-coded default (from containers-common-1-32.fc35.noarch) is 'journald' despite the upstream repository having this line commented-out. Containerized integration tests cannot run with 'journald' as there is no daemon/process there to receive them. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: force-install catatonitValentin Rothberg2021-11-15
| | | | | | A temporary workaround until the CI images are updated. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* rootlessport: reduce memory usage of the processPaul Holzinger2021-10-12
| | | | | | | | | | | | | | | | | | | | | | Don't use reexec for the rootlessport process, instead make it a separate binary to reduce the memory usage. The problem with reexec is that it will import all packages that podman uses and therefore loads a lot of stuff into the heap. The rootlessport process however only needs the rootlesskit library. The memory usage is a concern since the rootlessport process will spawn two process per container which has ports forwarded. The processes stay until the container dies. On my laptop the current reexec version uses 47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is more than a 90% improvement. The Makefile has been updated to compile the new binary and install it to the libexec directory. Fixes #10790 [NO TESTS NEEDED] Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Cirrus: Fix defunct package metadata breaking cacheChris Evich2021-10-01
| | | | | | | | | | | | | | | | | | | | | Original workaround https://github.com/containers/podman/pull/11821 During VM image build, a number of packages are downloaded but not installed, since they may interfere with some testing. Then at runtime, where required, the packages are installed from cache and used. However, between image build and runtime it's possible the repository contents change, which will invalidate the package cache. Since the `--no-download --ignore-missing` options were used, the install will fail. Ref: https://github.com/containers/automation_images/issues/95 Fortunately, when it comes to the docker packages, no other dependencies are required and so `apt-get` isn't required. Switch to using a simple dpkg install command on the necessary files. If this ever breaks due to new dependencies, the list of files may simply be updated. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: gitlab: download packagesValentin Rothberg2021-10-01
| | | | | | | | | | It looks like the containerd.io package is not present anymore in the package cache which ultimately breaks CI since it's a requirement for docker. Hence, download the few packages instead of relying on the cache. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Cirrus: Add gitlab podman runner testChris Evich2021-09-27
| | | | | | | | | | Add execution of the downstream gitlab-runner tests using rootless podman through the magic of socket-level docker compatibility. Include a comment suggesting how to temporarily disable the test in case it fails beyond podman code scope. Signed-off-by: Chris Evich <cevich@redhat.com>
* CI: load ipv6 kernel modules for rootless testsPaul Holzinger2021-09-15
| | | | | | | | Rootless cni with ipv6 needs the `ip6_tables` module loaded, normally the cni plugins will load this module but as rootless it does not have the necessary permission to do so. Therefore we load it manually. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Cirrus: Confirm CGv1 / CGv2 VM expectationsChris Evich2021-08-18
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Fix AVC denials in tests of volume mountsChris Evich2021-08-18
| | | | | | | | | This becomes a problem on hosts with upgraded policies. Ref: https://github.com/containers/podman/issues/10522 Also, made a small change to compose-test setup to reduce runtime. Signed-off-by: Chris Evich <cevich@redhat.com>
* Enable docker-py compat. testing w/ ignored resultChris Evich2021-08-09
| | | | | | | | | | Significant bitrot results in almost immediate test failure. This commit adds only the very basic, bare-minimum needed to get them started. ***TESTING RESULTS ARE IGNORED*** Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Use Fedora 34beta imagesChris Evich2021-04-07
| | | | | | Also, revert 4875a8fb Signed-off-by: Chris Evich <cevich@redhat.com>
* [NO TESTS NEEDED] Turn on podman-remote build --isolationDaniel J Walsh2021-03-26
| | | | | | | | | | Currently podman only works with --isolation chroot. This PR fixes this by allowing the isolation mode to default to OCI and to also allow users to pass the isolation mode into the containers. The current tests for --isolation should cause this code to be tested. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* WIP: run buildah bud tests using podmanEd Santiago2021-03-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set of scripts to run buildah's bud.bats test using podman build in podman CI. podman build is not 100% compatible with buildah bud. In particular: * podman defaults to --layers=true; buildah to false * podman defaults to --force-rm=true; buildah to false * podman error exit status is 125; buildah is 2 * differences in error messages, command-line arguments Some of the above can be dealt with programmatically, by tweaking the buildah helpers.bash (BATS helpers). Some need to be tweaked by patching bud.bats itself. This PR includes a patch that will, I fear, need to be periodically maintained over time. There will likely be failures when vendoring in a new buildah, possibly because new tests were added for new features that don't exist in podman, possibly (I hope unlikely) if existing tests are changed in ways that make the patch file fail to apply. I've tried to write good instructions and to write the run script in such a way that it will offer helpful hints on failure. My instructions and code will be imperfect; I hope they will be good enough to merit continued use of this test (possibly with improvements to the instructions as we learn more about real-world failures). Signed-off-by: Ed Santiago <santiago@redhat.com>
* podman upgrade testsEd Santiago2021-02-23
| | | | | | | | Initial validation of using podman-in-podman to create an old-podman root, then use new-podman to play with the containers created therein. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: Fix running Validate task on branchesChris Evich2021-01-27
| | | | | | | | | | | | | | | | | | | | | | Followup to dbb9943 Despite skipping the `Smoke` check, it was observed on a *new* branch, the `validate` task (specifically `git-validation`) will fail. This is because: * `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch. * `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push * `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`. Fix this by eliminating the `Smoke` task entirely, simplifying all the `make validate` operations into the `validate` cirrus task. Ensure this task does not run when a new branch or tag is pushed. Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not actually used anywhere. It was formerly used for building VM images, but this has moved to another repo entirely. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Upload swagger YAML in every contextChris Evich2021-01-21
| | | | | | | | | | | | | | | | | | | The podman documentation site uses javascript to display API documentation at: http://docs.podman.io/en/latest/Reference.html As input, the javascript sources from a CORS-enabled Google Cloud Storage object. This commit ensures the storage object is present and updated for every Cirrus-CI execution context: Tags, Branches, and PRs. As of this commit, the documentation site only utilizes the object uploaded by the Cirrus-CI run on the `master` branch: `swagger-master.yaml`. The file produced and uploaded due to a PR is intended for testing purposes: Confirm it's generation and uploading are both functional. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: add bindings checksValentin Rothberg2021-01-20
| | | | | | | | | Make sure that bindings are in sync with the code. The check is similar to what's already being done with `make vendor`, so integrate the two. [NO TESTS NEEDED] Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Cirrus: Add support for Ubuntu 20.xChris Evich2020-12-15
| | | | | | | | | | | | | | | | | | | | | | | | | Previously automation always dropped the minor version number for distributions. This was intended for presentation and conditional simplicity. Bash does not support non-integer comparison natively. With the release of version 20.10, supporting testing with it and the LTS release (20.04) requires scripts to consider minor version numbers for Ubuntu VMs. This is necessary because many times in the past, some behaviors needed to be conditional on the release version number. With this commit, the images and embedded scripts/tooling uses an altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially) `$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing in the official version (from `/etc/os-release`) are dropped, and the result is concatenated. For example the current Ubuntu LTS version is `20.04`. Prior to this commit, `$OS_RELEASE_VER` would have been `20`. With this change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10` is shown as `2010`. Signed-off-by: Chris Evich <cevich@redhat.com>
* add compose regression to cibaude2020-12-11
| | | | | | | to prevent any regressions, we should be running regression tests using compose. Signed-off-by: baude <bbaude@redhat.com>
* Fix storage.conf to define driver in the VMDaniel J Walsh2020-12-09
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Use F33beta VM imageChris Evich2020-11-04
| | | | | | | | | | | | Includes disk-space increase for all Fedora images to accommodate the static-build job disk space requirements. This job substantially leverages task-cache, which was previously failing to restore early on in the Cirrus-CI task setup, due to disk-space limitations. Also simplify .cirrus.yml slightly by removing an unncessary setup and run directory change step. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Workaround F32 BFQ Kernel bugChris Evich2020-10-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #8046 from cevich/simplify_environmentOpenShift Merge Robot2020-10-29
|\ | | | | Cirrus: Simplify setting/passing env. vars.
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-10-08 23:21:43 +0000