summaryrefslogtreecommitdiff
path: root/contrib/cirrus
Commit message (Collapse)AuthorAge
* Github-Actions: Send e-mail on Cirrus cron failureChris Evich2020-11-18
| | | | | | | | | | | | | | | This repository has a number of automaticly triggered branch-level testing enabled. However, other than remembering to go look at a specific WebUI, there is no way for anybody to notice if/when these jobs fail. This commit introduces a github-action workflow which runs periodically, checking for failed cron-triggered Cirrus-CI jobs. When it finds any, it formats a simple report for e-mail delivery. The list of destination addresses is configurable at any time by merging changes to a simple CSV file. Signed-off-by: Chris Evich <cevich@redhat.com>
* logformatter: highlight timing resultsEd Santiago2020-11-16
| | | | | | | | Add a :hover style to rows in the 'integration timing results' section. Without that, it's really hard for my eye to scan across and match a time to a test name. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Merge pull request #8107 from cevich/measure_testing_statsOpenShift Merge Robot2020-11-12
|\ | | | | Cirrus: Collect runner.sh stats
| * Cirrus: Detailed CPU/Memory/Time runner.sh statsChris Evich2020-11-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On several occasions, there have been questions about CPU/Memory/IO trends in testing over time. Start collecting this data for all jobs, using a common/stable format so that trending analysis can be performed within/across multiple Cirrus-CI builds. This PR doesn't add any related tooling, it simply arranges for the collection of the data. Stats generation is done at the orchestration level to guarantee they reflect everything happening inside `runner.sh`. For example, the container-based tests re-exec `runner.sh` inside a container, but we're only interested in the top-level stats. Update all tasks to include collection of the stats file. Unfortunately, due to the way the Cirrus-CI YAML parser works, it is *not* possible to alias the artifacts collection more clearly, for example: ```yaml always: <<: *runner_stats <<: *logs_artifacts ``` Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: update VMsEd Santiago2020-11-11
| | | | | | | | | | | | | | | | | | Reason: include perl-FindBin RPM in f33 VM, needed for hack/xref-helpmsgs-manpages Ref: https://github.com/containers/automation_images/pull/39 Signed-off-by: Ed Santiago <santiago@redhat.com>
* | [CI:DOCS] Restore man page cross-checkerEd Santiago2020-11-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Somewhere in the CIv2 migration we lost the man page vs --help cross-checker. Add it back, by adding it into the man-page-check Makefile target; this is part of 'make validate', which is run in CI even on CI:DOCS PRs. As happens when CI doesn't run, things broke. Man pages got out of sync with --help. This PR: 1) Fixes hack/xref-helpmsgs-manpages to deal with the new "Options" (instead of "Flags") form of podman help. #8034 did part of that, but one of my review comments was accidentally left out. 2) Fixes hack/xref-helpmsgs-manpages to deal with the new option syntax in man pages, post- #8292, in which each option is preceded by four hashes so as to make them HTML <h4> elements with named anchors. 3) Fixes man pages that #8292 accidentally missed. 4) Adds man page entries for two flags that got added to podman but not documented (pod create --network-alias, play kube --log-driver) Fixes: #8296 Signed-off-by: Ed Santiago <santiago@redhat.com>
* | Update CI tests to run python docker library against APIJhon Honce2020-11-09
|/ | | | | | | | | | | * Update reference to docker-py to docker to reflect change in library name * Update tests to create storage sandbox * Enable all tests that endpoints support * Refactor containers/{id}/rename to return 404 not 500 * Refactor tests to use quay.io vs. docker.io Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Cirrus: Use F33beta VM imageChris Evich2020-11-04
| | | | | | | | | | | | Includes disk-space increase for all Fedora images to accommodate the static-build job disk space requirements. This job substantially leverages task-cache, which was previously failing to restore early on in the Cirrus-CI task setup, due to disk-space limitations. Also simplify .cirrus.yml slightly by removing an unncessary setup and run directory change step. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Workaround F32 BFQ Kernel bugChris Evich2020-10-30
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #8046 from cevich/simplify_environmentOpenShift Merge Robot2020-10-29
|\ | | | | Cirrus: Simplify setting/passing env. vars.
| * Cirrus: Simplify setting/passing env. vars.Chris Evich2020-10-29
| | | | | | | | | | | | | | | | | | | | | | | | Test VMs by design are to be single-purpose, single-use, and readily disposable. Therefore it's unnecessary to overcomplicate storage of runtime environment variables. This commit makes these points clear, and reorganizes all CI-related env. vars on the system into a single location, `/etc/ci_environment`. This file is then automatically loaded, and variables exported, (by `lib.sh`) from `runner.sh` prior to executing all forms of testing. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Use google mirror for docker.ioChris Evich2020-10-28
| | | | | | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Always record runc/crun versionsChris Evich2020-10-28
|/ | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Add a system test to verify --runtime is preservedMatthew Heon2020-10-20
| | | | Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Use runtime names instead of paths in E2E testsMatthew Heon2020-10-20
| | | | | | | | | | | | | | | | | | My patches to fix `--runtime /usr/bin/crun` being allowed to use a different version of the crun runtime revealed a problem: we were actually relying on that exact behavior in our E2E tests. We specified the runtime path as `/usr/bin/runc` for the Ubuntu tests, but that didn't exist, so Podman was actively looking for a different, usable runc binary and using that, instead of the path we explicitly hardcoded. Fixing the bug broke this, and thus broke the tests. Instead of hard-coding OCI runtime paths, swap to just using the runtime name, `runc` or `crun`, and letting Podman figure out where the runtime lives - it's quite good at that. This should un-break the tests and make them more durable. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Cirrus: Restore APIv2 TestingChris Evich2020-10-13
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix obtaining a CI VMChris Evich2020-10-07
| | | | | | | | | Also removed automatic exection of setup_environment.sh since most people using this script are podman developers (not automation/CI folks). If executing the automation scripts is necessary, manual attendance to required variables like `$TEST_FLAVOR` is mandatory. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix running shellcheck locallyChris Evich2020-10-06
| | | | | | Also, check the contents of hack/get_ci_vm.sh Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus CI runner: refactorEd Santiago2020-10-06
| | | | | | | | | | | | | | | While reviewing #6784 I found myself having a lot of trouble with this script: it was a complicated mix of case statement and helper functions, requiring a reader to jump back and forth between the two. This PR defines a convention such that a given TEST_FLAVOR=foo must have a corresponding _run_foo() handler function. The goal is to have all TEST_FLAVOR-related code in one place, or at least less scattered (integration and system tests still rely on other helper functions). Signed-off-by: Ed Santiago <santiago@redhat.com>
* logformatter: run on system tests & bindingsEd Santiago2020-10-05
| | | | | | | | | | | | (that got accidentally dropped in the new Cirrus makeover). Note that 'dotest' does not actually 'do tests', it's only used for a small subset of tests. Also, make logformatter work better in the new Cirrus setup. Remove duplicate test/subtest, remove no-longer-used SPECIALMODE, and make the Cirrus build/task display a little cleaner. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: Fix branch-validation failureChris Evich2020-10-05
| | | | | | | | | | | When validating code on a branch, determining a starting commit to check from isn't as straightforward as it would seem. Default to using the SHA from last time CI was green. If for some reason that isn't available, use an obviously wrong value to cause an intentional failure. Entirely skip this check on tag-push, since determining a starting point is incredibly difficult to do automatically. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Implement podman automation 2.0Chris Evich2020-10-02
| | | | | | | | | | | | | | | | | | Reimplement CI-automation to remove accumulated technical-debt and optimize workflow. The task-dependency graph designed goal was to shorten it's depth and increase width (i.e. more parallelism). A reduction in redundant building (and 3rd party module download) was also realized by caching `$GOPATH` and `$GOCACHE` early on. This cache is then reused in favor of a fresh clone of the repository (when possible). Note: The system tests typically execute MUCH faster than the integration tests. However, contrary to a fail-fast/fail-early principal, they are executed last. This was implemented due to debug-ability related concerns/preferences of the primary (golang-centric) project developers. Signed-off-by: Chris Evich <cevich@redhat.com>
* logformatter: add Synopsis at top of each pageEd Santiago2020-10-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At the top of each generated page, add a Synopsis table with: PR number/name, and link to github Author name(s) Test name (fedora/ubuntu, rootless, etc) Cirrus build ID (usually uninteresting) Cirrus task ID (usu. important), with link to Cirrus The value of $SPECIALMODE This is all we can get from the Cirrus environment in which logformatter runs; we can't get things like cgroup manager or username that the test runs under. Note that the table is at the top, which is usually unseen because we autoscroll to the bottom on page load. I tentatively think that top is a more natural place for this info than bottom, but am willing to listen to arguments against. Also, one minor tweak: highlight podman commands in the BATS output. The idea is to make it easier for the eye to spot those, then copy/paste them to find a reproducer. And, sigh, disable the new 'podman network create' system test. It is flaking much too much. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Fix up errors found by codespellDaniel J Walsh2020-09-11
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Obsolete CI:IMG process & related filesChris Evich2020-09-09
| | | | | | | | | | | All VM-building functionality has been migrated to https://github.com/containers/automation_images Some container-build functions are still maintained here but are on a very-short list to also be migrated to the repository linked above. Signed-off-by: Chris Evich <cevich@redhat.com>
* WIP: update VM imagesEd Santiago2020-09-03
| | | | | | | | | | | | | | | | (This is an adoption of #7533 because Brent is on PTO). Pick up new crun and crio-runc. Also: renames from useful fedora-32 and -31 to less-useful names; presumably this is needed by something-something in the new VM setup. Also: tweak two e2e tests to more properly handle a kernel (5.8.4) with a greater set of capabilities than what we or crun can yet handle. Signed-off-by: Ed Santiago <santiago@redhat.com>
* use crio runc on CICID ubuntuBrent Baude2020-08-31
| | | | | | | when running CICD on Ubuntu where no cgroups v2, we need to use a newer runc for things like seccomp and the default ubuntu runc is not new enough. Signed-off-by: Brent Baude <bbaude@redhat.com>
* Use `bash` binary from env instead of /bin/bash for scriptsSascha Grunert2020-08-17
| | | | | | | | It's not possible to run any of the scripts on distributions which do have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env bash` instead. Signed-off-by: Sascha Grunert <sgrunert@suse.com>
* Remove TEST_REMOTE_CLIENT from RCLIDaniel J Walsh2020-08-10
| | | | | | | | We know these are TEST_, hoping this makes the display in cirrus easier for users to see true|false, since this is the valuable information is. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Cirrus: Install golang 1.14 on UbuntuChris Evich2020-07-31
| | | | | | | This more/less reverts efd142214 + updates to 1.13 on all Ubuntus for all `containers` projects. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add python packages to imagesChris Evich2020-07-31
| | | | | | | | | They are needed in support of future testing additions. Also reduce unnecessary output by not printing the downloaded package list. The set can be examined using other tooling if/when necessary. Signed-off-by: Chris Evich <cevich@redhat.com>
* System tests: new system-df and passwd testsEd Santiago2020-07-30
| | | | | | | | | | | | | | | | | | | | | | - New test for #6991 - passwd file is writable even when run with --userns=keep-id - Enable another keep-id test, commented out due to #6593 - New test for podman system df Also, independently, removed this line: apt-get -y upgrade conmon ...because it's causing CI failures, probably because of the boothole CVE, probably because the Ubuntu grub update was rushed out. I believe it is safe to remove this, because both Ubuntu 19 and 20 report: conmon is already the newest version (2.0.18~1). Signed-off-by: Ed Santiago <santiago@redhat.com>
* logformatter: more libpod-podman falloutEd Santiago2020-07-29
| | | | | | | | | | | Problem: formatted logs no longer have live links to sources in error-report lines. Cause: script was searching for '/libpod'. Solution: make it more flexible. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Switch all references to github.com/containers/libpod -> podmanDaniel J Walsh2020-07-28
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* CI - various fixesEd Santiago2020-07-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Primary purpose: upgrade crun to 0.14 on f31, in hopes of eliminating the 'cgroups.freeze' flake that is plaguing CI. While I'm at it: - remove a no-longer-needed dnf upgrade that was running in CI itself (not image building, in each actual CI run). The purpose was to upgrade conmon, but that was added a long time ago and the required conmon is now in stable. The effect of this dnf upgrade today was simply to cause flakes when fedora repos were offline. - remove a no-longer-needed check for varlink. - networking.sh : add a timeout! 'openssl s_client' will happily hang forever if a host is unreachable, which means we waste two hours waiting for Cirrus to time out. - timestamp.awk : include date (not just time) in START/END msgs. There are times when I'm looking at a CI log and it is ultra important to know if it is from yesterday or today. - add progress messages in some places where I've previously struggled to understand context in logs; and improve some unlikely error messages to include script name. ...then, after all that, wrote a new README about how to to all this. Hope it helps someone. Signed-off-by: Ed Santiago <santiago@redhat.com>
* CI: fix rootless permission errorEd Santiago2020-07-23
| | | | | | | | | | | | | CI runs are failing in special_testing_rootless: mkdir /var/tmp/go/pkg: permission denied Probable cause: #6822, which universally set GOPATH. Solution: in rootless setup, chown -R GOPATH as well as GOSRC (the latter was already being chowned). Signed-off-by: Ed Santiago <santiago@redhat.com>
* Merge pull request #7050 from edsantiago/logformat_trim_remoteOpenShift Merge Robot2020-07-22
|\ | | | | logformatter: handle podman-remote
| * logformatter: handle podman-remoteEd Santiago2020-07-22
| | | | | | | | | | | | | | | | | | | | Oops! Logs of podman-remote tests are unreadable, they have multiple (useless) --remote options plus '--url /something/long' that makes it impossible to read the actual command being run. This commit strips off '--remote' entirely, and hides '--url' and its arg in the only-on-mouse-hover '[options]' text. Signed-off-by: Ed Santiago <santiago@redhat.com>
* | Cirrus: Add packages that provide htpasswdChris Evich2020-07-22
| | | | | | | | | | | | | | | | | | Mainly needed for buildah testing: the htpasswd command was removed from the upstream registry container image. Making it available on the host-side enables configuring details needed by the registry during it's initial setup. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Ensure GOPATH is properly set during image-buildsChris Evich2020-07-22
|/ | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #6992 from rhatdan/apparmorOpenShift Merge Robot2020-07-22
|\ | | | | Support default profile for apparmor
| * Support default profile for apparmorDaniel J Walsh2020-07-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently you can not apply an ApparmorProfile if you specify --privileged. This patch will allow both to be specified simultaniosly. By default Apparmor should be disabled if the user specifies --privileged, but if the user specifies --security apparmor:PROFILE, with --privileged, we should do both. Added e2e run_apparmor_test.go Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* | logformatter: update MAGIC BLOB stringEd Santiago2020-07-20
|/ | | | | | | | | | | | | | | | | | | | | Fallout from libpod->podman repo name move: the HTML logs created by logformatter are no longer accessible. They render as: https://storage.googleapis.com/SECRET-5385732420009984-fcae48/artifacts/containers/podman/6313596734930944/html/integration_test.log.html (yes, "SECRET" instead of "cirrus-ci". Possibly because the GCE_SSH_USERNAME key, "cirrus-ci", was overzealously encrypted, making Cirrus censor any instances of the string in output. Let's see if this fixes it. But anyway this is a secondary unrelated bug). Reason: it looks like Cirrus "generated a new magic blob" when we renamed libpod -> podman. Chris was kind enough to locate the new magic blob and to give me a link to where we can discover it ourselves. I added that as a code comment. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Fix systemd pid 1 testBrent Baude2020-07-13
| | | | | | | | | | fedora removed the systemd package from its standard container image causing our systemd pid1 test to fail. Replacing usage of fedora to ubi-init. adding ubi images to the cache for local tests. also, remove installation of test/policy.json to the system wide /etc/containers Signed-off-by: Brent Baude <bbaude@redhat.com>
* system tests: invoke with abs path to podmanEd Santiago2020-06-23
| | | | | | | | | | | | | | | | | | | | | | Reversion of one part of #6679: my handling of 'realpath' would not work when $PODMAN is 'podman-remote --url etc'. Trying to handle that case got unmaintainable; so instead let's just force 'make {local,remote}system' to invoke with a full PODMAN path. This breaks down if someone runs the tests with a manual 'bats' invocation, but I think I'm the only one who ever does that. Since podman path will now be very long in the logs, add code to logformatter to abbreviate it like we do for the ginkgo logs. And, one thing that has bugged me for a long time: in the error logs, show a different prompt ('#' vs '$') to distinguish root vs rootless. This should make it much easier to see at-a-glance whether a log file is root or not. Add tests for it. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Do not share container log driver for execMatthew Heon2020-06-17
| | | | | | | | | | | | | | | | | | | | | When the container uses journald logging, we don't want to automatically use the same driver for its exec sessions. If we do we will pollute the journal (particularly in the case of healthchecks) with large amounts of undesired logs. Instead, force exec sessions logs to file for now; we can add a log-driver flag later (we'll probably want to add a `podman logs` command that reads exec session logs at the same time). As part of this, add support for the new 'none' logs driver in Conmon. It will be the default log driver for exec sessions, and can be optionally selected for containers. Great thanks to Joe Gooch (mrwizard@dok.org) for adding support to Conmon for a null log driver, and wiring it in here. Fixes #6555 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Merge pull request #6530 from edsantiago/test_podman_remoteOpenShift Merge Robot2020-06-10
|\ | | | | Enable, then partially disable, podman-remote testing
| * WIP: Enable (and disable) remote testingEd Santiago2020-06-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | podman-remote has not been tested. A principal part of the problem was #5387 - the YAML I wrote did not have the intended effect, it did not set TEST_REMOTE_CLIENT=true and because of my multiple iterations I did not catch this during testing. Part 1 of this PR is to fix .cirrus.yml to enable remote tests. Part 2 -- what I had first noticed and tried to fix -- is that rootless_test.sh was never running remote because, of course, envariables are not sent via ssh. I reworked integration_test.sh and rootless_test.sh to use a command-line decision instead. Part 3, sigh, is to disable one failing integration test and *all* system tests, because so many of the latter are failing. Addressing those failures needs to be done in subsequent PRs. Issues #6538, #6539, #6540 are filed for some of the problems I isolated. There will be more. Also, minor, fixed some stale references to varlink. Signed-off-by: Ed Santiago <santiago@redhat.com>
* | Merge pull request #6193 from cevich/conmon_ci_packagesOpenShift Merge Robot2020-06-09
|\ \ | |/ |/| Cirrus: Include packages for containers/conmon CI
| * Cirrus: Include packages for containers/conmon CIChris Evich2020-06-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the containers/conmon repository to share the same VM images produced by containers/libpod. Included are several packages which are downloaded only since they might otherwise interfere with testing for some repos. This allows stable versions to be at the ready at testing runtime, avoiding any version updates surprising developers. Also, re-enable running the VM-image check test which was not working due to a logic problem in Cirrus-CI configuration. Update the neglected tests so that they pass on all distros. Signed-off-by: Chris Evich <cevich@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-16 16:18:19 +0000