summaryrefslogtreecommitdiff
path: root/contrib/cirrus/lib.sh
Commit message (Collapse)AuthorAge
* Merge pull request #4222 from cevich/collect_varlink_logOpenShift Merge Robot2019-10-09
|\ | | | | Cirrus: Produce and collect varlink output
| * Cirrus: Produce and collect varlink outputChris Evich2019-10-08
| | | | | | | | | | | | | | | | | | | | | | | | | | When executing 'make remotesystem' testing, a varlink process is started up but it's stdio is dumped due to the production of excessive data. However, this also means if the process has a problem, any errors will not be accessible. Instead, grab only the last 100 lines and direct them into a file. Also update automation's log collection to retrieve this file when the `$REMOTE_CLIENT` env. var. is `true`. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Install conmon in Fedora VMsChris Evich2019-10-07
|/ | | | | | | | | | This is needed because older versions of podman (1.5.1) do not automatically install the new conmon package. Also, include removal of `/usr/libexec/podman/conmon` when preparing to install and test podman built from source. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Disable boottime Ubuntu package updateChris Evich2019-09-26
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: disable Evil Units in base-imagesChris Evich2019-09-20
| | | | | | Also, minor update to prevent harmless 'Fatal: not a git repo' error. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add latest ubuntuChris Evich2019-09-20
| | | | | | | | | | | | Add the latest Ubuntu version into the testing matrix and image-build workflow. This is also needed to support other containers projects which share use of VM images from this one. Update package lists to include needs for contianers/storage use of images. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Prevent resident pollutionChris Evich2019-09-10
| | | | | | | | | | | | | When constructing VM cache-images, the latest/greatest podman package is installed to ensure all necessary dependencies are met. Prior to testing source-built binaries, most of of the packaged files are removed. However, if the `io.podman` service or socket is enabled/running, it could cause the packaged podman and varlink binaries to be both resident and cached. Since this condition would cause very difficult to diagnose behaviors, add preventative measures to ensure these services are absent prior to removing packaged podman files. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: On success, add IRC nick mention to msgChris Evich2019-08-30
| | | | | | | | | | | Rather than spamming the podman channel with impersonal success messages referring to PR numbers, mention the author by nick name and include the PR title and link. Also avoid needless logging of all bot-script interactions with IRC when there is no error detected. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #3710 from cevich/release_redoOpenShift Merge Robot2019-08-29
|\ | | | | Release redo
| * Cirrus: Reimplement release archive + uploadChris Evich2019-08-28
| | | | | | | | | | | | | | | | The initial implementation was far more complicated than necessary. Strip out the complexities in favor of a simpler and more direct approach. Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Block CNI use of google VPCsChris Evich2019-08-28
|/ | | | | | | | | | | | | | | | | | | | | | | | | | Specifically pertaining to executing tests in google cloud, there are default, pre-allocated class-a subnetworks for each region (data-center). Each includes a gateway using a `.1` LSB and all are routable from other regions in google cloud via these gateways. Because the default CNI configuration also utilizes class-a subnetworks, this creates the possibility for IPv4 address-space clashes. Since the default regional cloud subnets are pre-defined/known, preventing clashes can be accomplished by seeding these subnets in a dummy CNI configuration. The default behavior of podman is to grab the highest priority CNI configuration. Name the dummy config. appropriate so it always loads last. Also name the bridge itself with an obvious name `do-not-use`, such that any related testing errors should be easier to debug. Also: * Minor cleanup of `install_test_configs()` * Move install_test_configs in `setup_environment.sh` to after possible run of `remove_packaged_podman_files()` because that also strips out `/etc/cni/net.d/87-podman-bridge.conflist`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add experimental fedora VM image & testChris Evich2019-08-12
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Minor, use newer Ubuntu base imageChris Evich2019-08-12
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add Second partition for storage testingChris Evich2019-08-01
| | | | | | | | | | This is mainly/initially to support use of Cirrus-CI in https://github.com/containers/buildah since that setup re-uses the VM images from this project. However, it also opens doors here, if libpod ever needs/wants to do things with a dedicated storage device and/or storage-drivers. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Silence systemd-banish noiseChris Evich2019-07-25
| | | | | | | | | | It's somewhat hard to predict which units are certinly present for any given base-image. Therefore, at image-build time, it's distracting and unhelpful to see all the errors about units that don't exist, on every platform. Simply ignore them and rely on the `check_image.sh` test to confirm none are enabled. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add image-test for locked dpkgChris Evich2019-07-16
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Disable most periodic services/timersChris Evich2019-07-16
| | | | | | | | | | For CI testing, it's important to remove as much variability from the overall system as possible. This permits focusing just on problems closely related to code-changes. To this end, and because VMs are very short-lived (2 hours at most), disable all systemd services and timers which perform periodic activities. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Abstract destination branch refs.Chris Evich2019-07-15
| | | | | | | | | | | | | | | | | Various tasks and scripts behave differently depending on whether or not the build is running against a PR or on a branch, post-merge. However, a great number of them are hard-coded to the string 'master' as the destination. Since this is not always the case (there are other relevant branches), it makes sense to abstract the references with a single definition. Add a top-level `$DEST_BRANCH` variable to CI, and otherwise default to 'master' when unset. This enables running CI builds on additional branches without the overhead of updating all the static references to 'master'. Simply update `$DEST_BRANCH` at the top-level and all branch-conditional logic will function as intended. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix missing removal of packaged podmanChris Evich2019-07-12
| | | | | | This was originally intended, but somehow omitted from #1936 Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #3106 from cevich/cirrus_releaseOpenShift Merge Robot2019-07-10
|\ | | | | Cirrus: Automate releasing of tested binaries
| * Cirrus: Automate releasing of tested binariesChris Evich2019-07-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's desirable to make archives available of builds containing actual tested content. While not official distro-releases, these will enable third-party testing, experimentation, and development for both branches (e.g. "master") and pull requests (e.g. "pr3106"). * Add a Makefile targets for archiving both regular podman binaries and the remote-client. Encode release metadata within these archives so that their exact source can be identified. * Fix bug with cross-compiling remote clients for the Windows and Darwin platforms. * Add unit-testing of cross-compiles for Windows and Darwin platforms. * A few small CI-script typo-fixes * Add a script which operates in two modes: 1. Call Makefile targets which produce release archives. Upload the archive to Cirrus-CI's built-in caching system using reproducible cache keys. 2. Utilize reproduced cache keys to attempt download of cache from each tasks. When successful, parse the file's release metadata, using it to name the archive file. Upload all recovered archives to a publicly accessible storage bucket for future reference. * Update the main testing task to call the script in mode #1 for all primary platforms. * Add a new `$SPECIALMODE` task to call the script in mode #1 for Windows and Darwin targets. * Add a new 'release' task to the CI system, dependent upon all other tasks. This new tasks executes the script in mode #2. * Update CI documentation Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Use packaged dependenciesChris Evich2019-06-27
|/ | | | | | | | | | | | | | | | | | Building/installing dependencies from fixed source-version ensures testing is reliable, but introduces a maintenance burden and risks testing far outside of a real-world environment. The sensible alternative is to install dependencies from distro-packaging systems. Install all development and testing dependencies at VM cache-image build time, to help ensure testing remains stable. The existing cache-image build workflow can be utilized at any future time to build/test with updated packages. ***N/B***: This does not update any dockerfiles used by testing, that is left up to future efforts. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: More tests to verify cache_imagesChris Evich2019-06-25
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Fix F30 ssh guaranteeChris Evich2019-06-14
| | | | | | | The original solution using --wait does not function on F30, waiting forever. Replace it with a simple 5-minute timeout loop. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add support for testing F30Chris Evich2019-06-14
| | | | | | | | | | | | | | | | Remove disused `build_cache_images` task, and update relevant dockerfiles for F30. Fix problem of cloud-init failing to expand root-device on boot (/var/lib/cloud/instance left in improper state). Fix problem of cloud-init racing with google-network-daemon.service on boot (looking for cloudconfig metadata too early). Causing root-device to _sometimes_ fail to expand. Fix problem of hack/get_ci_vm.sh argument passing. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Guarantee ssh is running for rootlessChris Evich2019-06-13
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* split rootless local and remote testingbaude2019-05-31
| | | | Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: Stub in F30 supportChris Evich2019-05-21
| | | | | | | | | | | New base-image boots, a cache-image builds, but more work is needed for it to be prime-time ready. This commit just adds some updates to the scafolding necessary to build the base-image. Future work will make F30 more of a reality. Also add log-collection scripts to test image verification task Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: fixups based on review feedbackChris Evich2019-05-21
| | | | | | Also remove disused distros (RHEL/CentOS/FAH) and fix get_ci_vm script Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Overhaul/Simplify env. var setupChris Evich2019-05-21
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Run tests on test-built cache-imagesChris Evich2019-05-21
| | | | | | Also, add jq and catatonit installs to images. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Support testing of VM cache-image changesChris Evich2019-05-21
| | | | | | | | | | | | Previously, it was quite difficult to affect changes to VM cache images without lots of manual work. This commit adds a new optional testing task which mirrors the official-image build task which only runs on master. In contrast, the new task may be run at any time in a PR, but including a magic phrase in the PR description. Update documentation to describe the new task and inform on it's usage. Signed-off-by: Chris Evich <cevich@redhat.com>
* Use containers/conmonPeter Hunt2019-05-17
| | | | | | There were some build issues updating cri-o to cri-o/cri-o. Since the only thing we need cri-o for is conmon, we should just build using conmon. Signed-off-by: Peter Hunt <pehunt@redhat.com>
* s|kubernetes-sigs/cri-o|cri-o/cri-o|gPeter Hunt2019-05-17
| | | | Signed-off-by: Peter Hunt <pehunt@redhat.com>
* Cirrus: timestamp all output script outputChris Evich2019-05-02
| | | | | | Also remove direct time-stamping by CI scripts that would compete. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus lib.sh: refactor req_env_var()Ed Santiago2019-05-02
| | | | | | | | | | | | | | | Existing code was not working due to a bash gotcha ('exit' from a pipeline). It also had unnecessary duplication. New version is safer; also includes unit tests run under localunit. Existing invocations of req_env_var replaced via: $ [ edit setup_environment.sh, move one closing quote to its own line ] $ perl -ni -e 's/(?<=req_env_var )"(\S+)\s+\$\1"/$1/; if (/req_env_var "$/ .. /^\s*"/) { chomp; s/(?<=\S)\s.*//; if (/^\s*"/) { print "\n" } else { unless (/req_env_var/) { s/^\s+//; print " ";} print;} } else { print }' $(ack -l req_env_var) $ [ hand-massage an incorrect instance of '@' in lib.sh:ircmsg() ] Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: Temp. override container-selinux on F29Chris Evich2019-04-24
| | | | | | Also, undo oooooold runc package hack Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Support special-case modes of testingChris Evich2019-04-03
| | | | | | | | | | | | | | | | | | | | | | Previously libpod CI was fairly straight-forward, run unit and integration tests in a standard set of 3 VMs. Off on the side was a single special case of running tests as an ordinary user. There is a desire to stop using the PAPR system to support testing inside of a container. Since having two special cases potentially invites more down the road, make provisions to handle them more gracefully. This commit introduces an environment variable: ``$SPECIALMODE``. It's value has the following meanings within the CI scripts: Mode 'none': Nothing special, business as usual (default) Mode 'rootless': Rootless testing Mode 'in_podman': Build container, run integration tests in it. This will make adding additional special-cases later easier, as well as extending the special cases in a Matrix across multiple OS's. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Various fixes for rootless testingChris Evich2019-03-19
| | | | | | | | | * Randomize the user's UID and GID * Simplify `setup_environment.sh` * Support new "-r" option for `hack/get_ci_vm.sh` setting up rootless * Connect as $ROOTLESS_USER when using "-r" with `hack/get_ci_vm.sh` Signed-off-by: Chris Evich <cevich@redhat.com>
* Enable rootless integration testsbaude2019-03-19
| | | | Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: Notify on IRC if post-merge testing failsChris Evich2019-03-15
| | | | | | | | | | | Until recently it was very difficult to execute any scripts if part of a task failed. A new feature in Cirrus-CI makes this easy. Use it to post a notice on IRC when any task fails. Also: Add quotes around yaml-string values for consistency and syntax-highlighting correctness. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add dedicated rootless mode testingChris Evich2019-03-05
| | | | | | | | | | | | | | | | | Certain integration tests require execution as a regular user. This is acomplished by `PodmanTest.PodmanAsUserBase()` wrapping a specialized execution environment, in `test/utils/utils.go`. However, doing this requires passing through python, which vastly increases the complexity of debugging low-level problems. This commit introduces a new parallel task, run as a regular user on the VM as set by three environment variables. All commands executed in the ``rootless_test.sh`` script, will occur as a real user with a name and home directory, just as `$DIETY` intended. All env. vars established during `environment_setup.sh` (for root) are available. The PR source in `$GOSRC` and `$GOPATH` are owned by this user, and ready for use. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Drop ginkgo, gomega, easyjson installChris Evich2019-03-04
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Install Go 1.11 on Ubuntu VMsChris Evich2019-02-22
| | | | | | | | | | | | | | There is no native package for this, so the packaged version must also be installed, otherwise all the support/dependencies would be removed also (like go-md2man). Fix this by installing from the google released tarball, into /usr/local/go and set $GOROOT to point there. Also, include a small fix for hack/get_ci_vm.sh not installing testing dependencies because of an old assumption. ***CIRRUS: REBUILD IMAGES*** Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Enable AppArmor build and testChris Evich2019-01-29
| | | | | | | | | This was specifically disabled early on in transitioning to cirrus due to an excessive number of test failures and lack of knowledge. This commit reverses the stance and unifies build and test commands across all current platforms. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Record start/end time of important thingsChris Evich2019-01-17
| | | | | | | | | | | | | Previously we only reported the date/time at the beginning of a run since it's not available in the Cirrus-CI system. This commit generalizes the solution, recording start/end times for all major events. Also the timestamps are recorded into a local file on the VMs. This is intended for future use, for example tracking execution-time trends. Signed-off-by: Chris Evich <cevich@redhat.com>
* (Minor) Cirrus: Print timestamp at startChris Evich2019-01-10
| | | | | | Also record into a file in case a later reference is required Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Use updated images including new cruiChris Evich2018-12-21
| | | | | | | Also add two minor tweaks which were preventing images from building properly. Signed-off-by: Chris Evich <cevich@redhat.com>
* Use existing CRIU packages in CI setupAdrian Reber2018-12-21
| | | | | | | | | | | | For all in testing involved distribution a new enough CRIU exists as a package. Fedora provides CRIU packages directly. For CentOS/RHEL there is a COPR with the latest CRIU version and for Ubuntu there is a PPA for the latest CRIU version. Let's use these packages and fall back to building from git if necessary. Signed-off-by: Adrian Reber <adrian@lisas.de>
* Cirrus: One IRC notice onlyChris Evich2018-12-18
| | | | | | | | | The podbot messages are becoming obnoxious as more distributions are tested. Only call the `success.sh` script once, after all testing was successful. Also make update the message to include more helpful text and url. Signed-off-by: Chris Evich <cevich@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-12-28 22:38:36 +0000