summaryrefslogtreecommitdiff
path: root/contrib/cirrus/integration_test.sh
Commit message (Collapse)AuthorAge
* Cirrus: Handle runc->crun when both are possibleChris Evich2020-02-25
| | | | | | | | | | In some distributions it's possible to have both runc and crun installed and/or for podman to be confused about which to use. In these instances, force the decision by adding `OCI_RUNTIME=/usr/bin/crun` into `/etc/environment`. Also in-place modify libpod.conf to use 'crun' instead of 'runc' Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Support testing with F31Chris Evich2020-02-25
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* enable ci on go binding testsBrent Baude2020-02-22
| | | | Signed-off-by: Brent Baude <bbaude@redhat.com>
* Cirrus: Produce and collect varlink outputChris Evich2019-10-08
| | | | | | | | | | | | | When executing 'make remotesystem' testing, a varlink process is started up but it's stdio is dumped due to the production of excessive data. However, this also means if the process has a problem, any errors will not be accessible. Instead, grab only the last 100 lines and direct them into a file. Also update automation's log collection to retrieve this file when the `$REMOTE_CLIENT` env. var. is `true`. Signed-off-by: Chris Evich <cevich@redhat.com>
* tests: use crun packageGiuseppe Scrivano2019-09-19
| | | | Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Add support for launching containers without CGroupsMatthew Heon2019-09-10
| | | | | | | This is mostly used with Systemd, which really wants to manage CGroups itself when managing containers via unit file. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Cirrus: Reimplement release archive + uploadChris Evich2019-08-28
| | | | | | | | The initial implementation was far more complicated than necessary. Strip out the complexities in favor of a simpler and more direct approach. Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #3824 from baude/varlinkendpointtestOpenShift Merge Robot2019-08-26
|\ | | | | Create framework for varlink endpoint integration tests
| * Create framework for varlink endpoint integration testsbaude2019-08-16
| | | | | | | | | | | | | | add the ability to write integration tests similar to our e2e tests for the varlink endpoints. Signed-off-by: baude <bbaude@redhat.com>
* | cirrus: enable cgroups v2 tests with crunGiuseppe Scrivano2019-08-13
|/ | | | Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Cirrus: Add experimental fedora VM image & testChris Evich2019-08-12
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Merge pull request #3106 from cevich/cirrus_releaseOpenShift Merge Robot2019-07-10
|\ | | | | Cirrus: Automate releasing of tested binaries
| * Cirrus: Automate releasing of tested binariesChris Evich2019-07-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's desirable to make archives available of builds containing actual tested content. While not official distro-releases, these will enable third-party testing, experimentation, and development for both branches (e.g. "master") and pull requests (e.g. "pr3106"). * Add a Makefile targets for archiving both regular podman binaries and the remote-client. Encode release metadata within these archives so that their exact source can be identified. * Fix bug with cross-compiling remote clients for the Windows and Darwin platforms. * Add unit-testing of cross-compiles for Windows and Darwin platforms. * A few small CI-script typo-fixes * Add a script which operates in two modes: 1. Call Makefile targets which produce release archives. Upload the archive to Cirrus-CI's built-in caching system using reproducible cache keys. 2. Utilize reproduced cache keys to attempt download of cache from each tasks. When successful, parse the file's release metadata, using it to name the archive file. Upload all recovered archives to a publicly accessible storage bucket for future reference. * Update the main testing task to call the script in mode #1 for all primary platforms. * Add a new `$SPECIALMODE` task to call the script in mode #1 for Windows and Darwin targets. * Add a new 'release' task to the CI system, dependent upon all other tasks. This new tasks executes the script in mode #2. * Update CI documentation Signed-off-by: Chris Evich <cevich@redhat.com>
* | Cirrus: Use packaged dependenciesChris Evich2019-06-27
|/ | | | | | | | | | | | | | | | | | Building/installing dependencies from fixed source-version ensures testing is reliable, but introduces a maintenance burden and risks testing far outside of a real-world environment. The sensible alternative is to install dependencies from distro-packaging systems. Install all development and testing dependencies at VM cache-image build time, to help ensure testing remains stable. The existing cache-image build workflow can be utilized at any future time to build/test with updated packages. ***N/B***: This does not update any dockerfiles used by testing, that is left up to future efforts. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add support for testing F30Chris Evich2019-06-14
| | | | | | | | | | | | | | | | Remove disused `build_cache_images` task, and update relevant dockerfiles for F30. Fix problem of cloud-init failing to expand root-device on boot (/var/lib/cloud/instance left in improper state). Fix problem of cloud-init racing with google-network-daemon.service on boot (looking for cloudconfig metadata too early). Causing root-device to _sometimes_ fail to expand. Fix problem of hack/get_ci_vm.sh argument passing. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: minor cleanup and refactoringEd Santiago2019-06-03
| | | | | | | | | | | | | | | | | | | | | | ...with the goal of (very soon) reusing this code, in #2947, to run system tests in CI. This is the cleanest way I can think of to do so without duplication or a large maintenance burden. Changes are: - replace references to 'ginkgo' with 'integration'. That target is already in Makefile, and is not only more readable, it's also more abstract. There is no reason for this level of code to know about ginkgo. - allow rootless_test.sh to accept an argument, that being the name of the test suite to run (default: integration). #2947 will enable 'system'. - allow integration_test.sh to serve multiple purposes, by checking its filename. #2947 will add a symlink, system_test.sh, which will then cascade down to invoke system tests. Signed-off-by: Ed Santiago <santiago@redhat.com>
* split rootless local and remote testingbaude2019-05-31
| | | | Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: fixups based on review feedbackChris Evich2019-05-21
| | | | | | Also remove disused distros (RHEL/CentOS/FAH) and fix get_ci_vm script Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Overhaul/Simplify env. var setupChris Evich2019-05-21
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* split remote tests from distro testsbaude2019-05-13
| | | | | | | | We want the remote tests for our distributions to be tested in a different VM than the local tests. This allows for faster CI runs and easier debug as well as seperation of flakes. Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: timestamp all output script outputChris Evich2019-05-02
| | | | | | Also remove direct time-stamping by CI scripts that would compete. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus lib.sh: refactor req_env_var()Ed Santiago2019-05-02
| | | | | | | | | | | | | | | Existing code was not working due to a bash gotcha ('exit' from a pipeline). It also had unnecessary duplication. New version is safer; also includes unit tests run under localunit. Existing invocations of req_env_var replaced via: $ [ edit setup_environment.sh, move one closing quote to its own line ] $ perl -ni -e 's/(?<=req_env_var )"(\S+)\s+\$\1"/$1/; if (/req_env_var "$/ .. /^\s*"/) { chomp; s/(?<=\S)\s.*//; if (/^\s*"/) { print "\n" } else { unless (/req_env_var/) { s/^\s+//; print " ";} print;} } else { print }' $(ack -l req_env_var) $ [ hand-massage an incorrect instance of '@' in lib.sh:ircmsg() ] Signed-off-by: Ed Santiago <santiago@redhat.com>
* podman-remote pause|unpausebaude2019-04-18
| | | | | | | | Add the ability to pause and unpause containers with the remote client. Also turned on the pause tests! Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: Support special-case modes of testingChris Evich2019-04-03
| | | | | | | | | | | | | | | | | | | | | | Previously libpod CI was fairly straight-forward, run unit and integration tests in a standard set of 3 VMs. Off on the side was a single special case of running tests as an ordinary user. There is a desire to stop using the PAPR system to support testing inside of a container. Since having two special cases potentially invites more down the road, make provisions to handle them more gracefully. This commit introduces an environment variable: ``$SPECIALMODE``. It's value has the following meanings within the CI scripts: Mode 'none': Nothing special, business as usual (default) Mode 'rootless': Rootless testing Mode 'in_podman': Build container, run integration tests in it. This will make adding additional special-cases later easier, as well as extending the special cases in a Matrix across multiple OS's. Signed-off-by: Chris Evich <cevich@redhat.com>
* Globally increase test timeout to 90-minutesChris Evich2019-03-04
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add 20m extra timeout for UbuntuChris Evich2019-02-22
| | | | | | | Tests running slower than normally-slow, bump timeout to allow them to pass until better solution (for slow Ubuntu tests) can be found. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Enable AppArmor build and testChris Evich2019-01-29
| | | | | | | | | This was specifically disabled early on in transitioning to cirrus due to an excessive number of test failures and lack of knowledge. This commit reverses the stance and unifies build and test commands across all current platforms. Signed-off-by: Chris Evich <cevich@redhat.com>
* cirrus: Record start/end time of important thingsChris Evich2019-01-17
| | | | | | | | | | | | | Previously we only reported the date/time at the beginning of a run since it's not available in the Cirrus-CI system. This commit generalizes the solution, recording start/end times for all major events. Also the timestamps are recorded into a local file on the VMs. This is intended for future use, for example tracking execution-time trends. Signed-off-by: Chris Evich <cevich@redhat.com>
* Run integrations test with remote-clientbaude2019-01-14
| | | | | | | | | | | | Add the ability to run the integration (ginkgo) suite using the remote client. Only the images_test.go file is run right now; all the rest are isolated with a // +build !remotelinux. As more content is developed for the remote client, we can unblock the files and just block single tests as needed. Signed-off-by: baude <bbaude@redhat.com>
* Cirrus: Document and codify base-image productionChris Evich2018-12-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A number of images required for future testing are not present in GCE. Importing them is a long proscribed process prone to errors and complications. Improve this situation by documenting, and encoding the majority of the steps required. Due to the required complexity, these are clearly identified as 'semi-automated'. This means a discerning eye is sometimes needed to address unforeseen problems (networking issues, format or packaging changes, etc). Nevertheless, having these steps in writing, will reduce current and future maintenance burden while supporting future testing needs of RHEL, Fedora and Fedora Atomic Host. Also: * Add necessary configuration, scripts, and Makefile updates needed to prepare RHEL, Fedora, & FAH cloud images for use in GCE. This is a complex, multi-step process where the cloud image is booted un a local user-mod qemu-kvm instance, where it can be modified. From there, it's converted into a specific format, and imported into GCE. Lastly, the imported raw disk data is made available as a GCE VM image. Note: As of this commit, the RHEL base-image builds (CentOS has native image), however neither RHEL or CentOS cache-images build correctly. * Left testing on FAH disabled, the GCE/Cirrus integration needs needs more work. Specifically, the python3-based google startup script service throws a permission-denied (as root) when trying to create a temp. directory. Did not investigate further, though manually running the startup script does allow the libpod tests to start running. * Enabled Fedora 29 image to execute tests and general use. * Utilize the standardized F28-based container image for gating of more the intensive unit and integration testing. Update documentation to reflect this as the standard platform for these checks. Rename tasks with shorter names and to better reflect their purpose. * Cirrus: Trim unnecessary env vars before testing since the vast majority are only required for orchestration purposes. Since most are defined within `.cirrus.yml`, it's a good place to store the list of undesirables. Since each of the cirrus-scripts runs in it's own shell, unsetting these near the end will have no consequence. Also trim down the number of calls to show_env_vars() Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Use Makefile for image-buildingChris Evich2018-12-06
| | | | | | | | | | | | | | | | | | | | The packer tool takes JSON as input for the details of producing VM images to be used for PR CI-testing. JSON is not a very human-friendly format, without support for comments and frequently containing lots of duplicate data. Fix this by using a Makefile + simple python one-liner to convert from a human-friendly YAML format into packer-native JSON. This allows use of anchors/aliases to reduce duplication, and allows inline comments for easier maintainability. This also allows separating the 'test' action from the 'build' action, for earlier and better syntax problem detection. Lastly, there are some minor ``lib.sh`` and ``integration_test.sh`` updates to support future work, and slightly improve the build and test environments. Signed-off-by: Chris Evich <cevich@redhat.com>
* Add configuration for Cirrus-CIChris Evich2018-10-04
Testing podman requires exercising on a full-blown VM. The current containerized-approach is complicated, and mostly a band-aid over shortcomings in the other CI systems. Namely, we want: * To pre-build environments with dependencies to reduce the setup time needed for testing. * The ability to verify the pre-built environments are working before utilizing them for further testing. * A simple, single set of flexible automation instructions to reduce maintenance burden. * Ease of environment reproduction across clouds or locally, for debugging failures. This change leverages Cirrus-CI + Packer + collection of shell scripts to realize all of the above. Signed-off-by: Chris Evich <cevich@redhat.com>