summaryrefslogtreecommitdiff
path: root/contrib/cirrus/logcollector.sh
Commit message (Collapse)AuthorAge
* Cirrus: Collect ginkgo node logs artifactsChris Evich2021-02-03
| | | | | | | | | | | | | | | | | | | | In rare cases, it's possible for one of the ginkgo processes to "hang". When this occurs, the main output will contain this message: ``Ginkgo timed out waiting for all parallel nodes to report`` The only way to debug this was to look through concatenated printing of the ginkgo node logs. This is a tedious and daunting task, requiring special search knowledge, facing a "wall of text". Simplify the situation by collecting the node logs separately, as individual files in a cirrus-artifact. In this way, it's faster to figure out which test "hung" by examining each log individually. The log file which does not have a pass/fail summary at the end, indicates the last test hung (for whatever reason), and includes it's output (if any). Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Add support for Ubuntu 20.xChris Evich2020-12-15
| | | | | | | | | | | | | | | | | | | | | | | | | Previously automation always dropped the minor version number for distributions. This was intended for presentation and conditional simplicity. Bash does not support non-integer comparison natively. With the release of version 20.10, supporting testing with it and the LTS release (20.04) requires scripts to consider minor version numbers for Ubuntu VMs. This is necessary because many times in the past, some behaviors needed to be conditional on the release version number. With this commit, the images and embedded scripts/tooling uses an altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially) `$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing in the official version (from `/etc/os-release`) are dropped, and the result is concatenated. For example the current Ubuntu LTS version is `20.04`. Prior to this commit, `$OS_RELEASE_VER` would have been `20`. With this change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10` is shown as `2010`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Detailed CPU/Memory/Time runner.sh statsChris Evich2020-11-09
| | | | | | | | | | | | | | | | | | | | | | | | | | On several occasions, there have been questions about CPU/Memory/IO trends in testing over time. Start collecting this data for all jobs, using a common/stable format so that trending analysis can be performed within/across multiple Cirrus-CI builds. This PR doesn't add any related tooling, it simply arranges for the collection of the data. Stats generation is done at the orchestration level to guarantee they reflect everything happening inside `runner.sh`. For example, the container-based tests re-exec `runner.sh` inside a container, but we're only interested in the top-level stats. Update all tasks to include collection of the stats file. Unfortunately, due to the way the Cirrus-CI YAML parser works, it is *not* possible to alias the artifacts collection more clearly, for example: ```yaml always: <<: *runner_stats <<: *logs_artifacts ``` Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Always record runc/crun versionsChris Evich2020-10-28
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Implement podman automation 2.0Chris Evich2020-10-02
| | | | | | | | | | | | | | | | | | Reimplement CI-automation to remove accumulated technical-debt and optimize workflow. The task-dependency graph designed goal was to shorten it's depth and increase width (i.e. more parallelism). A reduction in redundant building (and 3rd party module download) was also realized by caching `$GOPATH` and `$GOCACHE` early on. This cache is then reused in favor of a fresh clone of the repository (when possible). Note: The system tests typically execute MUCH faster than the integration tests. However, contrary to a fail-fast/fail-early principal, they are executed last. This was implemented due to debug-ability related concerns/preferences of the primary (golang-centric) project developers. Signed-off-by: Chris Evich <cevich@redhat.com>
* Use `bash` binary from env instead of /bin/bash for scriptsSascha Grunert2020-08-17
| | | | | | | | It's not possible to run any of the scripts on distributions which do have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env bash` instead. Signed-off-by: Sascha Grunert <sgrunert@suse.com>
* Remove TEST_REMOTE_CLIENT from RCLIDaniel J Walsh2020-08-10
| | | | | | | | We know these are TEST_, hoping this makes the display in cirrus easier for users to see true|false, since this is the valuable information is. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Merge pull request #4340 from cevich/libseccomp_updateOpenShift Merge Robot2020-03-28
|\ | | | | Log libseccomp package version
| * Cirrus: Log libseccomp package versionChris Evich2020-02-26
| | | | | | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* | Four small CI fixes:Ed Santiago2020-03-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) 'podman system info' (in logcollector): has been silently failing in special_testing_rootless, with: logcollector.sh: line 16: podman: command not found Use ./bin/podman instead of just podman; this is probably the right thing to do in the general case anyway 2) logformatter: highlight 'panic:', seen in bindings test: https://storage.googleapis.com/cirrus-ci-5385732420009984-fcae48/artifacts/containers/libpod/6693715108429824/html/integration_test.log.html 3) logformatter: handle Unicode bullet in front of 'Running', seen in bindings test. 4) logformatter: turn down contrast on BATS 'ok' results, for legibility Signed-off-by: Ed Santiago <santiago@redhat.com>
* | CI: package_versions: include hostinfo, kernelEd Santiago2020-02-27
|/ | | | | | | | | | | | | | In the package_versions CI step, include Fedora/Ubuntu version, uname -r, and cgroups version. Cgroups version is simply the FS type of /sys/fs/cgroup, which shows 'tmpfs' for v1 and 'cgroup2fs' for v2. I don't think it's worth the effort to prettify those into 'v1/v2' - I think our readers are sophisticated enough to figure it out from context - but am willing to add that feature if requested. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Cirrus: collect podman system infoChris Evich2020-02-25
| | | | Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Produce and collect varlink outputChris Evich2019-10-08
| | | | | | | | | | | | | When executing 'make remotesystem' testing, a varlink process is started up but it's stdio is dumped due to the production of excessive data. However, this also means if the process has a problem, any errors will not be accessible. Instead, grab only the last 100 lines and direct them into a file. Also update automation's log collection to retrieve this file when the `$REMOTE_CLIENT` env. var. is `true`. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Simplify package NVR loggingChris Evich2019-10-03
| | | | | | Also include runc/crun version for Fedora Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Show names/versions of critical packagesChris Evich2019-10-01
| | | | | | | Adds an output to all testing tasks which lists the names/versions of critical/essential packages present on the VM. Signed-off-by: Chris Evich <cevich@redhat.com>
* Cirrus: Simplify log collection commandsChris Evich2019-06-14
Signed-off-by: Chris Evich <cevich@redhat.com>