summaryrefslogtreecommitdiff
path: root/test/buildah-bud/run-buildah-bud-tests
Commit message (Collapse)AuthorAge
* buildah-bud tests: reenable skipped 'bud' testsEd Santiago2022-06-14
| | | | | | | | | | | | | | | | | | | | | | | | | Sigh. Buildah PR https://github.com/containers/buildah/pull/3368 changed 'bud' to 'build' in tests. Podman #11585 well-intentionedly did the same for run-buildah-bud tests ... but did so by *replacing* 'bud' with 'build', not by *adding* 'build' to the list of commands handled by podman-build. Hence, all tests invoking 'run_buildah bud' have been completely untested since then. This remedies that, and deals with all the fallout. Principal among which is the discovery that our exit-code changes are no longer necessary: that thing we did where buildah exit status 1 or 2 became podman exit status 125? That no longer applies. podman now exits with the same status as buildah. This simplifies our diffs, and lets us enable a bunch more tests. Also: - in run-buildah-bud-tests script, run 'sudo --validate' early on. Reason: otherwise, the sudo step happens a few minutes after the script starts (after the git-pull), by which time the user may have stepped away to get coffee, then comes back ten or twenty minutes later to find a stupid sudo prompt and no tests run. Signed-off-by: Ed Santiago <santiago@redhat.com>
* buildah bud tests under podman-remoteEd Santiago2021-07-28
| | | | | | | | | | | | | | | | New functionality -- mostly in the diffs we apply to buildah's helpers.bash -- to enable running buildah-bud tests under podman-remote. The gist of it is, we start a 'podman system service' before each test, and clean it up on test exit. Design decision: the diff file for helpers.bash is no longer trailing-whitespace-clean: that ended up producing diffs that git wouldn't apply, because in some cases the whitespace is actually important. In order to pass CI, we need to exclude this file from some checks. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Bump github.com/containers/storage from 1.32.3 to 1.32.5Daniel J Walsh2021-06-29
| | | | | | | | | | | | | | | | | Bumps [github.com/containers/storage](https://github.com/containers/storage) from 1.32.3 to 1.32.5. - [Release notes](https://github.com/containers/storage/releases) - [Changelog](https://github.com/containers/storage/blob/main/docs/containers-storage-changes.md) - [Commits](https://github.com/containers/storage/compare/v1.32.3...v1.32.5) --- updated-dependencies: - dependency-name: github.com/containers/storage dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* buildah-bud tests: simplifyEd Santiago2021-04-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Experience this week has shown that managing .diff files is too difficult for humans, and too fragile. Opportunities for errors abound. So, let's try to minimize the diffs. We can't eliminate the diffs to helpers.bash: those are true code changes that are absolutely required for running tests using podman instead of buildah. We need to carry those ourselves: they are not appropriate for the buildah repo itself. What we can do is simplify the patching of bud.bats. That is fragile, because bud.bats changes often, and context- sensitive git patch files can easily get confused. Recognizing that the changes to bud.bats fall under two types: - tests that are skipped - tests in which podman error messages differ from buildah's ...we now have a new script, apply-podman-deltas, which is (I hope) much user-friendlier. It understands two directives: errmsg - alter the expected error message skip - skip a test Both operate based on a bats test name. The test name must match exactly. These directives use 'sed' to update bud.bats. If any directive fails, the script will keep going (so you get as many errors as possible in a run), then exits failure. Instructions (README.md) now explain the process for dealing with all expected test failures. (Sneak checkin: add '--filter=NAME' option to test runner, allowing for targeted and much shorter test runs). Signed-off-by: Ed Santiago <santiago@redhat.com>
* buildah-bud tests: handle go pseudoversions, plus...Ed Santiago2021-04-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Handle go pseudoversions, e.g. a custom non-released buildah used during testing of a PR. This will be something like: v1.20.1-0.20210402144408-36a37402d0c8 ...and it makes it impossible (AFAIK) to do a shallow checkout; we need to do a full clone of buildah, then git-checkout the SHA (last element of the long string above). FIXME: this is great for testing, but we almost certainly want some way to block this PR from merging, don't we? And, while testing this, found and fixed three bugs: - quote "$failhint" when echoing it on failure; otherwise we lose original whitespace. - invoke git-am with --reject! This makes it SO MUCH EASIER to identify the failing part of our patch! - sigh: generate the make-new-buildah-diffs helper *BEFORE* we try git-am! Otherwise, duh, if git-am fails we have no way to help the developer create a new diff file. Signed-off-by: Ed Santiago <santiago@redhat.com>
* WIP: run buildah bud tests using podmanEd Santiago2021-03-21
Set of scripts to run buildah's bud.bats test using podman build in podman CI. podman build is not 100% compatible with buildah bud. In particular: * podman defaults to --layers=true; buildah to false * podman defaults to --force-rm=true; buildah to false * podman error exit status is 125; buildah is 2 * differences in error messages, command-line arguments Some of the above can be dealt with programmatically, by tweaking the buildah helpers.bash (BATS helpers). Some need to be tweaked by patching bud.bats itself. This PR includes a patch that will, I fear, need to be periodically maintained over time. There will likely be failures when vendoring in a new buildah, possibly because new tests were added for new features that don't exist in podman, possibly (I hope unlikely) if existing tests are changed in ways that make the patch file fail to apply. I've tried to write good instructions and to write the run script in such a way that it will offer helpful hints on failure. My instructions and code will be imperfect; I hope they will be good enough to merit continued use of this test (possibly with improvements to the instructions as we learn more about real-world failures). Signed-off-by: Ed Santiago <santiago@redhat.com>