summaryrefslogtreecommitdiff
path: root/test/system/001-basic.bats
Commit message (Collapse)AuthorAge
* system tests: add assert(), and start using itEd Santiago2022-04-20
| | | | | | | | | | | | | | | | | | | | | Problem: the system test 'is()' checker was poorly thought out. For example, there is no way to check for inequality or for absence of a substring. Solution, step 1: introduce new assert(), copied almost verbatim from buildah, where it has been successful in addressing the gaps in is(). The logical next step is to search the tests for 'die' and for 'run', looking for negative assertions which we can replace with assert(). There were a lot, and in the process I found a number of ugly bugs in the tests themselves. I've taken the liberty of fixing these. Important note: at this time we have both assert() and is(). Replacing all instances of is() would be impossible to review. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Merge pull request #13296 from ↵OpenShift Merge Robot2022-02-21
|\ | | | | | | | | Romain-Geissler-1A/url-and-connection-implies-remote Option --url and --connection should imply --remote.
| * Option --url and --connection should imply --remote.Romain Geissler2022-02-19
| | | | | | | | | | | | Closes #13242 Signed-off-by: Romain Geissler <romain.geissler@amadeus.com>
* | Merge pull request #13306 from Luap99/flag-errorOpenShift Merge Robot2022-02-21
|\ \ | | | | | | provide better error on invalid flag
| * | provide better error on invalid flagPaul Holzinger2022-02-21
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a extra `See 'podman command --help'` to the error output. With this patch you now get: ``` $ podman run -h Error: flag needs an argument: 'h' in -h See 'podman run --help' ``` Fixes #13082 Fixes #13002 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* / System tests: show one-line config overviewEd Santiago2022-02-21
|/ | | | | | | | | | | | | We're running into problems that are impossible to diagnose because we have no idea if the SUT is using netavark or CNI. We've previously run into similar problems with runc/crun, or cgroups 1/2. This adds a one-line 'echo' with important system info. Now, when viewing a full test log, it will be possible to view system settings in one glance. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Add podman rm --dependDaniel J Walsh2022-01-11
| | | | | | | | | | | This option causes Podman to not only remove the specified containers but all of the containers that depend on the specified containers. Fixes: https://github.com/containers/podman/issues/10360 Also ran codespell on the code Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Refactor podman system to report.FormatterJhon Honce2021-12-03
| | | | | | | | | | | | | | [NO NEW TESTS NEEDED] Support better compatibility output for podman system commands * Format and content of output from podman version changed to be more compatible See #10974 Depends on containers/common#831 Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Merge pull request #11953 from markusthoemmes/help-defaultOpenShift Merge Robot2021-11-08
|\ | | | | Display help text on empty subcommand by default
| * Keep error semantics intactMarkus Thömmes2021-11-05
| | | | | | | | Signed-off-by: Markus Thömmes <markusthoemmes@me.com>
| * Adjust tests to verify all subcommands show the help messageMarkus Thömmes2021-10-14
| | | | | | | | Signed-off-by: Markus Thömmes <markusthoemmes@me.com>
* | system tests: CONTAINER_* and --help: cleanupEd Santiago2021-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A small part of this test was written in a confusing and fragile way: it was very hard to understand, and in fact only worked through pure luck (using 'echo $output', which emitted everything in one long line, vs the standard quoted 'echo "$output"' which would've kept the formatting and caused the test to pass, incorrectly, no matter whether --remote was in the output or not). Plus, the '$?' check in the next line would never trigger on failure anyway, so the failure message would've been unhelpful if the test were ever to fail. Anyhow. Make it readable and make it work. (Followup to #11990) Signed-off-by: Ed Santiago <santiago@redhat.com>
* | Move CONTAINER_HOST and _CONNECTION to IsRemote FunctionDaniel J Walsh2021-10-15
| | | | | | | | | | | | | | | | Current code does not check early enough. Follow up to https://github.com/containers/podman/pull/11978 Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* | If CONTAINER_HOST env variable is set default podman --remote=trueDaniel J Walsh2021-10-14
|/ | | | | | | | | | | Users enabling CONTAINER_HOST==PATH is indicating to podman they intend to use remote functionality. Fixes: https://github.com/containers/podman/issues/11196 Update man pages to document all of the environment variables. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* System tests: tighten 'is' operatorEd Santiago2021-09-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix day-one sloppiness: when I first wrote this framework it compared strings using 'expr', not '=', to be more forgiving of extra cruft in output. This was a bad decision. It means that warnings or additional text are ignored: is "all is ok, NOT!" "all is ok" <-- this would pass Solution: tighten up the 'is' check. Use '=' (direct compare) first. If it fails, look for wild cards ('*') or character classes ('[') in the expect string. If so, and only then, use 'expr'. And, thanks to a clever suggestion from Luap99, include '(using expr)' in the error message when we do so; this could make it easier for a developer to understand a string mismatch. This change exposes a lot of instances in which we weren't doing proper comparisons. Fix those. Thankfully, there weren't as many as I'd feared. Also, and completely unrelated, add '-T' flag to bats helper, for showing timing results. (I will open this as a separate PR if requested. I too find it offensive to jumble together unrelated commits.) Signed-off-by: Ed Santiago <santiago@redhat.com>
* remote untag: support digestsValentin Rothberg2021-09-20
| | | | | | | | | Fix a bug when remotely untagging an image via tag@digest. The digest has been lost in the remote client and hence led to a wrong behaviour on the server. Fixes: #11557 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* support tag@digest notationValentin Rothberg2021-05-31
| | | | | | | | | Vendor in the latest HEAd of containers/common to implicitly support the tag@digest notation for images. To remain compatible with Docker, the tag will be stripped off the image reference and is entirely ignored. Fixes: #6721 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Remove obsolete skipsEd Santiago2021-05-10
| | | | | | | | | | | | | The following bugs seem to be fixed: * #6510 (e2e tests) - podman rmi gives "layer not known" * #9915 (buildah-bud tests) - podman build --arch * #10248 - spurious warning from first-ever invocation of rootless podman Signed-off-by: Ed Santiago <santiago@redhat.com>
* CI: run rootless tests under ubuntuEd Santiago2021-05-06
| | | | | | | | | | | | | | | | | | | | Reason: to catch errors before they surface in RHEL. One of the Ubuntus is specially crafted to run with cgroups v1 and runc. Although this isn't quite the same as RHEL, it's as close as we can come in our CI environment, and I suspect it would have caught #10234 (a regression). Sorry, team. Also: play kube limits test: skip on all rootless, not just rootless+fedora. There was a complicated and unnecessary check in there for Fedora. Also: workaround for bug #10248, a spurious error message on the first invocation of rootless podman on Ubuntu.Old Signed-off-by: Ed Santiago <santiago@redhat.com>
* Test that we don't error out on advertised --log-level valuesNalin Dahyabhai2021-04-14
| | | | Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
* Add support for podman --context defaultDaniel J Walsh2021-04-05
| | | | | | | | This is a noop but helps with scripting and docker-compose. Fixes: https://github.com/containers/podman/issues/9806 Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* podman-remote does not support most of the global flagsDaniel J Walsh2020-10-02
| | | | | | | podman-remote --help is showing a bunch of global flags that it does not support Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Don't setup the Image/ContainerEngine when calling a cmd with subcmdsPaul Holzinger2020-09-08
| | | | | | | | | | | There is no need to setup the image and container engine when calling a command with subcommands since we only print a usage message. e.g `podman`,`podman container` This also allows the remote client to show the usage message on these commands without a running endpoint. I added a test for this. Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
* fix podman version output to include git commit and builttimePaul Holzinger2020-08-18
| | | | | | Add the go module version v2 to the libpod path. Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
* system tests: podman-remote, image treeEd Santiago2020-08-10
| | | | | | | | | | | | | | | | | | | | | - new sanity checks for podman-remote: - first, confirm that when PODMAN is "-remote", we actually talk to a server (validated by presence of "Server:" string in "podman version"). - second, add test for #7212, in which we run "podman --remote" (podman with --remote flag, not podman-remote command) and make sure --remote is allowed both as the first option and also with other flag options preceding. - new test for "podman image tree" (piggybacking on top of a "podman build" test, because that gives us lots of layers). - skip "podman exec - basic test" when remote. It is consistently causing CI failures, breaking all of CI, due to #7241. Signed-off-by: Ed Santiago <santiago@redhat.com>
* podman version --format ... was not workingDaniel J Walsh2020-05-21
| | | | | | This patch fixes the podman --version --format command. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* implement 'make remotesystem'Ed Santiago2019-08-08
| | | | | | | | | | | | | | | | | | | | | podman-remote rm now works; that's the only thing we were waiting for to enable podman-remote (varlink) system tests. Add a (too-complicated, sorry) Makefile target that will define a random socket path, start the podman varlink server, and run the test suite using podman-remote. Also: add two convenience functions, is_rootless and is_remote, and use those in skip_if_rootless/if_remote and elsewhere Also: workarounds for broken tests: - basic version test: podman-remote emits an empty 'Client' line. Just ignore it. - looks like 'podman-remote pod' doesn't work; skip test. Also: minor documentation update Signed-off-by: Ed Santiago <santiago@redhat.com>
* Implement review feedbackEd Santiago2019-03-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - document a recommended convention for fail-fast tests - document the requirement for jq. (And, add a fail-fast test for its presence; remove the duplicated checks in subtests) - add further sanity checks to 'help' test. Add missing documentation. Remove a no-longer-needed workaround for usage-message bug fixed in #2486 - add a documented TEMPLATE - and, since we're at 1.1, enable 'Remote API' check in version test - better diagnostics in setup/teardown; add vim filetype hint; better formatting of actual-vs-expect errors - new pod-top, logs, build tests - improve error messages - add $IMAGE alias for ridiculous $PODMAN_TEST_IMAGE_FQN - final cleanup, in prep for merge Signed-off-by: Ed Santiago <santiago@redhat.com>
* new system tests under BATSEd Santiago2019-03-07
Initial attempt at writing a framework for podman system tests. The idea is to define a useful set of primitives that will make it easy to write actual tests and to interpret results of failing ones. This is a proof-of-concept right now; only a small number of tests, by no means comprehensive. I am requesting review in order to find showstopper problems: reasons why this approach cannot work. Should there be none, we can work toward running these as gating tests for Fedora and RHEL8. Signed-off-by: Ed Santiago <santiago@redhat.com>