summaryrefslogtreecommitdiff
path: root/test/system/015-help.bats
Commit message (Collapse)AuthorAge
* Switch use of Flags to OptionsDaniel J Walsh2020-10-21
| | | | | | | | Want to have man pages match commands, since we have lots of printed man pages with using Options, we will change the command line to use Options in --help. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Usability: prevent "-l" with argumentsEd Santiago2020-09-14
| | | | | | | | | | | Add new system check confirming that "podman foo -l arg" throws an error; and fix lots of instances where code was not doing this check. I'll probably need to add something similar for --all but that can wait. Signed-off-by: Ed Santiago <santiago@redhat.com>
* system tests: enable more remote tests; cleanupEd Santiago2020-08-19
| | | | | | | | | | | | | | | | | | | | | | | | info, images, run, networking tests: remove some skip_if_remote()s that were added in the varlink days. All of these tests now seem to work with APIv2. help test: check that first output line from 'podman --help' is the program description (regression check for #7273). load test: clean up stray images, rewrite test to make it conform to existing convention. In the process, discover and file #7337 exec test (and networking): file #7360, and add FIXME comment to skip()s suggesting evaluating those tests once that is fixed. pod test: now that #6328 is fixed, use 'podman pod inspect --format' instead of relying on jq Various other tests: add an explanation of why test is disabled so we can more easily distinguish "this will never be meaningful under remote" vs "hey, doesn't work for now, but maybe someday". Signed-off-by: Ed Santiago <santiago@redhat.com>
* BATS help-message test: improve diagnosticsEd Santiago2020-07-21
| | | | | | | | | | | The error messages from the 'podman xxx --help' cross-check test are unhelpful, and cause much wasted time when they trigger. Solution: instead of using the built-in exit-status check in run_podman, do an explicit check outside of run_podman. This lets us die() with a custom, hopefully useful, message. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Docs: consistency between man / --helpEd Santiago2020-06-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New functionality in hack/man-page-checker: start cross- referencing the man page 'Synopsis' line against the output of 'podman foo --help'. This is part 1, flag/option consistency. Part 2 (arg consistency) is too big and will have to wait for later. flag/option consistency means: if 'podman foo --help' includes the string '[flags]' in the Usage message, make sure the man page includes '[*options*]' in its Synopsis line, and vice-versa. This found several inconsistencies, which I've fixed. While doing this I realized that Cobra automatically includes a 'Flags:' subsection in its --help output for all subcommands that have defined flags. This is great - it lets us cross-check against the usage synopsis, and make sure that '[flags]' is present or absent as needed, without fear of human screwups. If a flag-less subcommand ever gets extended with flags, but the developer forgets to add '[flags]' and remove DisableFlagsInUseLine, we now have a test that will catch that. (This, too, caught two instances which I fixed). I don't actually know if the new man-page-checker functionality will work in CI: I vaguely recall that it might run before 'make podman' does; and also vaguely recall that some steps were taken to remedy that. Signed-off-by: Ed Santiago <santiago@redhat.com>
* BATS tests: new too-many-arguments testEd Santiago2020-06-23
| | | | | | | | | | | | | | | | | | | | | | | | | ...plus a few others. And fixes to actual parsing. If a command's usage message includes '...' in the argument list, assume it can take unlimited arguments. Nothing we can check. For all others, though, the ALL-CAPS part on the right-hand side of the usage message will define an upper bound on the number of arguments accepted by the command. So in our 'podman --help' test, generate N+1 args and run that command. We expect a 125 exit status and a suitably helpful error message. Not all podman commands or subcommands were checking, so I fixed that. And, fixed some broken usage messages (all-caps FLAGS, and '[flags]' at the end of 'ARGS'). Add new checks to the help test to prevent those in the future. Plus a little refactoring/cleanup where necessary. Signed-off-by: Ed Santiago <santiago@redhat.com>
* BATS help test: check usage stringEd Santiago2020-05-05
| | | | | | | | | | Now that we've agreed that usage messages should match what the user typed, confirm it. IOW 'podman foo --help' should not issue a usage message for 'podman container foo'. Fix one broken instance, 'unpause'. Signed-off-by: Ed Santiago <santiago@redhat.com>
* fix commands without inputbaude2020-04-30
| | | | | | in cases where commands require input and we dont provide it, we often would segv. This can be attributed in many cases to the subcommand not picked up the cobra Args attribute or neither had them. Signed-off-by: baude <bbaude@redhat.com>
* System tests: help messages: check required-argEd Santiago2020-04-30
| | | | | | | | | | | | | | | | | | | If a usage message is of the form '... [flags] ARGNAME', where ARGNAME is all-caps and not in brackets, it must be a required argument. Try running podman subcommand without ARGNAME, and make sure that podman bails out with an informative message. (Since this message is freeform in each subcommand, not Cobra-generated, we have a lot of possible variations to check for). Fix podman login/logout Use messages to indicate that REGISTRY is now optional (as of #5233). This test has actually been in place for over a year but due to a typo on my part -- a missing space -- it was not being run. "For want of a space, much testing was lost". Signed-off-by: Ed Santiago <santiago@redhat.com>
* stats: list all running containers unless specified otherwiseValentin Rothberg2019-10-23
| | | | | | | | | | Unless specified otherwise by --all, --latest or via arguments, list all running containers. This matches the behaviour of Docker and is also illustrated in the man pages where containers and options are marked to be optional. Fixes: #4274 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* BATS tests: start supporting podman-remoteEd Santiago2019-04-10
| | | | | | | | | | | | | | podman-remote now supports rm! That's what we needed to start running BATS tests. Although most tests don't actually work, some do, and maybe the rest will start working over time. For now, disable them. The only significant difference found is that podman-remote strips fractional seconds from timestamps in JSON output. Probably not something worth caring about. Signed-off-by: Ed Santiago <santiago@redhat.com>
* BATS: new tests, and improvements to existing onesEd Santiago2019-03-18
| | | | | | | | | | | | | | | | | | | | | New: - podman exec - podman load (requires #2674) - CLI parsing (regression test for #2574) Improved: - help: test "podman NoSuchCommand", and subcommands - help: test "podman cmd" without required args - pod: start with --infra=false; this allows running rootless - log: also run 'logs' after container is run - log: test -f with two containers Also, use helpful descriptions for skip_if_rootless Tested on f29, root and rootless. As soon as podman-remote supports rm, I'll start testing that too. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Implement review feedbackEd Santiago2019-03-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - document a recommended convention for fail-fast tests - document the requirement for jq. (And, add a fail-fast test for its presence; remove the duplicated checks in subtests) - add further sanity checks to 'help' test. Add missing documentation. Remove a no-longer-needed workaround for usage-message bug fixed in #2486 - add a documented TEMPLATE - and, since we're at 1.1, enable 'Remote API' check in version test - better diagnostics in setup/teardown; add vim filetype hint; better formatting of actual-vs-expect errors - new pod-top, logs, build tests - improve error messages - add $IMAGE alias for ridiculous $PODMAN_TEST_IMAGE_FQN - final cleanup, in prep for merge Signed-off-by: Ed Santiago <santiago@redhat.com>
* new system tests under BATSEd Santiago2019-03-07
Initial attempt at writing a framework for podman system tests. The idea is to define a useful set of primitives that will make it easy to write actual tests and to interpret results of failing ones. This is a proof-of-concept right now; only a small number of tests, by no means comprehensive. I am requesting review in order to find showstopper problems: reasons why this approach cannot work. Should there be none, we can work toward running these as gating tests for Fedora and RHEL8. Signed-off-by: Ed Santiago <santiago@redhat.com>