summaryrefslogtreecommitdiff
path: root/test/apiv2/test-apiv2
Commit message (Collapse)AuthorAge
* Exit with 0 when receiving SIGTERMJhon Honce2022-03-15
| | | | | | | * systemctl stop podman.service will now return exit code 0 * Update test framework to support JSON boolean and numeric values Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Move all python tests to pytestJhon Honce2022-03-04
| | | | | | | * Add configuration to add report header for python client used in tests * Move report headers into the individual test runners vs runner.sh Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Add version guard to libpod API endpointsJhon Honce2022-02-09
| | | | | | | | | * Ensure meaningful behaviour when called with /v3.x.x semantics * Change return code to 409 from 500 when client attempts to use an existing network name * Update API bats test runner to support /v4.0.0 endpoints by default Signed-off-by: Jhon Honce <jhonce@redhat.com>
* APIv2 tests: followup to recent log testEd Santiago2022-01-19
| | | | | | | | | | | | | Followup to #12919, which merged while I was writing review feedback. This actually confirms log output. This required a minor change to the 't' helper: stripping NUL chars from the http result. And, while I'm at it, a bunch of cleanup for running rootless: - set $CONTAINERS_HELPER_BINARY_DIR, so we can find rootlessport - add a few conditionals for different expectations Signed-off-by: Ed Santiago <santiago@redhat.com>
* Refactor manifest list operationsJhon Honce2022-01-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update method/function signatures use the manifest list name and images associated with the operation explicitly, in general func f(ctx context.Context, manifestListName string, ImageNames []string, options *fOptions) * Leverage gorilla/mux Subrouters to support API v3.x and v4.x for manifests * Make manifest API endpoints more RESTful * Add PUT /manifest/{id} to update existing manifests * Add manifests.Annotate to go bindings, uncommented unit test * Add DELETE /manifest/{Id} to remove existing manifest list, use PUT /manifest/{id} to remove images from a list * Deprecated POST /manifest/{id}/add and /manifest/{id}/remove, use PUT /manifest/{id} instead * Corrected swagger godoc and updated to cover API changes * Update podman manifest commands to use registry.Context() * Expose utils.GetVar() to obtain query parameters by name * Unexpose server.registerSwaggerHandlers, not sure why this was ever exposed. * Refactored code to use http.Header instead of map[string]string when operating on HTTP headers. * Add API-Version header support in bindings to allow calling explicate versions of the API. Header is _NOT_ forwarded to the API service. Signed-off-by: Jhon Honce <jhonce@redhat.com>
* APIv2 tests: fail on syntax/logic errorsEd Santiago2021-12-14
| | | | | | | | | | | | | | | | (i.e. not test failures, but actual programming bugs). We've had a number of syntax errors creep into this test, usually caused by a missing backslash on a test command. I've long wanted to 'set -e' but that causes other problems. This PR introduces error handling via 'trap', with useful diagnostics on failure. This PR also catches and fixes two previously-unknown bugs that were causing tests to not actually run. And, since /events takes eons on my high-uptime laptop, add /since Signed-off-by: Ed Santiago <santiago@redhat.com>
* apiv2 tests: refactor complicated curlsEd Santiago2021-12-14
| | | | | | | | | | | | | | | | | | | | | Some months ago, apiv2 tests got added that needed new functionality: passing a tarball to the remote server. There was no mechanism to do so in the 't' helper, so these tests used complicated (and actually not-really- working) curl commands. This PR introduces and documents a new usage of 't', in which passing an argument ending in '.tar' adds the right magic syntax (--data-binary @PATH) to the existing curl. This lets us use all standard 't' checks, making for simpler tests and in the process fixing some bugs. Also: drive-by fix of a typo bug in the networks test. Also: set CONTAINERS_REGISTRIES_CONF when starting server and when running direct podman, to avoid docker.io throttling. Signed-off-by: Ed Santiago <santiago@redhat.com>
* api: handle nil pointer dereference in rest endpointsJelle van der Waa2021-09-15
| | | | | | | | When `?all=garbage` is passed to an API endpoint schema validation fails and err is nil. Wrapf uses err to create an error message causing a nil pointer dereference. Signed-off-by: Jelle van der Waa <jvanderwaa@redhat.com>
* Add support for mount options to APIJhon Honce2021-08-27
| | | | | | | | When creating containers the specialized mount options where not populated via the API. Fixes: #10831 Signed-off-by: Jhon Honce <jhonce@redhat.com>
* (minor) typo fix: timeout variableEd Santiago2021-07-19
| | | | | | timeout variable was inconsistently spelled in helper function Signed-off-by: Ed Santiago <santiago@redhat.com>
* apiv2 tests: finally fix POST as originally intendedEd Santiago2021-03-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When I originally wrote this code I had no idea what POST would look like so I did a sloppy job, deferring making it usable. Now that we have some real-world examples in place, I have a better understanding of what params look like and how to make tests more readable/maintainable. (Deferring isn't always bad: one of my early ideas was to separate params using commas; that would've been a disaster because some JSON values, such as arrays, include commas). This commit implements a better way of dealing with POST: * The main concept is still 'key=value' * When value is a JSON object (dictionary, array), it can be quoted. * Multiple params are simply separated by spaces. The 3-digit HTTP code is a prominent, readable separator between POST params and expected results. The parsing code is a little uglier, but test developers need never see that. The important thing is that writing tests is now easier. * POST params can be empty (this removes the need for a useless '') I snuck in one unrelated change: one of the newly-added tests, .NetworkSettings, was failing when run rootless (which is how I test on my setup). I made it conditional. Signed-off-by: Ed Santiago <santiago@redhat.com>
* apiv2 tests: add helpers to start/stop a local registryEd Santiago2021-03-08
| | | | | | | | | | | | | | | | | | | ...and a rudimentary set of /auth tests for PR#9589 (disabled). This simply adds a new start_registry() helper function that allocates a random unused port, pulls a registry image, creates a local certificate + random username + random password, and fires everything up. Since none of this is (yet) used in CI, this is very low risk. The only infinitessimally-risky change is using a dedicated subdirectory of $WORKDIR (instead of $WORKDIR itself) as the podman root. This fixes a dumb oversight on my part: the workdir has grown to be used for much more than just podman root; this change removes clutter and makes it easier for humans to debug in cases of problems. Signed-off-by: Ed Santiago <santiago@redhat.com>
* APIv2 tests: make more maintainableEd Santiago2021-03-01
| | | | | | | | | | | | | | | While I wasn't looking, some completely unreadable cruft crept in here, and it's totally my fault: I never knew you could pass JSON to a GET query. Everyone who DID know that, did so, but had to URL-escape it into a completely gobbledygook mess to make curl happy. Solution: trivial, do the URL-escaping in 't' itself. I just never realized that was needed. I'm so sorry. I hope this helps. Signed-off-by: Ed Santiago <santiago@redhat.com>
* APIv2 tests: lots of cleanupEd Santiago2021-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's been a while since I last looked at these; some cruft has crept in, generating noise and hence unreadable test results. Clean it up: * remove pushd/popd in one subtest, replace with 'tar -C'. (Also remove confusing quotation marks). This removes spurious directory names from output. * in like(), show only first line of actual output. Some commands ('tree', 'generate kube') produce voluminous multi-line output, which is super useless and distracting when reading a test run. * Recognize that some queries will not generate output, e.g. HEAD requests and some POSTs. Deal with that. This fixes "curl.result.out: no such file" and "parse error" warnings. * In cleanup, 'podman rm -a' and 'rmi -af'; this gets rid of errors when deleting $WORKDIR. (EBUSY error when root, EPERM when rootless). And, the original reason for poking in here: refactor the wait-for-port part of start_server() into its own helper function, so we can use it when starting a local registry in 12-imagesMore. (Ref: #9270) Signed-off-by: Ed Santiago <santiago@redhat.com>
* Tests: Fix common flakes, and improve apiv2 test logEd Santiago2020-10-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - apiv2 - the 'ten /info requests' test is flaking often, taking ~8 seconds (our limit is 7, up from 5 a few weeks ago). Brent suggested that the first /info call might be expensive, because it needs to access storage. So, let's prime it by running one /info outside the timing loop. And, because even that continues to fail, bump it up to 10 seconds and file #8076 to track the slowdown. - toolbox test - WaitForReady() has timed out, even on one occasion causing a run failure because it failed 3 times. Solution: bump up timeout from 2s to 5s. Not really great, but CI systems are underpowered, and it's not unreasonable that 2s might be too low. - sdnotify test - add a 'podman wait' between stop & rm. This may prevent a "cannot rm container as it is running" race condition. While working on this, Brent and I noticed a few ways that test-apiv2 logging can be improved: - test name: when request is POST, display the jsonified parameters, not the original input ones. This should make it much easier to reproduce failures. - use curl's "--write-out" option to capture http code, content type, and request time. We were getting the first two via grep from logged headers; this is cleaner. And there was no other way to get timing. We now include the timing as X-Response-Time in the log file. - abort on *any* curl error, not just 7 (cannot connect). Any error at all from curl is bad news. Signed-off-by: Ed Santiago <santiago@redhat.com>
* APIv2 tests: try again to fix themEd Santiago2020-10-12
| | | | | | | | | | | | | | | | CI discovered that a lot of networking tests are failing; my fault, for not having run my tests as root on my laptop. Disable those. Also: bump up the ten-request time limit, from 5 to 7 seconds. Looks like something keeps getting slower and slower, but I guess there's not much we can do about it. Also: when we get a mismatch response code (e.g. 500 when we expect 200), dump the response body and skip any subsequent response checks. Signed-off-by: Ed Santiago <santiago@redhat.com>
* APIv2 tests: get them passing againEd Santiago2020-10-12
| | | | | | | | | | | | | | | | | | | | | In the new-Cirrus transition, APIv2 tests were inadvertently disabled. As expected when tests get disabled, they break. This commit fixes some failing tests, and comments out others (with big FIXMEs) because I have neither the expertise nor time to figure out the real problems. The big change to test-apiv2 is due to a recently-added test that looks for an '=' sign in json output. My '=' vs '~' detector completely barfed on that, and there's just no way to make it work in a bash 'case' statement. So, switch to an 'if' with 'expr'. And, unrelated, fix a longstanding (harmless) bug that was issuing spurious "expected" messages to the test log; those should've been going to the full results log. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Use `bash` binary from env instead of /bin/bash for scriptsSascha Grunert2020-08-17
| | | | | | | | It's not possible to run any of the scripts on distributions which do have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env bash` instead. Signed-off-by: Sascha Grunert <sgrunert@suse.com>
* APIv2 tests: usability: better test loggingEd Santiago2020-06-30
| | | | | | | | | | | | | | | | | | | | | | | | test-apiv2 has two basic comparisons of returned JSON: equality and likeness ('=' and '~'). When logging failures, the test runner shows both actual and expected values. When logging success, for '=' there's no need to show both actual and expected. But for '~', it can be helpful (for verifying test correctness) to show the actual returned value. To be specific: old: ok ... .MemTotal~[0-9]\+ new: ok ... .MemTotal ('33509068800') ~ [0-9]\+ old: ok ... .[0].State~\(exited\|stopped\) new: ok ... .[0].State ('exited') ~ \(exited\|stopped\) The main benefit is that a developer or end user can easily see precisely what was returned; this can help confirm that the test is working as intended, and/or help fine-tune how the test is written. Signed-off-by: Ed Santiago <santiago@redhat.com>
* test.apiv2: add testing for image and deal with API returning binaryAlex Jia2020-05-28
| | | | | | | | | | | | | Add testing for displaying image history and exporting image Deal with API returning binary (Content-Type =~ 'octet'). When so, set $output to the output of 'file'. Bug fix: in 't' helper, declare loop var $i as local to avoid contaminating caller Signed-off-by: Ed Santiago <santiago@redhat.com> Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
* Attempt to turn on build_without_cgo testsDaniel J Walsh2020-05-22
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* logformat: handle apiv2 results, add anchor linksEd Santiago2020-04-08
| | | | | | | | | | | | | | apiv2 tests emit TAP-compliant output; recognize it and highlight it the same way we do BATS tests. Add anchor links to TAP output, so other tools (e.g. cirrus-flake-summarize) can link to particular lines And, remove a "-f" from "wait" in test-apiv2; looks like there's some version of bash used in some CI VM that doesn't grok it. Signed-off-by: Ed Santiago <santiago@redhat.com>
* API v2 tests: usability improvementsEd Santiago2020-03-27
| | | | | | | | | | | * Allow for descriptive comment in 't' invocations, making it easier to distinguish similar requests * Include test file basename (eg 40-pods) in 'ok/not ok' line * Always symlink $TMPDIR/test-apiv2.log to latest YYMMDDetc file * Include test result ('ok', 'not ok') in said log * When curl results are JSON, filter them through jq into log Signed-off-by: Ed Santiago <santiago@redhat.com>
* CI: add API v2 testsEd Santiago2020-03-02
| | | | | | | | | | API v2 has been quiet for a few days, and the test script is actually passing. Let's take advantage of this opportunity to get them running in CI. Requires adding a check for cgroupsv2 Signed-off-by: Ed Santiago <santiago@redhat.com>
* apiv2 tests: add more pod tests, timing checkEd Santiago2020-02-25
| | | | | | | | | | | | | | | | Looks like /libpod/pods/create has been fixed to return an actual pod ID. Extend those tests. Also, update timeout in the server command: it's now seconds, not milliseconds. Also, update FIXME comments in /pods/prune . Still doesn't work, but clarify what we're seeing. Also, add a new test that runs ten /info requests and barfs if it takes more than 5 seconds. Signed-off-by: Ed Santiago <santiago@redhat.com>
* API v2 tests: catch up to moving targetEd Santiago2020-02-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots has changed since I first checked this in: * Switch to new podman system service invocation * /containers API has changed drastically * /pods API has some fixes; check for them (e.g. container-exists is now 409 Conflict, not 500) * One test ('?invalidparam=x') still doesn't work; comment it out so we can get everything passing. Also, some work on the test framework itself: * Cleaner port-open testing (the bash /dev/tcp check). * Add a 'podman' function to invoke local podman and log its output. The above two allow us to: * Get rid of stderr special-casing Furthermore: * t() no longer needs leading '.'; this allows jq features such as 'length' and perhaps other filters * special-case handling of 204 and 304: rfc2616 demands that they return no message body; assert that it is so. * new root & rootless helper functions (check server) * remove the "unlikely to work" message for rootless; it seems to be working fine * fix pod tests for rootless * BUT: add a bolder FIXME because the ID field seems wrong Signed-off-by: Ed Santiago <santiago@redhat.com>
* Tests for API v2Ed Santiago2020-01-17
Initial framework for testing the version 2 (HTTP) API. Includes a collection of tests for some of the existing endpoints. Not all tests are currently passing. Signed-off-by: Ed Santiago <santiago@redhat.com>