| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Mostly fix a bad design decision I made early on, re: registry.
old: registry starts once, runs to the end
new: registry is brought up on demand, then stopped
Reason: there are times when we need a password-controlled
registry, and times when we need it open.
As long as I'm in here, I've also cleaned up some confusing code
and fixed things so tests can run rootless again.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
Also:
- It fixes a regression in parsing "images" parameter in
ManifestAddV3 handler.
- Refactors 12-imagesMore.at to use start_registry helper.
- Removes some unsafe "exit 1" statements which skip clean up.
Signed-off-by: Vladimir Kochnev <hashtable@yandex.ru>
|
| |
|
|
|
|
|
| |
* systemctl stop podman.service will now return exit code 0
* Update test framework to support JSON boolean and numeric values
Signed-off-by: Jhon Honce <jhonce@redhat.com>
|
| |
|
|
|
|
|
| |
* Add configuration to add report header for python client used in tests
* Move report headers into the individual test runners vs runner.sh
Signed-off-by: Jhon Honce <jhonce@redhat.com>
|
| |
|
|
|
|
|
|
|
| |
* Ensure meaningful behaviour when called with /v3.x.x semantics
* Change return code to 409 from 500 when client attempts to use an
existing network name
* Update API bats test runner to support /v4.0.0 endpoints by default
Signed-off-by: Jhon Honce <jhonce@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Followup to #12919, which merged while I was writing
review feedback. This actually confirms log output.
This required a minor change to the 't' helper: stripping
NUL chars from the http result.
And, while I'm at it, a bunch of cleanup for running rootless:
- set $CONTAINERS_HELPER_BINARY_DIR, so we can find rootlessport
- add a few conditionals for different expectations
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Update method/function signatures use the manifest list name and
images associated with the operation explicitly, in general
func f(ctx context.Context, manifestListName string,
ImageNames []string, options *fOptions)
* Leverage gorilla/mux Subrouters to support API v3.x and v4.x for
manifests
* Make manifest API endpoints more RESTful
* Add PUT /manifest/{id} to update existing manifests
* Add manifests.Annotate to go bindings, uncommented unit test
* Add DELETE /manifest/{Id} to remove existing manifest list, use
PUT /manifest/{id} to remove images from a list
* Deprecated POST /manifest/{id}/add and /manifest/{id}/remove, use
PUT /manifest/{id} instead
* Corrected swagger godoc and updated to cover API changes
* Update podman manifest commands to use registry.Context()
* Expose utils.GetVar() to obtain query parameters by name
* Unexpose server.registerSwaggerHandlers, not sure why this was ever
exposed.
* Refactored code to use http.Header instead of map[string]string when
operating on HTTP headers.
* Add API-Version header support in bindings to allow calling explicate
versions of the API. Header is _NOT_ forwarded to the API service.
Signed-off-by: Jhon Honce <jhonce@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(i.e. not test failures, but actual programming bugs).
We've had a number of syntax errors creep into this test, usually
caused by a missing backslash on a test command. I've long wanted
to 'set -e' but that causes other problems. This PR introduces
error handling via 'trap', with useful diagnostics on failure.
This PR also catches and fixes two previously-unknown bugs that
were causing tests to not actually run.
And, since /events takes eons on my high-uptime laptop, add /since
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some months ago, apiv2 tests got added that needed new
functionality: passing a tarball to the remote server.
There was no mechanism to do so in the 't' helper, so
these tests used complicated (and actually not-really-
working) curl commands.
This PR introduces and documents a new usage of 't', in
which passing an argument ending in '.tar' adds the
right magic syntax (--data-binary @PATH) to the existing
curl. This lets us use all standard 't' checks, making
for simpler tests and in the process fixing some bugs.
Also: drive-by fix of a typo bug in the networks test.
Also: set CONTAINERS_REGISTRIES_CONF when starting server
and when running direct podman, to avoid docker.io throttling.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
| |
When `?all=garbage` is passed to an API endpoint schema validation fails
and err is nil. Wrapf uses err to create an error message causing a nil
pointer dereference.
Signed-off-by: Jelle van der Waa <jvanderwaa@redhat.com>
|
| |
|
|
|
|
|
|
| |
When creating containers the specialized mount options where not
populated via the API.
Fixes: #10831
Signed-off-by: Jhon Honce <jhonce@redhat.com>
|
| |
|
|
|
|
| |
timeout variable was inconsistently spelled in helper function
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When I originally wrote this code I had no idea what POST
would look like so I did a sloppy job, deferring making it
usable. Now that we have some real-world examples in place,
I have a better understanding of what params look like and
how to make tests more readable/maintainable. (Deferring isn't
always bad: one of my early ideas was to separate params using
commas; that would've been a disaster because some JSON values,
such as arrays, include commas).
This commit implements a better way of dealing with POST:
* The main concept is still 'key=value'
* When value is a JSON object (dictionary, array), it
can be quoted.
* Multiple params are simply separated by spaces.
The 3-digit HTTP code is a prominent, readable separator
between POST params and expected results. The parsing
code is a little uglier, but test developers need
never see that. The important thing is that writing
tests is now easier.
* POST params can be empty (this removes the need for a
useless '')
I snuck in one unrelated change: one of the newly-added
tests, .NetworkSettings, was failing when run rootless
(which is how I test on my setup). I made it conditional.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
...and a rudimentary set of /auth tests for PR#9589 (disabled).
This simply adds a new start_registry() helper function that
allocates a random unused port, pulls a registry image, creates
a local certificate + random username + random password, and
fires everything up. Since none of this is (yet) used in CI,
this is very low risk.
The only infinitessimally-risky change is using a dedicated
subdirectory of $WORKDIR (instead of $WORKDIR itself) as
the podman root. This fixes a dumb oversight on my part:
the workdir has grown to be used for much more than just
podman root; this change removes clutter and makes it
easier for humans to debug in cases of problems.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While I wasn't looking, some completely unreadable cruft
crept in here, and it's totally my fault: I never knew
you could pass JSON to a GET query. Everyone who DID
know that, did so, but had to URL-escape it into a
completely gobbledygook mess to make curl happy.
Solution: trivial, do the URL-escaping in 't' itself. I
just never realized that was needed.
I'm so sorry. I hope this helps.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's been a while since I last looked at these; some cruft
has crept in, generating noise and hence unreadable test
results. Clean it up:
* remove pushd/popd in one subtest, replace with 'tar -C'.
(Also remove confusing quotation marks). This removes
spurious directory names from output.
* in like(), show only first line of actual output.
Some commands ('tree', 'generate kube') produce
voluminous multi-line output, which is super useless
and distracting when reading a test run.
* Recognize that some queries will not generate output,
e.g. HEAD requests and some POSTs. Deal with that.
This fixes "curl.result.out: no such file" and "parse
error" warnings.
* In cleanup, 'podman rm -a' and 'rmi -af'; this gets
rid of errors when deleting $WORKDIR. (EBUSY error
when root, EPERM when rootless).
And, the original reason for poking in here: refactor the
wait-for-port part of start_server() into its own helper
function, so we can use it when starting a local registry
in 12-imagesMore. (Ref: #9270)
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- apiv2 - the 'ten /info requests' test is flaking often,
taking ~8 seconds (our limit is 7, up from 5 a few weeks
ago). Brent suggested that the first /info call might be
expensive, because it needs to access storage. So, let's
prime it by running one /info outside the timing loop.
And, because even that continues to fail, bump it up
to 10 seconds and file #8076 to track the slowdown.
- toolbox test - WaitForReady() has timed out, even on one
occasion causing a run failure because it failed 3 times.
Solution: bump up timeout from 2s to 5s. Not really great,
but CI systems are underpowered, and it's not unreasonable
that 2s might be too low.
- sdnotify test - add a 'podman wait' between stop & rm.
This may prevent a "cannot rm container as it is running"
race condition.
While working on this, Brent and I noticed a few ways that
test-apiv2 logging can be improved:
- test name: when request is POST, display the jsonified
parameters, not the original input ones. This should
make it much easier to reproduce failures.
- use curl's "--write-out" option to capture http code,
content type, and request time. We were getting the
first two via grep from logged headers; this is cleaner.
And there was no other way to get timing. We now include
the timing as X-Response-Time in the log file.
- abort on *any* curl error, not just 7 (cannot connect).
Any error at all from curl is bad news.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CI discovered that a lot of networking tests are failing; my
fault, for not having run my tests as root on my laptop.
Disable those.
Also: bump up the ten-request time limit, from 5 to 7 seconds.
Looks like something keeps getting slower and slower, but I
guess there's not much we can do about it.
Also: when we get a mismatch response code (e.g. 500 when we
expect 200), dump the response body and skip any subsequent
response checks.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the new-Cirrus transition, APIv2 tests were inadvertently
disabled. As expected when tests get disabled, they break.
This commit fixes some failing tests, and comments out others
(with big FIXMEs) because I have neither the expertise nor
time to figure out the real problems.
The big change to test-apiv2 is due to a recently-added
test that looks for an '=' sign in json output. My '=' vs '~'
detector completely barfed on that, and there's just no
way to make it work in a bash 'case' statement. So, switch
to an 'if' with 'expr'.
And, unrelated, fix a longstanding (harmless) bug that was
issuing spurious "expected" messages to the test log; those
should've been going to the full results log.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
| |
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
test-apiv2 has two basic comparisons of returned JSON:
equality and likeness ('=' and '~'). When logging failures,
the test runner shows both actual and expected values. When
logging success, for '=' there's no need to show both actual
and expected. But for '~', it can be helpful (for verifying
test correctness) to show the actual returned value.
To be specific:
old: ok ... .MemTotal~[0-9]\+
new: ok ... .MemTotal ('33509068800') ~ [0-9]\+
old: ok ... .[0].State~\(exited\|stopped\)
new: ok ... .[0].State ('exited') ~ \(exited\|stopped\)
The main benefit is that a developer or end user can
easily see precisely what was returned; this can help
confirm that the test is working as intended, and/or
help fine-tune how the test is written.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Add testing for displaying image history and exporting image
Deal with API returning binary (Content-Type =~ 'octet').
When so, set $output to the output of 'file'.
Bug fix: in 't' helper, declare loop var $i as local
to avoid contaminating caller
Signed-off-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
|
| |
|
|
| |
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
apiv2 tests emit TAP-compliant output; recognize it and
highlight it the same way we do BATS tests.
Add anchor links to TAP output, so other tools (e.g.
cirrus-flake-summarize) can link to particular lines
And, remove a "-f" from "wait" in test-apiv2; looks
like there's some version of bash used in some CI VM
that doesn't grok it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
* Allow for descriptive comment in 't' invocations, making it
easier to distinguish similar requests
* Include test file basename (eg 40-pods) in 'ok/not ok' line
* Always symlink $TMPDIR/test-apiv2.log to latest YYMMDDetc file
* Include test result ('ok', 'not ok') in said log
* When curl results are JSON, filter them through jq into log
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
API v2 has been quiet for a few days, and the test script is
actually passing. Let's take advantage of this opportunity
to get them running in CI.
Requires adding a check for cgroupsv2
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Looks like /libpod/pods/create has been fixed to return an
actual pod ID. Extend those tests.
Also, update timeout in the server command: it's now seconds,
not milliseconds.
Also, update FIXME comments in /pods/prune . Still doesn't
work, but clarify what we're seeing.
Also, add a new test that runs ten /info requests and
barfs if it takes more than 5 seconds.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Lots has changed since I first checked this in:
* Switch to new podman system service invocation
* /containers API has changed drastically
* /pods API has some fixes; check for them (e.g.
container-exists is now 409 Conflict, not 500)
* One test ('?invalidparam=x') still doesn't work;
comment it out so we can get everything passing.
Also, some work on the test framework itself:
* Cleaner port-open testing (the bash /dev/tcp check).
* Add a 'podman' function to invoke local podman and
log its output.
The above two allow us to:
* Get rid of stderr special-casing
Furthermore:
* t() no longer needs leading '.'; this allows jq
features such as 'length' and perhaps other filters
* special-case handling of 204 and 304: rfc2616 demands
that they return no message body; assert that it is so.
* new root & rootless helper functions (check server)
* remove the "unlikely to work" message for rootless;
it seems to be working fine
* fix pod tests for rootless
* BUT: add a bolder FIXME because the ID field seems wrong
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
|
|
Initial framework for testing the version 2 (HTTP) API.
Includes a collection of tests for some of the existing
endpoints. Not all tests are currently passing.
Signed-off-by: Ed Santiago <santiago@redhat.com>
|
