summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorEd Santiago <santiago@redhat.com>2021-04-13 15:06:30 -0600
committerEd Santiago <santiago@redhat.com>2021-04-13 15:18:34 -0600
commit31e31aa9dd59668d3dee1569111ac0a793c68a61 (patch)
treefeedc7a93e24c52dc0f779307779710c0d6dc5ea
parent21d6b12689cc38823fef3772327990f9692d4379 (diff)
downloadpodman-31e31aa9dd59668d3dee1569111ac0a793c68a61.tar.gz
podman-31e31aa9dd59668d3dee1569111ac0a793c68a61.tar.bz2
podman-31e31aa9dd59668d3dee1569111ac0a793c68a61.zip
compose test: try to get useful data from flakes
docker-compose test continues to flake even after #9961. Let's try to get some useful data from the failures, by: * adding -S (--show-error) to curl. With just -s (--silent), curl is completely quiet. With -S, it displays errors. (Not in TAP form, but I'm OK with that) * oops, adding safety checks to the fix from #9961 (it was inadvertently clobbering the curl exit status) And, as long as I'm in this code: logformatter was not highlighting these results, because the '1..N' TAP line needs to be spit out at the end. Have test-compose emit a 'TAP' header <http://testanything.org/> and make logformatter recognize it. Signed-off-by: Ed Santiago <santiago@redhat.com>
-rwxr-xr-xcontrib/cirrus/logformatter10
-rwxr-xr-xtest/compose/test-compose23
2 files changed, 23 insertions, 10 deletions
diff --git a/contrib/cirrus/logformatter b/contrib/cirrus/logformatter
index 3fa0e5618..5156f9f8a 100755
--- a/contrib/cirrus/logformatter
+++ b/contrib/cirrus/logformatter
@@ -243,11 +243,17 @@ END_HTML
$cirrus_task = $1;
}
- # BATS handling (used also for apiv2 tests, which emit TAP output)
- if ($line =~ /^1\.\.(\d+)$/ || $line =~ m!/test-apiv2!) {
+ # BATS handling. This will recognize num_tests both at start and end
+ if ($line =~ /^1\.\.(\d+)$/) {
$looks_like_bats = 1;
$bats_count{expected_total} = $1;
}
+ # Since the number of tests can't always be predicted, recognize
+ # some leading text strings that indicate BATS output to come.
+ elsif ($line =~ /^TAP\s+version\s/ || $line =~ m!/test-apiv2!) {
+ $looks_like_bats = 1;
+ $bats_count{expected_total} = -1; # Expect to be overridden at end!
+ }
if ($looks_like_bats) {
my $css;
diff --git a/test/compose/test-compose b/test/compose/test-compose
index 7693041ac..abb957b43 100755
--- a/test/compose/test-compose
+++ b/test/compose/test-compose
@@ -163,18 +163,21 @@ function test_port() {
local op="$2" # '=' or '~'
local expect="$3" # what to expect from curl output
- local actual=$(curl --retry 3 --retry-all-errors -s http://127.0.0.1:$port/)
- # The test is flaking with an empty result. The curl retry doesn't solve this.
- # If the result is empty sleep one second and try again.
- if [[ "$actual" == "" ]]; then
+ # -s -S means "silent, but show errors"
+ local actual=$(curl --retry 3 --retry-all-errors -s -S http://127.0.0.1:$port/)
+ local curl_rc=$?
+
+ # FIXME 2021-04-13: test is flaking, curl succeeds but returns empty result.
+ # Could it be that the container is not actually ready? Wait, and retry.
+ if [[ $curl_rc -eq 0 && -z "$actual" ]]; then
sleep 1
- local actual=$(curl --retry 3 --retry-all-errors -s http://127.0.0.1:$port/)
+ echo "# Retrying curl:"
+ actual=$(curl --retry 3 --retry-all-errors -s -S http://127.0.0.1:$port/)
+ curl_rc=$?
fi
- local curl_rc=$?
+
if [ $curl_rc -ne 0 ]; then
_show_ok 0 "$testname - curl failed with status $curl_rc"
-### docker-compose down >>$logfile 2>&1
-### exit 1
fi
case "$op" in
@@ -285,6 +288,10 @@ fi
# Too hard to precompute the number of tests; just spit it out at the end.
n_tests=0
+
+# We aren't really TAP 13; this helps logformatter recognize our output as BATS
+echo "TAP version 13"
+
for t in ${tests_to_run[@]}; do
testdir="$(dirname $t)"
testname="$(basename $testdir)"