summaryrefslogtreecommitdiff
path: root/contrib/cirrus
diff options
context:
space:
mode:
authorEd Santiago <santiago@redhat.com>2022-03-17 10:19:14 -0600
committerEd Santiago <santiago@redhat.com>2022-03-17 14:25:09 -0600
commitd3e3ea843e48375ac41ea3d0ce1e95cb7d39ea03 (patch)
treec27ebd1b480dc695db1e32f44248df492f456eae /contrib/cirrus
parente59f71d230cf5435dc569add90e22cc644e38647 (diff)
downloadpodman-d3e3ea843e48375ac41ea3d0ce1e95cb7d39ea03.tar.gz
podman-d3e3ea843e48375ac41ea3d0ce1e95cb7d39ea03.tar.bz2
podman-d3e3ea843e48375ac41ea3d0ce1e95cb7d39ea03.zip
logformatter: link to bats sources on error
We already link to ginkgo sources, now add links to bats. Ugly, because we need to hardcode containers/podman (git repo) and test/system (test file path): those can't be determined from the log results like they can in ginkgo. Also, great suggestion from @Luap99: in addition to the 'Annotated results' link which we append to the basic log, include a short summary of failures. This should help a viewer see exactly which test(s) failed, which in turn can be helpful for diagnosing known-flake or real-problem. Signed-off-by: Ed Santiago <santiago@redhat.com>
Diffstat (limited to 'contrib/cirrus')
-rwxr-xr-xcontrib/cirrus/logformatter19
-rwxr-xr-xcontrib/cirrus/logformatter.t6
2 files changed, 24 insertions, 1 deletions
diff --git a/contrib/cirrus/logformatter b/contrib/cirrus/logformatter
index 43173eaa6..5b8e2f2a1 100755
--- a/contrib/cirrus/logformatter
+++ b/contrib/cirrus/logformatter
@@ -307,10 +307,18 @@ END_HTML
elsif ($line =~ /^#\s#\|\s/) { $css = 'log-esm' }
elsif ($line =~ /^#\s/) { $css = 'log' }
+ # Link to source file. This is ugly: we have to hardcode 'podman'
+ # and 'test/system' because there's no way to get them from log.
+ #
+ # 1 2 2 13 4 43 5
+ $line =~ s{(in(\stest)?\s+file\s+\S+/)(\S+\.(bats|bash)),\s+line\s+(\d+)}{$1<a class="codelink" href="https://github.com/containers/podman/blob/$git_commit/test/system/$3#L$5">$3, line $5</a>};
+
if ($css) {
# Make it linkable, e.g. foo.html#t--00001
- if ($line =~ /^(not\s+)?ok\s+(\d+)/) {
+ if ($line =~ /^(not\s+)?ok\s+(\d+)\s+(.*)/) {
$line = sprintf("<a name='t--%05d'>%s</a>", $2, $line);
+
+ push @{$bats_count{__fail_list}}, [ $2, $3 ] if $1;
}
$line = "<span class='bats-$css'>$line</span>";
@@ -515,6 +523,15 @@ END_HTML
}
}
+ # Grumble. Github only shows the last N lines of the log... which is
+ # anti-helpful when you want a quick synopsis of what failed. Write a
+ # summary at the tail, to make it easier for humans to see what went wrong.
+ if (my $fails = $bats_count{__fail_list}) {
+ print "\n";
+ printf "Failed tests (%d):\n", scalar(@$fails);
+ printf " - %d %s\n", @$_ for @$fails;
+ }
+
# If Cirrus magic envariables are available, write a link to results.
# FIXME: it'd be so nice to make this a clickable live link.
#
diff --git a/contrib/cirrus/logformatter.t b/contrib/cirrus/logformatter.t
index df8154cb9..7f5973a15 100755
--- a/contrib/cirrus/logformatter.t
+++ b/contrib/cirrus/logformatter.t
@@ -91,6 +91,9 @@ __END__
ok 1 hi
ok 2 bye # skip no reason
not ok 3 fail
+# (from function `assert' in file ./helpers.bash, line 343,
+# from function `expect_output' in file ./helpers.bash, line 370,
+# in test file ./run.bats, line 786)
# $ /path/to/podman foo -bar
# #| FAIL: exit code is 123; expected 321
ok 4 blah
@@ -99,6 +102,9 @@ ok 4 blah
<span class='bats-passed'><a name='t--00001'>ok 1 hi</a></span>
<span class='bats-skipped'><a name='t--00002'>ok 2 bye # skip no reason</a></span>
<span class='bats-failed'><a name='t--00003'>not ok 3 fail</a></span>
+<span class='bats-log'># (from function `assert&#39; in file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/helpers.bash#L343">helpers.bash, line 343</a>,</span>
+<span class='bats-log'># from function `expect_output&#39; in file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/helpers.bash#L370">helpers.bash, line 370</a>,</span>
+<span class='bats-log'># in test file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/run.bats#L786">run.bats, line 786</a>)</span>
<span class='bats-log'># $ <b><span title="/path/to/podman">podman</span> foo -bar</b></span>
<span class='bats-log-esm'># #| FAIL: exit code is 123; expected 321</span>
<span class='bats-passed'><a name='t--00004'>ok 4 blah</a></span>