logformatter: link to bats sources on error

We already link to ginkgo sources, now add links to bats.
Ugly, because we need to hardcode containers/podman (git
repo) and test/system (test file path): those can't be
determined from the log results like they can in ginkgo.

Also, great suggestion from @Luap99: in addition to the
'Annotated results' link which we append to the basic log,
include a short summary of failures. This should help a
viewer see exactly which test(s) failed, which in turn
can be helpful for diagnosing known-flake or real-problem.

Signed-off-by: Ed Santiago <santiago@redhat.com>

2 files changed, 24 insertions, 1 deletions

diff --git a/contrib/cirrus/logformatter b/contrib/cirrus/logformatter
index 43173eaa6..5b8e2f2a1 100755
--- a/contrib/cirrus/logformatter
+++ b/contrib/cirrus/logformatter
@@ -307,10 +307,18 @@ END_HTML
             elsif ($line =~ /^#\s#\|\s/)       { $css = 'log-esm' }
             elsif ($line =~ /^#\s/)            { $css = 'log'     }
 
+            # Link to source file. This is ugly: we have to hardcode 'podman'
+            # and 'test/system' because there's no way to get them from log.
+            #
+            #          1  2      2               13     4         43           5
+            $line =~ s{(in(\stest)?\s+file\s+\S+/)(\S+\.(bats|bash)),\s+line\s+(\d+)}{$1<a class="codelink" href="https://github.com/containers/podman/blob/$git_commit/test/system/$3#L$5">$3, line $5</a>};
+
             if ($css) {
                 # Make it linkable, e.g. foo.html#t--00001
-                if ($line =~ /^(not\s+)?ok\s+(\d+)/) {
+                if ($line =~ /^(not\s+)?ok\s+(\d+)\s+(.*)/) {
                     $line = sprintf("<a name='t--%05d'>%s</a>", $2, $line);
+
+                    push @{$bats_count{__fail_list}}, [ $2, $3 ] if $1;
                 }
                 $line = "<span class='bats-$css'>$line</span>";
 
@@ -515,6 +523,15 @@ END_HTML
         }
     }
 
+    # Grumble. Github only shows the last N lines of the log... which is
+    # anti-helpful when you want a quick synopsis of what failed. Write a
+    # summary at the tail, to make it easier for humans to see what went wrong.
+    if (my $fails = $bats_count{__fail_list}) {
+        print  "\n";
+        printf "Failed tests (%d):\n", scalar(@$fails);
+        printf " - %d %s\n", @$_ for @$fails;
+    }
+
     # If Cirrus magic envariables are available, write a link to results.
     # FIXME: it'd be so nice to make this a clickable live link.
     #
diff --git a/contrib/cirrus/logformatter.t b/contrib/cirrus/logformatter.t
index df8154cb9..7f5973a15 100755
--- a/contrib/cirrus/logformatter.t
+++ b/contrib/cirrus/logformatter.t
@@ -91,6 +91,9 @@ __END__
 ok 1 hi
 ok 2 bye # skip no reason
 not ok 3 fail
+# (from function `assert' in file ./helpers.bash, line 343,
+#  from function `expect_output' in file ./helpers.bash, line 370,
+#  in test file ./run.bats, line 786)
 # $ /path/to/podman foo -bar
 # #| FAIL: exit code is 123; expected 321
 ok 4 blah
@@ -99,6 +102,9 @@ ok 4 blah
 <span class='bats-passed'><a name='t--00001'>ok 1 hi</a></span>
 <span class='bats-skipped'><a name='t--00002'>ok 2 bye # skip no reason</a></span>
 <span class='bats-failed'><a name='t--00003'>not ok 3 fail</a></span>
+<span class='bats-log'># (from function `assert&#39; in file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/helpers.bash#L343">helpers.bash, line 343</a>,</span>
+<span class='bats-log'>#  from function `expect_output&#39; in file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/helpers.bash#L370">helpers.bash, line 370</a>,</span>
+<span class='bats-log'>#  in test file ./<a class="codelink" href="https://github.com/containers/podman/blob/ceci-nest-pas-une-sha/test/system/run.bats#L786">run.bats, line 786</a>)</span>
 <span class='bats-log'># $ <b><span title="/path/to/podman">podman</span> foo -bar</b></span>
 <span class='bats-log-esm'># #| FAIL: exit code is 123; expected 321</span>
 <span class='bats-passed'><a name='t--00004'>ok 4 blah</a></span>