diff options
| author | Valentin Rothberg <vrothberg@redhat.com> | 2022-04-11 13:32:23 +0200 |
|---|---|---|
| committer | Valentin Rothberg <vrothberg@redhat.com> | 2022-04-20 11:31:28 +0200 |
| commit | 0162f678c0e68e9ef0756f8cf521cf14d637be29 (patch) | |
| tree | f0af58bd32bca3c6b77002d4da85c28b2f108765 /Makefile | |
| parent | bc8d8737b760a1f42a9208c6067d2e163817d5f0 (diff) | |
| download | podman-0162f678c0e68e9ef0756f8cf521cf14d637be29.tar.gz podman-0162f678c0e68e9ef0756f8cf521cf14d637be29.tar.bz2 podman-0162f678c0e68e9ef0756f8cf521cf14d637be29.zip | |
benchmarking Podman: proof of concept
Add a proof of concept for benchmarking Podman. The benchmarks are
implemented by means of the end-to-end test suite but hidden behind
a `benchmarks` build tag. Running `make localbenchmarks` will run
`test/e2e` with the specific build tag and set ginkgo's "focus" to
the specific "Podman Benchmark Suite" to only run this spec and skip
all others.
ginkgo will print a report before terminating listing the CPU and memory
stats for each benchmark. New benchmarks can easily be added via the
`newBenchmark` function that also supports adding an `init()` function
to each benchmark which allows for performing certain setups for the
specific benchmark. For instance, benchmarking `podman start` requires
creating a container beforehand.
Podman may be called more than once in the main function of a benchmark
but note that the displayed memory consumption is then a sum of all
Podman invocations. The memory consumption is collected via
`/usr/bin/time`.
A benchmark's report is split into CPU and memory as displayed below:
```
[CPU] podman images:
Fastest Time: 0.146s
Slowest Time: 0.187s
Average Time: 0.180s ± 0.015s
[MEM] podman images:
Smallest: 41892.0KB
Largest: 42792.0KB
Average: 42380.7KB ± 286.4KB
```
Note that the benchmarks are not wired into the CI yet. They are meant
as a proof of concept. More benchmarks and the plumbing into CI will
happen in a later change.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Diffstat (limited to 'Makefile')
| -rw-r--r-- | Makefile | 12 |
1 files changed, 10 insertions, 2 deletions
@@ -553,8 +553,8 @@ test: localunit localintegration remoteintegration localsystem remotesystem ## .PHONY: ginkgo-run ginkgo-run: - $(GOBIN)/ginkgo version - $(GOBIN)/ginkgo -v $(TESTFLAGS) -tags "$(TAGS)" $(GINKGOTIMEOUT) -cover -flakeAttempts 3 -progress -trace -noColor -nodes 3 -debug test/e2e/. $(HACK) + ACK_GINKGO_RC=true $(GOBIN)/ginkgo version + ACK_GINKGO_RC=true $(GOBIN)/ginkgo -v $(TESTFLAGS) -tags "$(TAGS)" $(GINKGOTIMEOUT) -cover -flakeAttempts 3 -progress -trace -noColor -nodes 3 -debug test/e2e/. $(HACK) .PHONY: ginkgo ginkgo: @@ -570,6 +570,14 @@ localintegration: test-binaries ginkgo .PHONY: remoteintegration remoteintegration: test-binaries ginkgo-remote +.PHONY: localbenchmarks +localbenchmarks: test-binaries + ACK_GINKGO_RC=true $(GOBIN)/ginkgo \ + -focus "Podman Benchmark Suite" \ + -tags "$(BUILDTAGS) benchmarks" -noColor \ + -noisySkippings=false -noisyPendings=false \ + test/e2e/. + .PHONY: localsystem localsystem: # Wipe existing config, database, and cache: start with clean slate. |