aboutsummaryrefslogtreecommitdiff
path: root/pkg/checkpoint
Commit message (Collapse)AuthorAge
* Add support for checkpoint imageRadostin Stoyanov2022-04-20
| | | | | | | | | | | | | | | | | | | | | | | | This is an enhancement proposal for the checkpoint / restore feature of Podman that enables container migration across multiple systems with standard image distribution infrastructure. A new option `--create-image <image>` has been added to the `podman container checkpoint` command. This option tells Podman to create a container image. This is a standard image with a single layer, tar archive, that that contains all checkpoint files. This is similar to the current approach with checkpoint `--export`/`--import`. This image can be pushed to a container registry and pulled on a different system. It can also be exported locally with `podman image save` and inspected with `podman inspect`. Inspecting the image would display additional information about the host and the versions of Podman, criu, crun/runc, kernel, etc. `podman container restore` has also been extended to support image name or ID as input. Suggested-by: Adrian Reber <areber@redhat.com> Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
* fix a number of errcheck issuesValentin Rothberg2022-03-22
| | | | | | Numerous issues remain, especially in tests/e2e. Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
* linter: enable nilerrValentin Rothberg2022-03-22
| | | | | | | A number of cases looked suspicious, so I marked them with `FIXME`s to leave some breadcrumbs. Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
* Fix checkpoint/restore pod testsAdrian Reber2022-02-11
| | | | | | | | | Checkpoint/restore pod tests are not running with an older runc and now that runc 1.1.0 appears in the repositories it was detected that the tests were failing. This was not detected in CI as CI was not using runc 1.1.0 yet. Signed-off-by: Adrian Reber <areber@redhat.com>
* bump go module to version 4Valentin Rothberg2022-01-18
| | | | | | | | | | | | | Automated for .go files via gomove [1]: `gomove github.com/containers/podman/v3 github.com/containers/podman/v4` Remaining files via vgrep [2]: `vgrep github.com/containers/podman/v3` [1] https://github.com/KSubedi/gomove [2] https://github.com/vrothberg/vgrep Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* container restore/import: store networks from dbPaul Holzinger2021-12-14
| | | | | | | | It is important that we store the current networks from the db in the config. Also make sure to properly handle aliases and ignore static ip/mac addresses. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Use same runtime to restore a container as during checkpointingAdrian Reber2021-11-19
| | | | | | | | | | | | | | | | | | | | | | There are at least two runtimes that support checkpoint and restore: runc and crun. Although the checkpoints created by these are almost compatible, it is not (yet) possible to restore a checkpoint created with one runtime with the other runtime. To make checkpoint/restore usage more comfortable this adds code to look into the checkpoint archive during restore and to set the runtime to the one used during checkpointing. This also adds a check, if the user explicitly sets a runtime during restore, that the runtime is also the same as used during checkpointing. If a different runtime is selected than the one used during checkpointing the restore will fail early. If runc and crun will create compatible checkpoints in the future the check can be changed to treat crun and runc as compatible checkpoint/restore runtimes. Signed-off-by: Adrian Reber <areber@redhat.com>
* Do not store the exit command in container configPaul Holzinger2021-11-18
| | | | | | | | | | | | | | | | | | | There is a problem with creating and storing the exit command when the container was created. It only contains the options the container was created with but NOT the options the container is started with. One example would be a CNI network config. If I start a container once, then change the cni config dir with `--cni-config-dir` ans start it a second time it will start successfully. However the exit command still contains the wrong `--cni-config-dir` because it was not updated. To fix this we do not want to store the exit command at all. Instead we create it every time the conmon process for the container is startet. This guarantees us that the container cleanup process is startet with the correct settings. [NO NEW TESTS NEEDED] Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* fix remote checkpoint/restoreValentin Rothberg2021-11-16
| | | | | | | | | Nothing was working before, and it's too much to summarize. To make sure we're not regressing in the future again, enable the remote e2e tests. Fixes: #12007 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Merge pull request #12184 from adrianreber/2021-11-05-stats-dumpOpenShift Merge Robot2021-11-08
|\ | | | | Add 'stats-dump' file to exported checkpoint
| * Add 'stats-dump' file to exported checkpointAdrian Reber2021-11-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There was the question about how long it takes to create a checkpoint. CRIU already provides some statistics about how long it takes to create a checkpoint and similar. With this change the file 'stats-dump' is included in the checkpoint archive and the tool checkpointctl can be used to display these statistics: ./checkpointctl show -t /tmp/cp.tar --print-stats Displaying container checkpoint data from /tmp/dump.tar [...] CRIU dump statistics +---------------+-------------+--------------+---------------+---------------+---------------+ | FREEZING TIME | FROZEN TIME | MEMDUMP TIME | MEMWRITE TIME | PAGES SCANNED | PAGES WRITTEN | +---------------+-------------+--------------+---------------+---------------+---------------+ | 105405 us | 1376964 us | 504399 us | 446571 us | 492153 | 88689 | +---------------+-------------+--------------+---------------+---------------+---------------+ Signed-off-by: Adrian Reber <areber@redhat.com>
* | libpod: deduplicate ports in dbPaul Holzinger2021-10-27
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The OCICNI port format has one big problem: It does not support ranges. So if a users forwards a range of 1k ports with podman run -p 1001-2000 we have to store each of the thousand ports individually as array element. This bloats the db and makes the JSON encoding and decoding much slower. In many places we already use a better port struct type which supports ranges, e.g. `pkg/specgen` or the new network interface. Because of this we have to do many runtime conversions between the two port formats. If everything uses the new format we can skip the runtime conversions. This commit adds logic to replace all occurrences of the old format with the new one. The database will automatically migrate the ports to new format when the container config is read for the first time after the update. The `ParsePortMapping` function is `pkg/specgen/generate` has been reworked to better work with the new format. The new logic is able to deduplicate the given ports. This is necessary the ensure we store them efficiently in the DB. The new code should also be more performant than the old one. To prove that the code is fast enough I added go benchmarks. Parsing 1 million ports took less than 0.5 seconds on my laptop. Benchmark normalize PortMappings in specgen: Please note that the 1 million ports are actually 20x 50k ranges because we cannot have bigger ranges than 65535 ports. ``` $ go test -bench=. -benchmem ./pkg/specgen/generate/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/pkg/specgen/generate cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz BenchmarkParsePortMappingNoPorts-12 480821532 2.230 ns/op 0 B/op 0 allocs/op BenchmarkParsePortMapping1-12 38972 30183 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMapping100-12 18752 60688 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMapping1k-12 3104 331719 ns/op 223840 B/op 3018 allocs/op BenchmarkParsePortMapping10k-12 376 3122930 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMapping1m-12 3 390869926 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingReverse100-12 18940 63414 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMappingReverse1k-12 3015 362500 ns/op 223841 B/op 3018 allocs/op BenchmarkParsePortMappingReverse10k-12 343 3318135 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMappingReverse1m-12 3 403392469 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingRange1-12 37635 28756 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange100-12 39604 28935 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1k-12 38384 29921 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange10k-12 29479 40381 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1m-12 927 1279369 ns/op 143022 B/op 164 allocs/op PASS ok github.com/containers/podman/v3/pkg/specgen/generate 25.492s ``` Benchmark convert old port format to new one: ``` go test -bench=. -benchmem ./libpod/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/libpod cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz Benchmark_ocicniPortsToNetTypesPortsNoPorts-12 663526126 1.663 ns/op 0 B/op 0 allocs/op Benchmark_ocicniPortsToNetTypesPorts1-12 7858082 141.9 ns/op 72 B/op 2 allocs/op Benchmark_ocicniPortsToNetTypesPorts10-12 2065347 571.0 ns/op 536 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts100-12 138478 8641 ns/op 4216 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1k-12 9414 120964 ns/op 41080 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts10k-12 781 1490526 ns/op 401528 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1m-12 4 250579010 ns/op 40001656 B/op 4 allocs/op PASS ok github.com/containers/podman/v3/libpod 11.727s ``` Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* standardize logrus messages to upper caseDaniel J Walsh2021-09-22
| | | | | | | | Remove ERROR: Error stutter from logrus messages also. [ NO TESTS NEEDED] This is just code cleanup. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Support checkpoint/restore with podsAdrian Reber2021-07-27
| | | | | | | | | | | | | | | | | | | This adds support to checkpoint containers out of pods and restore container into pods. It is only possible to restore a container into a pod if it has been checkpointed out of pod. It is also not possible to restore a non pod container into a pod. The main reason this does not work is the PID namespace. If a non pod container is being restored in a pod with a shared PID namespace, at least one process in the restored container uses PID 1 which is already in use by the infrastructure container. If someone tries to restore container from a pod with a shared PID namespace without a shared PID namespace it will also fail because the resulting PID namespace will not have a PID 1. Signed-off-by: Adrian Reber <areber@redhat.com>
* Allow changing of port forward rules on restoreAdrian Reber2021-06-04
| | | | | | | | | | | | | | | | | | | | Restored containers, until now, had the same port mappings as the original started container. This commit adds the parameter '--publish' to 'podman container restore' with the same semantic as during create/run. With this change it is possible to create a copy from a container with a '--publish' rule and replace the original '--publish' setting with a new one. # podman run -p 2345:8080 container # podman container checkpoint -l --export=dump.tar # podman container restore -p 5432:8080 --import=dump.tar The restored container will now listen on localhost:5432 instead of localhost:2345 as the original created container. Signed-off-by: Adrian Reber <areber@redhat.com>
* migrate Podman to containers/common/libimageValentin Rothberg2021-05-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Migrate the Podman code base over to `common/libimage` which replaces `libpod/image` and a lot of glue code entirely. Note that I tried to leave bread crumbs for changed tests. Miscellaneous changes: * Some errors yield different messages which required to alter some tests. * I fixed some pre-existing issues in the code. Others were marked as `//TODO`s to prevent the PR from exploding. * The `NamesHistory` of an image is returned as is from the storage. Previously, we did some filtering which I think is undesirable. Instead we should return the data as stored in the storage. * Touched handlers use the ABI interfaces where possible. * Local image resolution: previously Podman would match "foo" on "myfoo". This behaviour has been changed and Podman will now only match on repository boundaries such that "foo" would match "my/foo" but not "myfoo". I consider the old behaviour to be a bug, at the very least an exotic corner case. * Futhermore, "foo:none" does *not* resolve to a local image "foo" without tag anymore. It's a hill I am (almost) willing to die on. * `image prune` prints the IDs of pruned images. Previously, in some cases, the names were printed instead. The API clearly states ID, so we should stick to it. * Compat endpoint image removal with _force_ deletes the entire not only the specified tag. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Use functions and defines from checkpointctlAdrian Reber2021-03-02
| | | | | | | | No functional changes. [NO TESTS NEEDED] - only moving code around Signed-off-by: Adrian Reber <areber@redhat.com>
* Move checkpoint/restore code to pkg/checkpoint/crutilsAdrian Reber2021-03-02
| | | | | | | | | | | To be able to reuse common checkpoint/restore functions this commit moves code to pkg/checkpoint/crutils. This commit has not functional changes. It only moves code around. [NO TESTS NEEDED] - only moving code around Signed-off-by: Adrian Reber <areber@redhat.com>
* Correct compat images/create?fromImage responseMilivoje Legenovic2021-02-26
| | | | Signed-off-by: Milivoje Legenovic <m.legenovic@gmail.com>
* bump go module to v3Valentin Rothberg2021-02-22
| | | | | | | | | We missed bumping the go module, so let's do it now :) * Automated go code with github.com/sirkon/go-imports-rename * Manually via `vgrep podman/v2` the rest Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Include named volumes in container migrationRadostin Stoyanov2021-01-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When migrating a container with associated volumes, the content of these volumes should be made available on the destination machine. This patch enables container checkpoint/restore with named volumes by including the content of volumes in checkpoint file. On restore, volumes associated with container are created and their content is restored. The --ignore-volumes option is introduced to disable this feature. Example: # podman container checkpoint --export checkpoint.tar.gz <container> The content of all volumes associated with the container are included in `checkpoint.tar.gz` # podman container checkpoint --export checkpoint.tar.gz --ignore-volumes <container> The content of volumes is not included in `checkpoint.tar.gz`. This is useful, for example, when the checkpoint/restore is performed on the same machine. # podman container restore --import checkpoint.tar.gz The associated volumes will be created and their content will be restored. Podman will exit with an error if volumes with the same name already exist on the system or the content of volumes is not included in checkpoint.tar.gz # podman container restore --ignore-volumes --import checkpoint.tar.gz Volumes associated with container must already exist. Podman will not create them or restore their content. Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
* Use Options as CRImportCheckpoint() argumentRadostin Stoyanov2021-01-07
| | | | | | | | | Instead of specifying restore option arguments individually from RestoreOptions, provide the 'options' object to the CRImportCheckpoint method. This change makes the code in CRImportCheckpoint easier to extend as it doesn't require excessive number of function parameters. Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
* Lowercase some errorsKir Kolyshkin2020-10-05
| | | | | | | | | | | | | | | | | | | | | This commit is courtesy of ``` for f in $(git ls-files *.go | grep -v ^vendor/); do \ sed -i 's/\(errors\..*\)"Error /\1"error /' $f; done for f in $(git ls-files *.go | grep -v ^vendor/); do \ sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f; done ``` etc. Self-reviewed using `git diff --word-diff`, found no issues. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* Remove excessive error wrappingKir Kolyshkin2020-10-05
| | | | | | | | | | | | | | | | | In case os.Open[File], os.Mkdir[All], ioutil.ReadFile and the like fails, the error message already contains the file name and the operation that fails, so there is no need to wrap the error with something like "open %s failed". While at it - replace a few places with os.Open, ioutil.ReadAll with ioutil.ReadFile. - replace errors.Wrapf with errors.Wrap for cases where there are no %-style arguments. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* Switch all references to github.com/containers/libpod -> podmanDaniel J Walsh2020-07-28
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* move go module to v2Valentin Rothberg2020-07-06
| | | | | | | | | | | | | | | With the advent of Podman 2.0.0 we crossed the magical barrier of go modules. While we were able to continue importing all packages inside of the project, the project could not be vendored anymore from the outside. Move the go module to new major version and change all imports to `github.com/containers/libpod/v2`. The renaming of the imports was done via `gomove` [1]. [1] https://github.com/KSubedi/gomove Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* podmanv2 checkpoint and restoreBrent Baude2020-04-03
add the ability to checkpoint and restore containers on v2podman Signed-off-by: Brent Baude <bbaude@redhat.com>