diff options
139 files changed, 2315 insertions, 2380 deletions
diff --git a/.cirrus.yml b/.cirrus.yml index c984c8859..cf97f4467 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -101,6 +101,10 @@ ext_svc_check_task: else git reset --hard $CIRRUS_CHANGE_IN_REPO fi + # Some test operations & checks require a git "identity" + _gc='git config --file /root/.gitconfig' + $_gc user.email "TMcTestFace@example.com" + $_gc user.name "Testy McTestface" make install.tools setup_script: &setup '$GOSRC/$SCRIPT_BASE/setup_environment.sh' @@ -5,7 +5,7 @@ Podman (the POD MANager) is a tool for managing containers and images, volumes mounted into those containers, and pods made from groups of containers. Podman is based on libpod, a library for container lifecycle management that is also contained in this repository. The libpod library provides APIs for managing containers, pods, container images, and volumes. -* [Latest Version: 4.0.0](https://github.com/containers/podman/releases/latest) +* [Latest Version: 4.0.3](https://github.com/containers/podman/releases/tag/v4.0.3) * Latest Remote client for Windows * Latest Remote client for macOS * Latest Static Remote client for Linux diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 4c07b033a..e7a496250 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,6 +1,62 @@ # Release Notes +## 4.0.3 +### Security +- This release fixes CVE-2022-27649, where containers run by Podman would have excess inheritable capabilities set. + +### Changes +- The `podman machine rm --force` command will now remove running machines as well (such machines are shut down first, then removed) ([#13448](https://github.com/containers/podman/issues/13448)). +- When a `podman machine` VM is started that is using a too-old VM image, it will now start in a reduced functionality mode, and provide instructions on how to recreate it (previously, VMs were effectively unusable) ([#13510](https://github.com/containers/podman/issues/13510)). + +### Bugfixes +- Fixed a bug where devices added to containers by the `--device` option to `podman run` and `podman create` would not be accessible within the container. +- Fixed a bug where Podman would refuse to create containers when the working directory in the container was a symlink ([#13346](https://github.com/containers/podman/issues/13346)). +- Fixed a bug where pods would be created with cgroups even if cgroups were disabled in `containers.conf` ([#13411](https://github.com/containers/podman/issues/13411)). +- Fixed a bug where the `podman play kube` command would produce confusing errors if invalid YAML with duplicated container named was passed ([#13332](https://github.com/containers/podman/issues/13332)). +- Fixed a bug where the `podman machine rm` command would not remove the Podman API socket on the host that was associated with the VM. +- Fixed a bug where the remote Podman client was unable to properly resize the TTYs of containers on non-Linux OSes. +- Fixed a bug where rootless Podman could hang indefinitely when starting containers on systems with IPv6 disabled ([#13388](https://github.com/containers/podman/issues/13388)). +- Fixed a bug where the `podman version` command could sometimes print excess blank lines as part of its output. +- Fixed a bug where the `podman generate systemd` command would sometimes generate systemd services with names beginning with a hyphen ([#13272](https://github.com/containers/podman/issues/13272)). +- Fixed a bug where locally building the pause image could fail if the current directory contained a `.dockerignore` file ([#13529](https://github.com/containers/podman/issues/13529)). +- Fixed a bug where root containers in VMs created by `podman machine` could not bind ports to specific IPs on the host ([#13543](https://github.com/containers/podman/issues/13543)). +- Fixed a bug where the storage utilization percentages displayed by `podman system df` were incorrect ([#13516](https://github.com/containers/podman/issues/13516)). +- Fixed a bug where the CPU utilization percentages displayed by `podman stats` were incorrect ([#13597](https://github.com/containers/podman/pull/13597)). +- Fixed a bug where containers created with the `--no-healthcheck` option would still display healthcheck status in `podman inspect` ([#13578](https://github.com/containers/podman/issues/13578)). +- Fixed a bug where the `podman pod rm` command could print a warning about a missing cgroup ([#13382](https://github.com/containers/podman/issues/13382)). +- Fixed a bug where the `podman exec` command could sometimes print a `timed out waiting for file` error after the process in the container exited ([#13227](https://github.com/containers/podman/issues/13227)). +- Fixed a bug where virtual machines created by `podman machine` were not tolerant of changes to the path to the qemu binary on the host ([#13394](https://github.com/containers/podman/issues/13394)). +- Fixed a bug where the remote Podman client's `podman build` command did not properly handle the context directory if a Containerfile was manually specified using `-f` ([#13293](https://github.com/containers/podman/issues/13293)). +- Fixed a bug where Podman would not properly detect the use of `systemd` as PID 1 in a container when the entrypoint was prefixed with `/bin/sh -c` ([#13324](https://github.com/containers/podman/issues/13324)). +- Fixed a bug where rootless Podman could, on systems that do not use `systemd` as init, print a warning message about the rootless network namespace ([#13703](https://github.com/containers/podman/issues/13703)). +- Fixed a bug where the default systemd unit file for `podman system service` did not delegate all cgroup controllers, resulting in `podman info` queries against the remote API returning incorrect cgroup controllers ([#13710](https://github.com/containers/podman/issues/13710)). +- Fixed a bug where the `slirp4netns` port forwarder for rootless Podman would only publish the first port of a range ([#13643](https://github.com/containers/podman/issues/13643)). + +### API +- Fixed a bug where the Compat Create API for containers did not properly handle permissions for tmpfs mounts ([#13108](https://github.com/containers/podman/issues/13108)). + +### Misc +- The static binary for Linux is now built with CGo disabled to avoid panics due to a Golang bug ([#13557](https://github.com/containers/podman/issues/13557)). +- Updated Buildah to v1.24.3 +- Updated the containers/storage library to v1.38.3 +- Updated the containers/image library to v5.19.2 +- Updated the containers/common library to v0.47.5 + +## 4.0.2 +### Bugfixes +- Revert "use GetRuntimeDir() from c/common" + +## 4.0.1 +### Bugfixes +- Fixed a bug where the `podman play kube` command did not honor the `mountPropagation` field in Pod YAML ([#13322](https://github.com/containers/podman/issues/13322)). +- Fixed a bug where the `--build=false` option to `podman play kube` was not honored ([#13285](https://github.com/containers/podman/issues/13285)). +- Fixed a bug where a container using volumes from another container (via `--volumes-from`) could, under certain circumstances, exit with errors that it could not delete some volumes if the other container did not exit before it ([#12808](https://github.com/containers/podman/issues/12808)). +- Fixed a bug where the `CONTAINERS_CONF` environment variable was not propagated to Conmon, which could result in Podman cleanup processes being run with incorrect configurations. + ## 4.0.0 +### Security +- This release addresses CVE-2022-1227, where running `podman top` on a container made from a maliciously-crafted image and using a user namespace could allow for code execution in the host context. + ### Features - Podman has seen an extensive rewrite of its network stack to add support for Netavark, a new tool for configuring container networks, in addition to the existing CNI stack. Netavark will be default on new installations when it is available. - The `podman network connect` command now supports three new options, `--ip`, `--ip6`, and `--mac-address`, to specify configuration for the new network that will be attached. @@ -196,6 +252,24 @@ - Updated the containers/common library to v0.47.1 - Updated the containers/psgo library to v1.7.2 +## 3.4.7 +### Security +- This release addresses CVE-2022-1227, where running `podman top` on a container made from a maliciously-crafted image and using a user namespace could allow for code execution in the host context. + +## 3.4.6 +### Security +- This release addresses CVE-2022-27191, where an attacker could potentially cause crashes in remote Podman by using incorrect SSH ciphers. + +## 3.4.5 +### Security +- This release addresses CVE-2022-27649, where Podman would set excess inheritable capabilities for processes in containers. + +### Bugfixes +- Fixed a bug where the `podman images` command could, under some circumstances, take an excessive amount of time to list images ([#11997](https://github.com/containers/podman/issues/11997)). + +### Misc +- Updates the containers/common library to v0.44.5 + ## 3.4.4 ### Bugfixes - Fixed a bug where the `podman exec` command would, under some circumstances, print a warning message about failing to move `conmon` to the appropriate cgroup ([#12535](https://github.com/containers/podman/issues/12535)). diff --git a/cmd/podman/containers/checkpoint.go b/cmd/podman/containers/checkpoint.go index f24e77106..40d689c4d 100644 --- a/cmd/podman/containers/checkpoint.go +++ b/cmd/podman/containers/checkpoint.go @@ -68,6 +68,10 @@ func init() { flags.BoolVarP(&checkpointOptions.PreCheckPoint, "pre-checkpoint", "P", false, "Dump container's memory information only, leave the container running") flags.BoolVar(&checkpointOptions.WithPrevious, "with-previous", false, "Checkpoint container with pre-checkpoint images") + createImageFlagName := "create-image" + flags.StringVarP(&checkpointOptions.CreateImage, createImageFlagName, "", "", "Create checkpoint image with specified name") + _ = checkpointCommand.RegisterFlagCompletionFunc(createImageFlagName, completion.AutocompleteNone) + flags.StringP("compress", "c", "zstd", "Select compression algorithm (gzip, none, zstd) for checkpoint archive.") _ = checkpointCommand.RegisterFlagCompletionFunc("compress", common.AutocompleteCheckpointCompressType) diff --git a/cmd/podman/containers/restore.go b/cmd/podman/containers/restore.go index 3b51f5f17..eeda5a05f 100644 --- a/cmd/podman/containers/restore.go +++ b/cmd/podman/containers/restore.go @@ -10,9 +10,9 @@ import ( "github.com/containers/podman/v4/cmd/podman/registry" "github.com/containers/podman/v4/cmd/podman/utils" "github.com/containers/podman/v4/cmd/podman/validate" + "github.com/containers/podman/v4/libpod/define" "github.com/containers/podman/v4/pkg/domain/entities" "github.com/containers/podman/v4/pkg/rootless" - "github.com/pkg/errors" "github.com/spf13/cobra" ) @@ -23,15 +23,16 @@ var ( Restores a container from a checkpoint. The container name or ID can be used. ` restoreCommand = &cobra.Command{ - Use: "restore [options] CONTAINER [CONTAINER...]", + Use: "restore [options] CONTAINER|IMAGE [CONTAINER|IMAGE...]", Short: "Restores one or more containers from a checkpoint", Long: restoreDescription, RunE: restore, Args: func(cmd *cobra.Command, args []string) error { return validate.CheckAllLatestAndCIDFile(cmd, args, true, false) }, - ValidArgsFunction: common.AutocompleteContainers, + ValidArgsFunction: common.AutocompleteContainersAndImages, Example: `podman container restore ctrID + podman container restore imageID podman container restore --latest podman container restore --all`, } @@ -60,7 +61,7 @@ func init() { _ = restoreCommand.RegisterFlagCompletionFunc(importFlagName, completion.AutocompleteDefault) nameFlagName := "name" - flags.StringVarP(&restoreOptions.Name, nameFlagName, "n", "", "Specify new name for container restored from exported checkpoint (only works with --import)") + flags.StringVarP(&restoreOptions.Name, nameFlagName, "n", "", "Specify new name for container restored from exported checkpoint (only works with image or --import)") _ = restoreCommand.RegisterFlagCompletionFunc(nameFlagName, completion.AutocompleteNone) importPreviousFlagName := "import-previous" @@ -78,7 +79,7 @@ func init() { ) _ = restoreCommand.RegisterFlagCompletionFunc("publish", completion.AutocompleteNone) - flags.StringVar(&restoreOptions.Pod, "pod", "", "Restore container into existing Pod (only works with --import)") + flags.StringVar(&restoreOptions.Pod, "pod", "", "Restore container into existing Pod (only works with image or --import)") _ = restoreCommand.RegisterFlagCompletionFunc("pod", common.AutocompletePodsRunning) flags.BoolVar( @@ -95,25 +96,51 @@ func restore(cmd *cobra.Command, args []string) error { var errs utils.OutputErrors podmanStart := time.Now() if rootless.IsRootless() { - return errors.New("restoring a container requires root") + return fmt.Errorf("restoring a container requires root") } - if restoreOptions.Import == "" && restoreOptions.ImportPrevious != "" { - return errors.Errorf("--import-previous can only be used with --import") + + // Find out if this is an image + inspectOpts := entities.InspectOptions{} + imgData, _, err := registry.ImageEngine().Inspect(context.Background(), args, inspectOpts) + if err != nil { + return err } - if restoreOptions.Import == "" && restoreOptions.IgnoreRootFS { - return errors.Errorf("--ignore-rootfs can only be used with --import") + + hostInfo, err := registry.ContainerEngine().Info(context.Background()) + if err != nil { + return err } - if restoreOptions.Import == "" && restoreOptions.IgnoreVolumes { - return errors.Errorf("--ignore-volumes can only be used with --import") + + for i := range imgData { + restoreOptions.CheckpointImage = true + checkpointRuntimeName, found := imgData[i].Annotations[define.CheckpointAnnotationRuntimeName] + if !found { + return fmt.Errorf("image is not a checkpoint: %s", imgData[i].ID) + } + if hostInfo.Host.OCIRuntime.Name != checkpointRuntimeName { + return fmt.Errorf("container image \"%s\" requires runtime: \"%s\"", imgData[i].ID, checkpointRuntimeName) + } + } + + notImport := (!restoreOptions.CheckpointImage && restoreOptions.Import == "") + + if notImport && restoreOptions.ImportPrevious != "" { + return fmt.Errorf("--import-previous can only be used with image or --import") } - if restoreOptions.Import == "" && restoreOptions.Name != "" { - return errors.Errorf("--name can only be used with --import") + if notImport && restoreOptions.IgnoreRootFS { + return fmt.Errorf("--ignore-rootfs can only be used with image or --import") } - if restoreOptions.Import == "" && restoreOptions.Pod != "" { - return errors.Errorf("--pod can only be used with --import") + if notImport && restoreOptions.IgnoreVolumes { + return fmt.Errorf("--ignore-volumes can only be used with image or --import") + } + if notImport && restoreOptions.Name != "" { + return fmt.Errorf("--name can only be used with image or --import") + } + if notImport && restoreOptions.Pod != "" { + return fmt.Errorf("--pod can only be used with image or --import") } if restoreOptions.Name != "" && restoreOptions.TCPEstablished { - return errors.Errorf("--tcp-established cannot be used with --name") + return fmt.Errorf("--tcp-established cannot be used with --name") } inputPorts, err := cmd.Flags().GetStringSlice("publish") @@ -125,17 +152,20 @@ func restore(cmd *cobra.Command, args []string) error { argLen := len(args) if restoreOptions.Import != "" { if restoreOptions.All || restoreOptions.Latest { - return errors.Errorf("Cannot use --import with --all or --latest") + return fmt.Errorf("cannot use --import with --all or --latest") } if argLen > 0 { - return errors.Errorf("Cannot use --import with positional arguments") + return fmt.Errorf("cannot use --import with positional arguments") } } if (restoreOptions.All || restoreOptions.Latest) && argLen > 0 { - return errors.Errorf("--all or --latest and containers cannot be used together") + return fmt.Errorf("--all or --latest and containers cannot be used together") } if argLen < 1 && !restoreOptions.All && !restoreOptions.Latest && restoreOptions.Import == "" { - return errors.Errorf("you must provide at least one name or id") + return fmt.Errorf("you must provide at least one name or id") + } + if argLen > 1 && restoreOptions.Name != "" { + return fmt.Errorf("--name can only be used with one checkpoint image") } responses, err := registry.ContainerEngine().ContainerRestore(context.Background(), args, restoreOptions) if err != nil { diff --git a/cmd/podman/system/service_abi.go b/cmd/podman/system/service_abi.go index d6b42ed29..f8abea3aa 100644 --- a/cmd/podman/system/service_abi.go +++ b/cmd/podman/system/service_abi.go @@ -23,7 +23,7 @@ import ( func restService(flags *pflag.FlagSet, cfg *entities.PodmanConfig, opts entities.ServiceOptions) error { var ( - listener *net.Listener + listener net.Listener err error ) @@ -44,17 +44,15 @@ func restService(flags *pflag.FlagSet, cfg *entities.PodmanConfig, opts entities // If it is activated by systemd, use the first LISTEN_FD (3) // instead of opening the socket file. f := os.NewFile(uintptr(3), "podman.sock") - l, err := net.FileListener(f) + listener, err = net.FileListener(f) if err != nil { return err } - listener = &l } else { - l, err := net.Listen(uri.Scheme, path) + listener, err = net.Listen(uri.Scheme, path) if err != nil { return errors.Wrapf(err, "unable to create socket") } - listener = &l } case "tcp": host := uri.Host @@ -62,11 +60,10 @@ func restService(flags *pflag.FlagSet, cfg *entities.PodmanConfig, opts entities // For backward compatibility, support "tcp:<host>:<port>" and "tcp://<host>:<port>" host = uri.Opaque } - l, err := net.Listen(uri.Scheme, host) + listener, err = net.Listen(uri.Scheme, host) if err != nil { return errors.Wrapf(err, "unable to create socket %v", host) } - listener = &l default: logrus.Debugf("Attempting API Service endpoint scheme %q", uri.Scheme) } @@ -101,7 +98,7 @@ func restService(flags *pflag.FlagSet, cfg *entities.PodmanConfig, opts entities err = server.Serve() if listener != nil { - _ = (*listener).Close() + _ = listener.Close() } return err } diff --git a/contrib/cirrus/pr-should-include-tests b/contrib/cirrus/pr-should-include-tests index 8103df41d..0d39047a6 100755 --- a/contrib/cirrus/pr-should-include-tests +++ b/contrib/cirrus/pr-should-include-tests @@ -30,8 +30,7 @@ fi # Nothing changed under test subdirectory. # # This is OK if the only files being touched are "safe" ones. -filtered_changes=$(git diff --name-status $base $head | - awk '{print $2}' | +filtered_changes=$(git diff --name-only $base $head | fgrep -vx .cirrus.yml | fgrep -vx .gitignore | fgrep -vx Makefile | diff --git a/contrib/cirrus/runner.sh b/contrib/cirrus/runner.sh index f33c6af29..8f956d7f5 100755 --- a/contrib/cirrus/runner.sh +++ b/contrib/cirrus/runner.sh @@ -46,6 +46,8 @@ function _run_validate() { } function _run_unit() { + _bail_if_test_can_be_skipped test/goecho test/version + # shellcheck disable=SC2154 if [[ "$PODBIN_NAME" != "podman" ]]; then # shellcheck disable=SC2154 @@ -55,31 +57,45 @@ function _run_unit() { } function _run_apiv2() { + _bail_if_test_can_be_skipped test/apiv2 + source .venv/requests/bin/activate make localapiv2 |& logformatter } function _run_compose() { + _bail_if_test_can_be_skipped test/compose + ./test/compose/test-compose |& logformatter } function _run_compose_v2() { + _bail_if_test_can_be_skipped test/compose + ./test/compose/test-compose |& logformatter } function _run_int() { + _bail_if_test_can_be_skipped test/e2e + dotest integration } function _run_sys() { + _bail_if_test_can_be_skipped test/system + dotest system } function _run_upgrade_test() { + _bail_if_test_can_be_skipped test/upgrade + bats test/upgrade |& logformatter } function _run_bud() { + _bail_if_test_can_be_skipped test/buildah-bud + ./test/buildah-bud/run-buildah-bud-tests |& logformatter } @@ -217,6 +233,9 @@ function _run_build() { } function _run_altbuild() { + # We can skip all these steps for test-only PRs, but not doc-only ones + _bail_if_test_can_be_skipped docs + local -a arches local arch req_env_vars ALT_NAME @@ -345,6 +364,54 @@ dotest() { |& logformatter } +# Optimization: will exit if the only PR diffs are under docs/ or tests/ +# with the exception of any given arguments. E.g., don't run e2e or upgrade +# or bud tests if the only PR changes are in test/system. +function _bail_if_test_can_be_skipped() { + local head base diffs + + # Cirrus sets these for PRs but not cron. In cron, we never want to skip. + for v in CIRRUS_CHANGE_IN_REPO DEST_BRANCH; do + if [[ -z "${!v}" ]]; then + msg "[ _cannot do selective skip: \$$v is undefined ]" + return 0 + fi + done + # And if this one *is* defined, it means we're not in PR-land; don't skip. + if [[ -n "$CIRRUS_TAG" ]]; then + msg "[ _cannot do selective skip: \$CIRRUS_TAG is defined ]" + return 0 + fi + + head=$CIRRUS_CHANGE_IN_REPO + base=$(git merge-base $DEST_BRANCH $head) + diffs=$(git diff --name-only $base $head) + + # If PR touches any files in an argument directory, we cannot skip + for subdir in "$@"; do + if egrep -q "^$subdir/" <<<"$diffs"; then + return 0 + fi + done + + # PR does not touch any files under our input directories. Now see + # if the PR touches files outside of the following directories, by + # filtering these out from the diff results. + for subdir in docs test; do + # || true needed because we're running with set -e + diffs=$(egrep -v "^$subdir/" <<<"$diffs" || true) + done + + # If we still have diffs, they indicate files outside of docs & test. + # It is not safe to skip. + if [[ -n "$diffs" ]]; then + return 0 + fi + + msg "SKIPPING: This is a doc- and/or test-only PR with no changes under $*" + exit 0 +} + # Nearly every task in .cirrus.yml makes use of this shell script # wrapped by /usr/bin/time to collect runtime statistics. Because the # --output option is used to log stats to a file, every child-process diff --git a/contrib/cirrus/setup_environment.sh b/contrib/cirrus/setup_environment.sh index 906a898b2..a7d0f7fa1 100755 --- a/contrib/cirrus/setup_environment.sh +++ b/contrib/cirrus/setup_environment.sh @@ -41,6 +41,11 @@ cp hack/podman-registry /bin # Make sure cni network plugins directory exists mkdir -p /etc/cni/net.d +# Some test operations & checks require a git "identity" +_gc='git config --file /root/.gitconfig' +$_gc user.email "TMcTestFace@example.com" +$_gc user.name "Testy McTestface" + # Ensure that all lower-level contexts and child-processes have # ready access to higher level orchestration (e.g Cirrus-CI) # variables. diff --git a/docs/source/markdown/podman-build.1.md b/docs/source/markdown/podman-build.1.md index 03ff88aeb..1080581d7 100644 --- a/docs/source/markdown/podman-build.1.md +++ b/docs/source/markdown/podman-build.1.md @@ -495,6 +495,10 @@ If the pull option is set to `always` (with *--pull=always*), pull the image from the first registry it is found in as listed in registries.conf. Raise an error if not found in the registries, even if the image is present locally. +If the pull option is set to `missing` (with *--pull=missing*), +Pull the image only if it is not present in the local storage. Raise an error if it +could neither be found in the local storage or on a registry. + If the pull option is set to `never` (with *--pull=never*), Do not pull the image from the registry, use only the local version. Raise an error if the image is not present locally. diff --git a/docs/source/markdown/podman-container-checkpoint.1.md b/docs/source/markdown/podman-container-checkpoint.1.md index 5c07cd975..a11897081 100644 --- a/docs/source/markdown/podman-container-checkpoint.1.md +++ b/docs/source/markdown/podman-container-checkpoint.1.md @@ -28,6 +28,60 @@ archives. Not compressing the checkpoint archive can result in faster checkpoint archive creation.\ The default is **zstd**. +#### **--create-image**=*image* + +Create a checkpoint image from a running container. This is a standard OCI image +created in the local image store. It consists of a single layer that contains +all of the checkpoint files. The content of this image layer is in the same format as a +checkpoint created with **--export**. A checkpoint image can be pushed to a +standard container registry and pulled on a different system to enable container +migration. In addition, the image can be exported with **podman image save** and +inspected with **podman inspect**. Inspecting a checkpoint image would display +additional information, stored as annotations, about the host environment used +to do the checkpoint: + +- **io.podman.annotations.checkpoint.name**: Human-readable name of the original + container. + +- **io.podman.annotations.checkpoint.rawImageName**: Unprocessed name of the + image used to create the original container (as specified by the user). + +- **io.podman.annotations.checkpoint.rootfsImageID**: ID of the image used to + create the original container. + +- **io.podman.annotations.checkpoint.rootfsImageName**: Image name used to + create the original container. + +- **io.podman.annotations.checkpoint.podman.version**: Version of Podman used to + create the checkpoint. + +- **io.podman.annotations.checkpoint.criu.version**: Version of CRIU used to + create the checkpoint. + +- **io.podman.annotations.checkpoint.runtime.name**: Container runtime (e.g., + runc, crun) used to create the checkpoint. + +- **io.podman.annotations.checkpoint.runtime.version**: Version of the container + runtime used to create the checkpoint. + +- **io.podman.annotations.checkpoint.conmon.version**: Version of conmon used + with the original container. + +- **io.podman.annotations.checkpoint.host.arch**: CPU architecture of the host + on which the checkpoint was created. + +- **io.podman.annotations.checkpoint.host.kernel**: Version of Linux kernel + of the host where the checkpoint was created. + +- **io.podman.annotations.checkpoint.cgroups.version**: cgroup version used by + the host where the checkpoint was created. + +- **io.podman.annotations.checkpoint.distribution.version**: Version of host + distribution on which the checkpoint was created. + +- **io.podman.annotations.checkpoint.distribution.name**: Name of host + distribution on which the checkpoint was created. + #### **--export**, **-e**=*archive* Export the checkpoint to a tar.gz file. The exported checkpoint can be used @@ -145,6 +199,11 @@ Make a checkpoint for the container "mywebserver". # podman container checkpoint mywebserver ``` +Create a checkpoint image for the container "mywebserver". +``` +# podman container checkpoint --create-image mywebserver-checkpoint-1 mywebserver +``` + Dumps the container's memory information of the latest container into an archive. ``` # podman container checkpoint -P -e pre-checkpoint.tar.gz -l diff --git a/docs/source/markdown/podman-container-restore.1.md b/docs/source/markdown/podman-container-restore.1.md index 5b1bf82c5..a70cc30d1 100644 --- a/docs/source/markdown/podman-container-restore.1.md +++ b/docs/source/markdown/podman-container-restore.1.md @@ -4,10 +4,11 @@ podman\-container\-restore - Restores one or more containers from a checkpoint ## SYNOPSIS -**podman container restore** [*options*] *container* [*container* ...] +**podman container restore** [*options*] *name* [...] ## DESCRIPTION -**podman container restore** restores a container from a checkpoint. The *container IDs* or *names* are used as input. +**podman container restore** restores a container from a container checkpoint or +checkpoint image. The *container IDs*, *image IDs* or *names* are used as input. ## OPTIONS #### **--all**, **-a** @@ -106,14 +107,16 @@ If the **--name, -n** option is used, Podman will not attempt to assign the same address to the *container* it was using before checkpointing as each IP address can only be used once and the restored *container* will have another IP address. This also means that **--name, -n** cannot be used in combination with **--tcp-established**.\ -*IMPORTANT: This OPTION is only available in combination with __--import, -i__.* +*IMPORTANT: This OPTION is only available for a checkpoint image or in combination +with __--import, -i__.* #### **--pod**=*name* Restore a container into the pod *name*. The destination pod for this restore has to have the same namespaces shared as the pod this container was checkpointed from (see **[podman pod create --share](podman-pod-create.1.md#--share)**).\ -*IMPORTANT: This OPTION is only available in combination with __--import, -i__.* +*IMPORTANT: This OPTION is only available for a checkpoint image or in combination +with __--import, -i__.* This option requires at least CRIU 3.16. @@ -175,6 +178,15 @@ $ podman run --rm -p 2345:80 -d webserver # podman container restore -p 5432:8080 --import=dump.tar ``` +Start a container with the name "foobar-1". Create a checkpoint image "foobar-checkpoint". Restore the container from the checkpoint image with a different name. +``` +# podman run --name foobar-1 -d webserver +# podman container checkpoint --create-image foobar-checkpoint foobar-1 +# podman inspect foobar-checkpoint +# podman container restore --name foobar-2 foobar-checkpoint +# podman container restore --name foobar-3 foobar-checkpoint +``` + ## SEE ALSO **[podman(1)](podman.1.md)**, **[podman-container-checkpoint(1)](podman-container-checkpoint.1.md)**, **[podman-run(1)](podman-run.1.md)**, **[podman-pod-create(1)](podman-pod-create.1.md)**, **criu(8)** @@ -6,18 +6,18 @@ require ( github.com/BurntSushi/toml v1.1.0 github.com/blang/semver v3.5.1+incompatible github.com/buger/goterm v1.0.4 - github.com/checkpoint-restore/checkpointctl v0.0.0-20211204171957-54b4ebfdb681 + github.com/checkpoint-restore/checkpointctl v0.0.0-20220321135231-33f4a66335f0 github.com/checkpoint-restore/go-criu/v5 v5.3.0 github.com/container-orchestrated-devices/container-device-interface v0.3.2 github.com/containernetworking/cni v1.0.1 github.com/containernetworking/plugins v1.1.1 github.com/containers/buildah v1.25.2-0.20220406205807-5b8e79118057 - github.com/containers/common v0.47.5-0.20220413182852-c23a4e11f91b + github.com/containers/common v0.47.5-0.20220421111103-112a47964ddb github.com/containers/conmon v2.0.20+incompatible - github.com/containers/image/v5 v5.21.1-0.20220405081457-d1b64686e1d0 + github.com/containers/image/v5 v5.21.1-0.20220421124950-8527e238867c github.com/containers/ocicrypt v1.1.3 github.com/containers/psgo v1.7.2 - github.com/containers/storage v1.39.1-0.20220414183333-eea4e0f5f1f9 + github.com/containers/storage v1.39.1-0.20220421071128-4899f8265d63 github.com/coreos/go-systemd/v22 v22.3.2 github.com/coreos/stream-metadata-go v0.0.0-20210225230131-70edb9eb47b3 github.com/cyphar/filepath-securejoin v0.2.3 @@ -29,7 +29,7 @@ require ( github.com/docker/go-plugins-helpers v0.0.0-20211224144127-6eecb7beb651 github.com/docker/go-units v0.4.0 github.com/dtylman/scp v0.0.0-20181017070807-f3000a34aef4 - github.com/fsnotify/fsnotify v1.5.1 + github.com/fsnotify/fsnotify v1.5.2 github.com/ghodss/yaml v1.0.0 github.com/godbus/dbus/v5 v5.1.0 github.com/google/gofuzz v1.2.0 @@ -126,6 +126,7 @@ github.com/OpenPeeDeeP/depguard v1.0.1/go.mod h1:xsIw86fROiiwelg+jB2uM9PiKihMMmU github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo= github.com/ProtonMail/go-crypto v0.0.0-20210920160938-87db9fbc61c7/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo= github.com/ProtonMail/go-crypto v0.0.0-20220113124808-70ae35bab23f/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo= +github.com/ProtonMail/go-crypto v0.0.0-20220407094043-a94812496cf5/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo= github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= @@ -208,8 +209,8 @@ github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cb github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/charithe/durationcheck v0.0.9/go.mod h1:SSbRIBVfMjCi/kEB6K65XEA83D6prSM8ap1UCpNKtgg= github.com/chavacava/garif v0.0.0-20210405164556-e8a0a408d6af/go.mod h1:Qjyv4H3//PWVzTeCezG2b9IRn6myJxJSr4TD/xo6ojU= -github.com/checkpoint-restore/checkpointctl v0.0.0-20211204171957-54b4ebfdb681 h1:Jj8mYL2K6peLJdvT10oGTyYyBPqOynmly37D+iL3xNw= -github.com/checkpoint-restore/checkpointctl v0.0.0-20211204171957-54b4ebfdb681/go.mod h1:67kWC1PXQLR3lM/mmNnu3Kzn7K4TSWZAGUuQP1JSngk= +github.com/checkpoint-restore/checkpointctl v0.0.0-20220321135231-33f4a66335f0 h1:txB5jvhzUCSiiQmqmMWpo5CEB7Gj/Hq5Xqi7eaPl8ko= +github.com/checkpoint-restore/checkpointctl v0.0.0-20220321135231-33f4a66335f0/go.mod h1:67kWC1PXQLR3lM/mmNnu3Kzn7K4TSWZAGUuQP1JSngk= github.com/checkpoint-restore/go-criu/v4 v4.1.0/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw= github.com/checkpoint-restore/go-criu/v5 v5.0.0/go.mod h1:cfwC0EG7HMUenopBsUf9d89JlCLQIfgVcNsNN0t6T2M= github.com/checkpoint-restore/go-criu/v5 v5.2.0/go.mod h1:E/eQpaFtUKGOOSEBZgmKAcn+zUUwWxqcaKZlF54wK8E= @@ -356,14 +357,14 @@ github.com/containernetworking/plugins v1.1.1/go.mod h1:Sr5TH/eBsGLXK/h71HeLfX19 github.com/containers/buildah v1.25.2-0.20220406205807-5b8e79118057 h1:lKSxhMBpcHyyQrj2QJYzcm56uiSeibRdSL2KoppF6rg= github.com/containers/buildah v1.25.2-0.20220406205807-5b8e79118057/go.mod h1:iSoopbYRb6K4b5c3hXgXNkGTI/T085t2+XiGjceud94= github.com/containers/common v0.47.5-0.20220331143923-5f14ec785c18/go.mod h1:Vr2Fn6EdzD6JNAbz8L8bTv3uWLv2p31Ih2O3EAK6Hyc= -github.com/containers/common v0.47.5-0.20220413182852-c23a4e11f91b h1:HVOojcjTGPke7oOh1T/Wj67DK74LBJOR6qU5uW+33zk= -github.com/containers/common v0.47.5-0.20220413182852-c23a4e11f91b/go.mod h1:nRW9288gdZqIGoRwoV23i3qO7Zznbd34sdDOBnq2GjY= +github.com/containers/common v0.47.5-0.20220421111103-112a47964ddb h1:TBrx1KcmWcesByqTb4Cq7F6bg7bDOjqCf6+6rbi8x4k= +github.com/containers/common v0.47.5-0.20220421111103-112a47964ddb/go.mod h1:r80nWTmJrG9EoLkuI6WfbWQDUNQVqkVuB8Oaj1VVjOA= github.com/containers/conmon v2.0.20+incompatible h1:YbCVSFSCqFjjVwHTPINGdMX1F6JXHGTUje2ZYobNrkg= github.com/containers/conmon v2.0.20+incompatible/go.mod h1:hgwZ2mtuDrppv78a/cOBNiCm6O0UMWGx1mu7P00nu5I= github.com/containers/image/v5 v5.19.2-0.20220224100137-1045fb70b094/go.mod h1:XoYK6kE0dpazFNcuS+a8lra+QfbC6s8tzv+cUuCrZpE= github.com/containers/image/v5 v5.20.1-0.20220404163228-d03e80fc66b3/go.mod h1:2nEPM0WuinC/0ssPsMv5Iy8YaRueUUTmTp3C7bn5uro= -github.com/containers/image/v5 v5.21.1-0.20220405081457-d1b64686e1d0 h1:Md1CckW9KSYkdtMdKG70Fc+YqCCVgT+HAr7NS9Ilf8E= -github.com/containers/image/v5 v5.21.1-0.20220405081457-d1b64686e1d0/go.mod h1:JhGkIpC7vKBpLc6mTBE4S8cZUAD+8HgicsxYaLv6BsQ= +github.com/containers/image/v5 v5.21.1-0.20220421124950-8527e238867c h1:hshgYt6RAs4L0KhOEc2/qLF++2MryOfAXvTWmxYu4v4= +github.com/containers/image/v5 v5.21.1-0.20220421124950-8527e238867c/go.mod h1:qpUuaiE2mON6xMA0PRO9GteyH9+KT+C6WygZzL5RhnE= github.com/containers/libtrust v0.0.0-20200511145503-9c3a6c22cd9a h1:spAGlqziZjCJL25C6F1zsQY05tfCKE9F5YwtEWWe6hU= github.com/containers/libtrust v0.0.0-20200511145503-9c3a6c22cd9a/go.mod h1:9rfv8iPl1ZP7aqh9YA68wnZv2NUDbXdcdPHVz0pFbPY= github.com/containers/ocicrypt v1.0.1/go.mod h1:MeJDzk1RJHv89LjsH0Sp5KTY3ZYkjXO/C+bKAeWFIrc= @@ -380,8 +381,8 @@ github.com/containers/storage v1.38.2/go.mod h1:INP0RPLHWBxx+pTsO5uiHlDUGHDFvWZP github.com/containers/storage v1.38.3-0.20220301151551-d06b0f81c0aa/go.mod h1:LkkL34WRi4dI4jt9Cp+ImdZi/P5i36glSHimT5CP5zM= github.com/containers/storage v1.39.0/go.mod h1:UAD0cKLouN4BOQRgZut/nMjrh/EnTCjSNPgp4ZuGWMs= github.com/containers/storage v1.39.1-0.20220330193934-f3200eb5a5d9/go.mod h1:IMa2AfBI+Fxxk2hQqLTGhpJX6z2pZS1/I785QJeUwUY= -github.com/containers/storage v1.39.1-0.20220414183333-eea4e0f5f1f9 h1:cB2AvqxpfyqyyffXtDN0txJhD0lIaZWktbSRI92WpN4= -github.com/containers/storage v1.39.1-0.20220414183333-eea4e0f5f1f9/go.mod h1:hFiHLMgNU0r3MiUpE97hEBaEKCN8fEIuEEBXoFC9eN0= +github.com/containers/storage v1.39.1-0.20220421071128-4899f8265d63 h1:57UXh6fThYqCUJ6iFwHnlFNoWSWlXylkW4H1VRs05mM= +github.com/containers/storage v1.39.1-0.20220421071128-4899f8265d63/go.mod h1:hFiHLMgNU0r3MiUpE97hEBaEKCN8fEIuEEBXoFC9eN0= github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= @@ -511,8 +512,9 @@ github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoD github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= -github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI= github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU= +github.com/fsnotify/fsnotify v1.5.2 h1:M+aHumjFDOySyAjWICQHrDfuizRTP7nzkRHxXfyRP68= +github.com/fsnotify/fsnotify v1.5.2/go.mod h1:VKyWoa5earkjWzuYFJOy3s0DLrlWgSh5nf5hjFuJcAw= github.com/fsouza/go-dockerclient v1.7.7/go.mod h1:njNCXvoZj3sLPjf3yO0DPHf1mdLdCPDYPc14GskKA4Y= github.com/fsouza/go-dockerclient v1.7.10 h1:KIda66AP88BWQpyg+8ve9LQmn1ZZ/usCbmxeBoMth3U= github.com/fsouza/go-dockerclient v1.7.10/go.mod h1:rdD3Eq3rHwMA8p/xrn+gLb+3ov7uRJGVkV1HsUFY39A= @@ -993,7 +995,6 @@ github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:F github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/mitchellh/mapstructure v1.4.2/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= -github.com/mitchellh/mapstructure v1.4.3 h1:OVowDSCllw/YjdLkam3/sm7wEtOy59d8ndGgCcyj8cs= github.com/mitchellh/mapstructure v1.4.3/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A= github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw= @@ -1313,8 +1314,9 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/sylabs/release-tools v0.1.0/go.mod h1:pqP/z/11/rYMQ0OM/Nn7TxGijw7KfZwW9UolD/J1TUo= github.com/sylabs/sif/v2 v2.3.2/go.mod h1:IrLX2pzmQ2O4qgv5iy3HdKJcBNYds9DTMd9Je8A9tX4= -github.com/sylabs/sif/v2 v2.4.2 h1:L4jcqeOF33JfSnH+8GJKC7/ooVpzpZ2K7wotGG4ZzqQ= github.com/sylabs/sif/v2 v2.4.2/go.mod h1:6gQvzNKRIqr4FS08XBfHpkpnxv9b7h58GLkSJ1zdK9A= +github.com/sylabs/sif/v2 v2.6.0 h1:nrWbtSAavp4T6gETg/QgZXxs67qTpSNEgqs2H1y228w= +github.com/sylabs/sif/v2 v2.6.0/go.mod h1:TiyBWsgWeh5yBeQFNuQnvROwswqK7YJT8JA1L53bsXQ= github.com/sylvia7788/contextcheck v1.0.4/go.mod h1:vuPKJMQ7MQ91ZTqfdyreNKwZjyUg6KO+IebVyQDedZQ= github.com/syndtr/gocapability v0.0.0-20170704070218-db04d3cc01c8/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= diff --git a/hack/buildah-vendor-treadmill b/hack/buildah-vendor-treadmill new file mode 100755 index 000000000..1feffaa60 --- /dev/null +++ b/hack/buildah-vendor-treadmill @@ -0,0 +1,582 @@ +#!/usr/bin/perl +# +# buildah-vendor-treadmill - daily vendor of latest-buildah onto latest-podman +# +package Podman::BuildahVendorTreadmill; + +use v5.14; +use utf8; +use open qw( :encoding(UTF-8) :std ); + +use strict; +use warnings; + +use File::Temp qw(tempfile); +use JSON; +use LWP::UserAgent; + +(our $ME = $0) =~ s|.*/||; +our $VERSION = '0.1'; + +# For debugging, show data structures using DumpTree($var) +#use Data::TreeDumper; $Data::TreeDumper::Displayaddress = 0; + +############################################################################### +# BEGIN user-customizable section + +# github path to buildah +our $Buildah = 'github.com/containers/buildah'; + +# FIXME FIXME FIXME: add 'main'? I hope we never need this script for branches. +our $Treadmill_PR_Title = 'DO NOT MERGE: buildah vendor treadmill'; + +our $API_URL = 'https://api.github.com/graphql'; + +# Use colors if available and if stdout is a tty +our $Highlight = ''; +our $Reset = ''; +eval ' + use Term::ANSIColor; + if (-t 1) { + $Highlight = color("green"); + $Reset = color("reset"); + } + $SIG{__WARN__} = sub { print STDERR color("bold red"), "@_", $Reset; }; + +'; + +# END user-customizable section +############################################################################### + +############################################################################### +# BEGIN boilerplate args checking, usage messages + +sub usage { + print <<"END_USAGE"; +Usage: $ME [OPTIONS] [--sync | --pick ] + +$ME is (2022-04-20) **EXPERIMENTAL** + +$ME is intended to solve the problem of vendoring +buildah into podman. + +Call me with one of two options: + + --sync The usual case. Mostly used by Ed. Called from a + development branch, this just updates everything so + we vendor in latest-buildah (main) on top of + latest-podman (main). With a few sanity checks. + + --pick Used for really-truly vendoring in a new buildah; will + cherry-pick a commit on your buildah-vendor working branch + +For latest documentation and best practices, please see: + + https://github.com/containers/podman/wiki/Buildah-Vendor-Treadmill + +OPTIONS: + + --help display this message + --version display program name and version +END_USAGE + + exit; +} + +# Command-line options. Note that this operates directly on @ARGV ! +our %action; +our $debug = 0; +our $force = 0; +our $verbose = 0; +our $NOT = ''; # print "blahing the blah$NOT\n" if $debug +sub handle_opts { + use Getopt::Long; + GetOptions( + 'sync' => sub { $action{sync}++ }, + 'pick' => sub { $action{pick}++ }, + + 'debug!' => \$debug, + 'dry-run|n!' => sub { $NOT = ' [NOT]' }, + 'force' => \$force, + 'verbose|v' => \$verbose, + + help => \&usage, + version => sub { print "$ME version $VERSION\n"; exit 0 }, + ) or die "Try `$ME --help' for help\n"; +} + +# END boilerplate args checking, usage messages +############################################################################### + +############################## CODE BEGINS HERE ############################### + +# The term is "modulino". +__PACKAGE__->main() unless caller(); + +# Main code. +sub main { + # Note that we operate directly on @ARGV, not on function parameters. + # This is deliberate: it's because Getopt::Long only operates on @ARGV + # and there's no clean way to make it use @_. + handle_opts(); # will set package globals + + # Fetch command-line arguments. Barf if too many. + # FIXME: if called with arg, that's the --sync branch? + # FIXME: if called with --pick + arg, that's the PR? + die "$ME: Too many arguments; try $ME --help\n" if @ARGV; + + my @action = keys(%action); + die "$ME: Please invoke me with one of --sync or --pick\n" + if ! @action; + die "$ME: Please invoke me with ONLY one of --sync or --pick\n" + if @action > 1; + + my $handler = __PACKAGE__->can("do_@action") + or die "$ME: No handler available for --@action\n"; + + # We've validated the command-line args. Before running action, check + # that repo is clean. None of our actions can be run on a dirty repo. + assert_clean_repo(); + + $handler->(); +} + +############################################################################### +# BEGIN sync and its helpers + +sub do_sync { + # Preserve current branch name, so we can come back after switching to main + my $current_branch = git_current_branch(); + + my $buildah_old = vendored_buildah(); + print "-> buildah old = $buildah_old\n"; + + # If HEAD is a buildah-vendor commit (usual case), drop it now. + if (head_is_buildah_vendor_commit()) { + if (is_treadmill_commit('HEAD^')) { + progress("HEAD is buildah vendor (as expected); dropping it..."); + git('reset', '--hard', 'HEAD^'); + } + else { + die "$ME: HEAD is a buildah commit, but HEAD^ is not a treadmill commit! Cannot continue.\n"; + } + } + # HEAD must now be a treadmill commit + is_treadmill_commit('HEAD') + or die "$ME: HEAD is not a treadmill commit!\n"; + + # HEAD is now a change to buildah-tests. Now update main and rebase. + pull_main(); + git('checkout', '-q', $current_branch); + my $forkpoint = git('merge-base', '--fork-point', 'main'); + my $main_commit = git('rev-parse', 'main'); + my $rebased; + if ($forkpoint eq $main_commit) { + progress("[Already rebased on podman main]"); + } + else { + # --empty=keep may be needed after a --pick commit, when we've + # vendored a new buildah into podman and incorporated the treadmill + # commit. Since this is a perpetual-motion workflow, in which we + # keep an in-progress PR open at all times, we need a baseline + # commit even if it's empty. + progress("Rebasing on podman main..."); + git('rebase', '--empty=keep', 'main'); + # FIXME: rebase can fail after --pick. If it does, offer instructions. + $rebased = 1; + } + + # We're now back on our treadmill branch, with one commit on top of main. + # Now vendor in latest buildah. + progress("Vendoring in buildah..."); + system('go', 'mod', 'edit', '--require' => "${Buildah}\@main") == 0 + or die "$ME: go mod edit failed\n"; + system('make', 'vendor') == 0 + or die "$ME: make vendor failed\n"; + my $buildah_new = vendored_buildah(); + print "-> buildah new = $buildah_new\n"; + git('commit', '-as', '-m', <<"END_COMMIT_MESSAGE"); +[DO NOT MERGE] vendor in buildah \@ $buildah_new + +This is a JUNK COMMIT from $ME v$VERSION. + +DO NOT MERGE. This is just a way to keep the buildah-podman +vendoring in sync. See script --help for details. +END_COMMIT_MESSAGE + + # if buildah is unchanged, and we did not pull main, exit cleanly + my $change_message = ''; + if ($buildah_new eq $buildah_old) { + if (! $rebased) { + progress("Nothing has changed (same buildah, same podman). Bye!"); + exit 0; + } + $change_message = "Podman has bumped, but Buildah is unchanged. There's probably not much point to testing this."; + } + else { + my $samenew = ($rebased ? 'new' : 'same'); + $change_message = "New buildah, $samenew podman. Good candidate for pushing."; + } + progress($change_message); + + build_and_check_podman(); + + progress("All OK. It's now up to you to 'git push --force'"); + progress(" --- Reminder: $change_message"); +} + +######################### +# is_treadmill_commit # ARG (HEAD or HEAD^) commit message =~ treadmill +######################### +sub is_treadmill_commit { + my $commit_message = git('log', '-1', '--format=%s', @_); + print "[$commit_message]\n" if $verbose; + $commit_message =~ /buildah.*treadmill/; +} + +############### +# pull_main # Switch to main, and pull latest from github +############### +sub pull_main { + progress("Pulling podman main..."); + git('checkout', '-q', 'main'); + git('pull', '-r', git_upstream(), 'main'); +} + +############################ +# build_and_check_podman # Run quick (local) sanity checks before pushing +############################ +sub build_and_check_podman { + my $errs = 0; + + # Confirm that we can still build podman + progress("Running 'make' to confirm that podman builds cleanly..."); + system('make') == 0 + or die "$ME: 'make' failed with new buildah. Cannot continue.\n"; + + # See if any new options need man pages + progress('Cross-checking man pages...'); + $errs += system('hack/xref-helpmsgs-manpages'); + + # Confirm that buildah-bud patches still apply. This requires knowing + # the name of the directory created by the bud-tests script. + progress("Confirming that buildah-bud-tests patches still apply..."); + system('rm -rf test-buildah-*'); + $errs += system('test/buildah-bud/run-buildah-bud-tests', '--no-test'); + # Clean up + system('rm -rf test-buildah-*'); + + return if !$errs; + warn "$ME: Errors found. Please address, then add to HEAD^ commit\n"; + die " ...see $ME --help for more information.\n"; +} + +# END sync and its helpers +############################################################################### +# BEGIN pick and its helpers +# +# This is what gets used on a real vendor-new-buildah PR + +sub do_pick { + my $current_branch = git_current_branch(); + + # Confirm that current branch is a buildah-vendor one + head_is_buildah_vendor_commit(1); + progress("HEAD is a buildah vendor commit. Good."); + + # Identify and pull the treadmill PR + my $treadmill_pr = treadmill_pr(); + my $treadmill_branch = "$ME/pr$treadmill_pr/tmp$$"; + progress("Fetching treadmill PR $treadmill_pr into $treadmill_branch"); + git('fetch', git_upstream(), "pull/$treadmill_pr/head:$treadmill_branch"); + + # read buildah go.mod from it, and from current tree, and compare + my $buildah_on_treadmill = vendored_buildah($treadmill_branch); + my $buildah_here = vendored_buildah(); + if ($buildah_on_treadmill ne $buildah_here) { + warn "$ME: Warning: buildah version mismatch:\n"; + warn "$ME: on treadmill: $buildah_on_treadmill\n"; + warn "$ME: on this branch: $buildah_here\n"; + # FIXME: should this require --force? A yes/no prompt? + # FIXME: I think not, because usual case will be a true tagged version + warn "$ME: Continuing anyway\n"; + } + + cherry_pick($treadmill_pr, $treadmill_branch); + + # Clean up + git('branch', '-D', $treadmill_branch); + + build_and_check_podman(); + + progress("Looks good! Please 'git commit --amend' before pushing."); +} + +################## +# treadmill_pr # Returns ID of open podman PR with the desired subject +################## +sub treadmill_pr { + my $query = <<'END_QUERY'; +{ + search( + query: "buildah vendor treadmill repo:containers/podman", + type: ISSUE, + first: 10 + ) { + edges { node { ... on PullRequest { number state title } } } + } +} +END_QUERY + + my $ua = LWP::UserAgent->new; + $ua->agent("$ME " . $ua->agent); # Identify ourself + + my %headers = ( + 'Accept' => "application/vnd.github.antiope-preview+json", + 'Content-Type' => "application/json", + ); + + # Use github token if available, but don't require it. (All it does is + # bump up our throttling limit, which shouldn't be an issue) (unless + # someone invokes this script hundreds of times per minute). + if (my $token = $ENV{GITHUB_TOKEN}) { + $headers{Authorization} = "bearer $token"; + } + $ua->default_header($_ => $headers{$_}) for keys %headers; + + # Massage the query: escape quotes, put it all in one line, collapse spaces + $query =~ s/\"/\\"/g; + $query =~ s/\n/\\n/g; + $query =~ s/\s+/ /g; + # ...and now one more massage + my $postquery = qq/{ "query": \"$query\" }/; + + print $postquery, "\n" if $debug; + my $res = $ua->post($API_URL, Content => $postquery); + if ((my $code = $res->code) != 200) { + print $code, " ", $res->message, "\n"; + exit 1; + } + + # Got something. Confirm that it has all our required fields + my $content = decode_json($res->content); + use Data::Dump; dd $content if $debug; + exists $content->{data} + or die "$ME: No '{data}' section in response\n"; + exists $content->{data}{search} + or die "$ME: No '{data}{search}' section in response\n"; + exists $content->{data}{search}{edges} + or die "$ME: No '{data}{search}{edges}' section in response\n"; + + # Confirm that there is exactly one such PR + my @prs = @{ $content->{data}{search}{edges} }; + @prs > 0 + or die "$ME: WEIRD! No 'buildah vendor treadmill' PRs found!\n"; + @prs = grep { $_->{node}{title} eq $Treadmill_PR_Title } @prs + or die "$ME: No PRs found with title '$Treadmill_PR_Title'\n"; + @prs = grep { $_->{node}{state} eq 'OPEN' } @prs + or die "$ME: Found '$Treadmill_PR_Title' PRs, but none are OPEN\n"; + @prs == 1 + or die "$ME: Multiple OPEN '$Treadmill_PR_Title' PRs found!\n"; + + # Yay. Found exactly one. + return $prs[0]{node}{number}; +} + +################# +# cherry_pick # cherry-pick a commit, updating its commit message +################# +sub cherry_pick { + my $treadmill_pr = shift; # e.g., 12345 + my $treadmill_branch = shift; # e.g., b-v-p/pr12345/tmpNNN + + progress("Cherry-picking from $treadmill_pr^"); + + # Create a temp script. Do so in /var/tmp because sometimes $TMPDIR + # (e.g. /tmp) has noexec. + my ($fh, $editor) = tempfile( "$ME.edit-commit-message.XXXXXXXX", DIR => "/var/tmp" ); + printf { $fh } <<'END_EDIT_SCRIPT', $ME, $VERSION, $treadmill_pr; +#!/bin/bash + +if [[ -z "$1" ]]; then + echo "FATAL: Did not get called with an arg" >&2 + exit 1 +fi + +msgfile=$1 +if [[ ! -e $msgfile ]]; then + echo "FATAL: git-commit file does not exist: $msgfile" >&2 + exit 1 +fi + +tmpfile=$msgfile.tmp +rm -f $tmpfile + +cat >$tmpfile <<EOF +WIP: Fixes for vendoring Buildah + +This commit was automatically cherry-picked +by %s v%s +from the buildah vendor treadmill PR, #%s + +/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv +> The git commit message from that PR is below. Please review it, +> edit as necessary, then remove this comment block. +\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +EOF + +# Strip the "DO NOT MERGE" header from the treadmill PR, print only +# the "Changes as of YYYY-MM-DD" and subsequent lines +sed -ne '/^Changes as of/,$ p' <$msgfile >>$tmpfile +mv $tmpfile $msgfile + +END_EDIT_SCRIPT + close $fh + or die "$ME: Error writing $editor: $!\n"; + chmod 0755 => $editor; + local $ENV{EDITOR} = $editor; + git('cherry-pick', '--allow-empty', '--edit', "$treadmill_branch^"); + unlink $editor; +} + +# END pick and its helpers +############################################################################### +# BEGIN general-purpose helpers + +############## +# progress # Progris riport Dr Strauss says I shud rite down what I think +############## +sub progress { + print $Highlight, "|\n+---> @_\n", $Reset; +} + +####################### +# assert_clean_repo # Don't even think of running with local changes +####################### +sub assert_clean_repo { + my @changed = git('status', '--porcelain', '--untracked=no') + or return; + + warn "$ME: Modified files in repo:\n"; + warn " $_\n" for @changed; + exit 1; +} + +######################## +# git_current_branch # e.g., 'vendor_buildah' +######################## +sub git_current_branch() { + my $b = git('rev-parse', '--abbrev-ref=strict', 'HEAD'); + + # There is no circumstance in which we can ever be called from main + die "$ME: must run from side branch, not main\n" if $b eq 'main'; + return $b; +} + +################## +# git_upstream # Name of true github upstream +################## +sub git_upstream { + for my $line (git('remote', '-v')) { + my ($remote, $url, $type) = split(' ', $line); + if ($url =~ m!github\.com.*containers/(podman|libpod)!) { + if ($type =~ /fetch/) { + return $remote; + } + } + } + + die "$ME: did not find a remote with 'github.com/containers/podman'\n"; +} + + +######### +# git # Run a git command +######### +sub git { + my @cmd = ('git', @_); + print "\$ @cmd\n" if $verbose || $debug; + open my $fh, '-|', @cmd + or die "$ME: Cannot fork: $!\n"; + my @results; + while (my $line = <$fh>) { + chomp $line; + push @results, $line; + } + close $fh + or die "$ME: command failed: @cmd\n"; + + return wantarray ? @results : join("\n", @results); +} + +################################### +# head_is_buildah_vendor_commit # Returns 1 if HEAD is buildah vendor +################################### +sub head_is_buildah_vendor_commit { + my $fatal = shift; # in: if true, die upon anything missing + + my @deltas = git('diff', '--name-only', 'HEAD^', 'HEAD'); + + # It's OK if there are more modified files than just these. + # It's not OK if any of these are missing. + my @expect = qw(go.mod go.sum vendor/modules.txt); + my @missing; + for my $expect (@expect) { + if (! grep { $_ eq $expect } @deltas) { + push @missing, "$expect is unchanged"; + } + } + + if (! grep { m!^vendor/\Q$Buildah\E/! } @deltas) { + push @missing, "no changes under $Buildah"; + } + + if (@missing) { + if ($fatal || $verbose) { + warn "$ME: HEAD does not look like a buildah vendor commit:\n"; + warn "$ME: - $_\n" for @missing; + if ($fatal) { + die "$ME: Cannot continue\n"; + } + warn "$ME: ...this might be okay, continuing anyway...\n"; + } + return; + } + + return 1; +} + +###################### +# vendored_buildah # Returns currently-vendored buildah +###################### +sub vendored_buildah { + my $gomod_file = 'go.mod'; + my @gomod; + if (@_) { + # Called with a branch argument; fetch that version of go.mod + $gomod_file = "@_:$gomod_file"; + @gomod = git('show', $gomod_file); + } + else { + # No branch argument, read file + open my $fh, '<', $gomod_file + or die "$ME: Cannot read $gomod_file: $!\n"; + while (my $line = <$fh>) { + chomp $line; + push @gomod, $line; + } + close $fh; + } + + for my $line (@gomod) { + if ($line =~ m!^\s+\Q$Buildah\E\s+(\S+)!) { + return $1; + } + } + + die "$ME: Could not find buildah in $gomod_file!\n"; +} + +# END general-purpose helpers +############################################################################### diff --git a/hack/make-and-check-size b/hack/make-and-check-size index f2345b815..5b0021d12 100755 --- a/hack/make-and-check-size +++ b/hack/make-and-check-size @@ -92,9 +92,10 @@ if [[ ! -d $context_dir ]]; then fi # This is the original (and primary) purpose of this check: if 'make' fails, -# there is no point in continuing +# there is no point in continuing. Show at least the commit title since +# the ID may not match anything human recognisable. echo -echo "Building: $(git rev-parse HEAD)" +echo "Building: $(git log -n 1 --no-show-signature --oneline)" make # Determine size of each built file. diff --git a/libpod/container_api.go b/libpod/container_api.go index 0b6139335..ebefed600 100644 --- a/libpod/container_api.go +++ b/libpod/container_api.go @@ -754,6 +754,9 @@ type ContainerCheckpointOptions struct { // TargetFile tells the API to read (or write) the checkpoint image // from (or to) the filename set in TargetFile TargetFile string + // CheckpointImageID tells the API to restore the container from + // checkpoint image with ID set in CheckpointImageID + CheckpointImageID string // Name tells the API that during restore from an exported // checkpoint archive a new name should be used for the // restored container @@ -781,6 +784,9 @@ type ContainerCheckpointOptions struct { // ImportPrevious tells the API to restore container with two // images. One is TargetFile, the other is ImportPrevious. ImportPrevious string + // CreateImage tells Podman to create an OCI image from container + // checkpoint in the local image store. + CreateImage string // Compression tells the API which compression to use for // the exported checkpoint archive. Compression archive.Compression diff --git a/libpod/container_inspect.go b/libpod/container_inspect.go index 14290ca0d..735790411 100644 --- a/libpod/container_inspect.go +++ b/libpod/container_inspect.go @@ -367,7 +367,7 @@ func (c *Container) generateInspectContainerConfig(spec *spec.Spec) *define.Insp // Leave empty if not explicitly overwritten by user if len(c.config.Entrypoint) != 0 { - ctrConfig.Entrypoint = c.config.Entrypoint + ctrConfig.Entrypoint = strings.Join(c.config.Entrypoint, " ") } if len(c.config.Labels) != 0 { diff --git a/libpod/container_internal.go b/libpod/container_internal.go index c7567a55e..b051b7f2d 100644 --- a/libpod/container_internal.go +++ b/libpod/container_internal.go @@ -132,6 +132,11 @@ func (c *Container) ControlSocketPath() string { return filepath.Join(c.bundlePath(), "ctl") } +// CheckpointVolumesPath returns the path to the directory containing the checkpointed volumes +func (c *Container) CheckpointVolumesPath() string { + return filepath.Join(c.bundlePath(), metadata.CheckpointVolumesDirectory) +} + // CheckpointPath returns the path to the directory containing the checkpoint func (c *Container) CheckpointPath() string { return filepath.Join(c.bundlePath(), metadata.CheckpointDirectory) diff --git a/libpod/container_internal_linux.go b/libpod/container_internal_linux.go index 9991003d6..f8d0a124c 100644 --- a/libpod/container_internal_linux.go +++ b/libpod/container_internal_linux.go @@ -24,6 +24,7 @@ import ( "github.com/checkpoint-restore/go-criu/v5/stats" cdi "github.com/container-orchestrated-devices/container-device-interface/pkg/cdi" "github.com/containernetworking/plugins/pkg/ns" + "github.com/containers/buildah" "github.com/containers/buildah/pkg/chrootuser" "github.com/containers/buildah/pkg/overlay" butil "github.com/containers/buildah/util" @@ -34,6 +35,7 @@ import ( "github.com/containers/common/pkg/config" "github.com/containers/common/pkg/subscriptions" "github.com/containers/common/pkg/umask" + is "github.com/containers/image/v5/storage" "github.com/containers/podman/v4/libpod/define" "github.com/containers/podman/v4/libpod/events" "github.com/containers/podman/v4/pkg/annotations" @@ -1098,6 +1100,124 @@ func (c *Container) addNamespaceContainer(g *generate.Generator, ns LinuxNS, ctr return nil } +func (c *Container) addCheckpointImageMetadata(importBuilder *buildah.Builder) error { + // Get information about host environment + hostInfo, err := c.Runtime().hostInfo() + if err != nil { + return fmt.Errorf("getting host info: %v", err) + } + + criuVersion, err := criu.GetCriuVestion() + if err != nil { + return fmt.Errorf("getting criu version: %v", err) + } + + rootfsImageID, rootfsImageName := c.Image() + + // Add image annotations with information about the container and the host. + // This information is useful to check compatibility before restoring the checkpoint + + checkpointImageAnnotations := map[string]string{ + define.CheckpointAnnotationName: c.config.Name, + define.CheckpointAnnotationRawImageName: c.config.RawImageName, + define.CheckpointAnnotationRootfsImageID: rootfsImageID, + define.CheckpointAnnotationRootfsImageName: rootfsImageName, + define.CheckpointAnnotationPodmanVersion: version.Version.String(), + define.CheckpointAnnotationCriuVersion: strconv.Itoa(criuVersion), + define.CheckpointAnnotationRuntimeName: hostInfo.OCIRuntime.Name, + define.CheckpointAnnotationRuntimeVersion: hostInfo.OCIRuntime.Version, + define.CheckpointAnnotationConmonVersion: hostInfo.Conmon.Version, + define.CheckpointAnnotationHostArch: hostInfo.Arch, + define.CheckpointAnnotationHostKernel: hostInfo.Kernel, + define.CheckpointAnnotationCgroupVersion: hostInfo.CgroupsVersion, + define.CheckpointAnnotationDistributionVersion: hostInfo.Distribution.Version, + define.CheckpointAnnotationDistributionName: hostInfo.Distribution.Distribution, + } + + for key, value := range checkpointImageAnnotations { + importBuilder.SetAnnotation(key, value) + } + + return nil +} + +func (c *Container) resolveCheckpointImageName(options *ContainerCheckpointOptions) error { + if options.CreateImage == "" { + return nil + } + + // Resolve image name + resolvedImageName, err := c.runtime.LibimageRuntime().ResolveName(options.CreateImage) + if err != nil { + return err + } + + options.CreateImage = resolvedImageName + return nil +} + +func (c *Container) createCheckpointImage(ctx context.Context, options ContainerCheckpointOptions) error { + if options.CreateImage == "" { + return nil + } + logrus.Debugf("Create checkpoint image %s", options.CreateImage) + + // Create storage reference + imageRef, err := is.Transport.ParseStoreReference(c.runtime.store, options.CreateImage) + if err != nil { + return errors.Errorf("Failed to parse image name") + } + + // Build an image scratch + builderOptions := buildah.BuilderOptions{ + FromImage: "scratch", + } + importBuilder, err := buildah.NewBuilder(ctx, c.runtime.store, builderOptions) + if err != nil { + return err + } + // Clean-up buildah working container + defer importBuilder.Delete() + + if err := c.prepareCheckpointExport(); err != nil { + return err + } + + // Export checkpoint into temporary tar file + tmpDir, err := ioutil.TempDir("", "checkpoint_image_") + if err != nil { + return err + } + defer os.RemoveAll(tmpDir) + + options.TargetFile = path.Join(tmpDir, "checkpoint.tar") + + if err := c.exportCheckpoint(options); err != nil { + return err + } + + // Copy checkpoint from temporary tar file in the image + addAndCopyOptions := buildah.AddAndCopyOptions{} + importBuilder.Add("", true, addAndCopyOptions, options.TargetFile) + + if err := c.addCheckpointImageMetadata(importBuilder); err != nil { + return err + } + + commitOptions := buildah.CommitOptions{ + Squash: true, + SystemContext: c.runtime.imageContext, + } + + // Create checkpoint image + id, _, _, err := importBuilder.Commit(ctx, imageRef, commitOptions) + if err != nil { + return err + } + logrus.Debugf("Created checkpoint image: %s", id) + return nil +} + func (c *Container) exportCheckpoint(options ContainerCheckpointOptions) error { if len(c.Dependencies()) == 1 { // Check if the dependency is an infra container. If it is we can checkpoint @@ -1159,7 +1279,7 @@ func (c *Container) exportCheckpoint(options ContainerCheckpointOptions) error { } // Folder containing archived volumes that will be included in the export - expVolDir := filepath.Join(c.bundlePath(), "volumes") + expVolDir := filepath.Join(c.bundlePath(), metadata.CheckpointVolumesDirectory) // Create an archive for each volume associated with the container if !options.IgnoreVolumes { @@ -1168,7 +1288,7 @@ func (c *Container) exportCheckpoint(options ContainerCheckpointOptions) error { } for _, v := range c.config.NamedVolumes { - volumeTarFilePath := filepath.Join("volumes", v.Name+".tar") + volumeTarFilePath := filepath.Join(metadata.CheckpointVolumesDirectory, v.Name+".tar") volumeTarFileFullPath := filepath.Join(c.bundlePath(), volumeTarFilePath) volumeTarFile, err := os.Create(volumeTarFileFullPath) @@ -1266,6 +1386,10 @@ func (c *Container) checkpoint(ctx context.Context, options ContainerCheckpointO return nil, 0, errors.Errorf("cannot checkpoint containers that have been started with '--rm' unless '--export' is used") } + if err := c.resolveCheckpointImageName(&options); err != nil { + return nil, 0, err + } + if err := crutils.CRCreateFileWithLabel(c.bundlePath(), "dump.log", c.MountLabel()); err != nil { return nil, 0, err } @@ -1325,6 +1449,10 @@ func (c *Container) checkpoint(ctx context.Context, options ContainerCheckpointO if err := c.exportCheckpoint(options); err != nil { return nil, 0, err } + } else { + if err := c.createCheckpointImage(ctx, options); err != nil { + return nil, 0, err + } } logrus.Debugf("Checkpointed container %s", c.ID()) @@ -1390,11 +1518,7 @@ func (c *Container) checkpoint(ctx context.Context, options ContainerCheckpointO return criuStatistics, runtimeCheckpointDuration, c.save() } -func (c *Container) importCheckpoint(input string) error { - if err := crutils.CRImportCheckpointWithoutConfig(c.bundlePath(), input); err != nil { - return err - } - +func (c *Container) generateContainerSpec() error { // Make sure the newly created config.json exists on disk g := generate.NewFromSpec(c.config.Spec) @@ -1405,6 +1529,51 @@ func (c *Container) importCheckpoint(input string) error { return nil } +func (c *Container) importCheckpointImage(ctx context.Context, imageID string) error { + img, _, err := c.Runtime().LibimageRuntime().LookupImage(imageID, nil) + if err != nil { + return err + } + + mountPoint, err := img.Mount(ctx, nil, "") + defer img.Unmount(true) + if err != nil { + return err + } + + // Import all checkpoint files except ConfigDumpFile and SpecDumpFile. We + // generate new container config files to enable to specifying a new + // container name. + checkpoint := []string{ + "artifacts", + metadata.CheckpointDirectory, + metadata.CheckpointVolumesDirectory, + metadata.DevShmCheckpointTar, + metadata.RootFsDiffTar, + metadata.DeletedFilesFile, + metadata.PodOptionsFile, + metadata.PodDumpFile, + } + + for _, name := range checkpoint { + src := filepath.Join(mountPoint, name) + dst := filepath.Join(c.bundlePath(), name) + if err := archive.NewDefaultArchiver().CopyWithTar(src, dst); err != nil { + logrus.Debugf("Can't import '%s' from checkpoint image", name) + } + } + + return c.generateContainerSpec() +} + +func (c *Container) importCheckpointTar(input string) error { + if err := crutils.CRImportCheckpointWithoutConfig(c.bundlePath(), input); err != nil { + return err + } + + return c.generateContainerSpec() +} + func (c *Container) importPreCheckpoint(input string) error { archiveFile, err := os.Open(input) if err != nil { @@ -1446,7 +1615,11 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti } if options.TargetFile != "" { - if err := c.importCheckpoint(options.TargetFile); err != nil { + if err := c.importCheckpointTar(options.TargetFile); err != nil { + return nil, 0, err + } + } else if options.CheckpointImageID != "" { + if err := c.importCheckpointImage(ctx, options.CheckpointImageID); err != nil { return nil, 0, err } } @@ -1531,7 +1704,7 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti } // Restoring from an import means that we are doing migration - if options.TargetFile != "" { + if options.TargetFile != "" || options.CheckpointImageID != "" { g.SetRootPath(c.state.Mountpoint) } @@ -1628,7 +1801,7 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti return nil, 0, err } - if options.TargetFile != "" { + if options.TargetFile != "" || options.CheckpointImageID != "" { for dstPath, srcPath := range c.state.BindMounts { newMount := spec.Mount{ Type: "bind", @@ -1678,9 +1851,9 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti // When restoring from an imported archive, allow restoring the content of volumes. // Volumes are created in setupContainer() - if options.TargetFile != "" && !options.IgnoreVolumes { + if !options.IgnoreVolumes && (options.TargetFile != "" || options.CheckpointImageID != "") { for _, v := range c.config.NamedVolumes { - volumeFilePath := filepath.Join(c.bundlePath(), "volumes", v.Name+".tar") + volumeFilePath := filepath.Join(c.bundlePath(), metadata.CheckpointVolumesDirectory, v.Name+".tar") volumeFile, err := os.Open(volumeFilePath) if err != nil { @@ -1770,11 +1943,16 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti if err != nil { logrus.Debugf("Non-fatal: removal of pre-checkpoint directory (%s) failed: %v", c.PreCheckPointPath(), err) } + err = os.RemoveAll(c.CheckpointVolumesPath()) + if err != nil { + logrus.Debugf("Non-fatal: removal of checkpoint volumes directory (%s) failed: %v", c.CheckpointVolumesPath(), err) + } cleanup := [...]string{ "restore.log", "dump.log", stats.StatsDump, stats.StatsRestore, + metadata.DevShmCheckpointTar, metadata.NetworkStatusFile, metadata.RootFsDiffTar, metadata.DeletedFilesFile, diff --git a/libpod/define/annotations.go b/libpod/define/annotations.go index 3964a1237..a83fbff0b 100644 --- a/libpod/define/annotations.go +++ b/libpod/define/annotations.go @@ -65,6 +65,76 @@ const ( // InspectResponseFalse is a boolean False response for an inspect // annotation. InspectResponseFalse = "FALSE" + + // CheckpointAnnotationName is used by Container Checkpoint when creating a + // checkpoint image to specify the original human-readable name for the + // container. + CheckpointAnnotationName = "io.podman.annotations.checkpoint.name" + + // CheckpointAnnotationRawImageName is used by Container Checkpoint when + // creating a checkpoint image to specify the original unprocessed name of + // the image used to create the container (as specified by the user). + CheckpointAnnotationRawImageName = "io.podman.annotations.checkpoint.rawImageName" + + // CheckpointAnnotationRootfsImageID is used by Container Checkpoint when + // creating a checkpoint image to specify the original ID of the image used + // to create the container. + CheckpointAnnotationRootfsImageID = "io.podman.annotations.checkpoint.rootfsImageID" + + // CheckpointAnnotationRootfsImageName is used by Container Checkpoint when + // creating a checkpoint image to specify the original image name used to + // create the container. + CheckpointAnnotationRootfsImageName = "io.podman.annotations.checkpoint.rootfsImageName" + + // CheckpointAnnotationPodmanVersion is used by Container Checkpoint when + // creating a checkpoint image to specify the version of Podman used on the + // host where the checkpoint was created. + CheckpointAnnotationPodmanVersion = "io.podman.annotations.checkpoint.podman.version" + + // CheckpointAnnotationCriuVersion is used by Container Checkpoint when + // creating a checkpoint image to specify the version of CRIU used on the + // host where the checkpoint was created. + CheckpointAnnotationCriuVersion = "io.podman.annotations.checkpoint.criu.version" + + // CheckpointAnnotationRuntimeName is used by Container Checkpoint when + // creating a checkpoint image to specify the runtime used on the host where + // the checkpoint was created. + CheckpointAnnotationRuntimeName = "io.podman.annotations.checkpoint.runtime.name" + + // CheckpointAnnotationRuntimeVersion is used by Container Checkpoint when + // creating a checkpoint image to specify the version of runtime used on the + // host where the checkpoint was created. + CheckpointAnnotationRuntimeVersion = "io.podman.annotations.checkpoint.runtime.version" + + // CheckpointAnnotationConmonVersion is used by Container Checkpoint when + // creating a checkpoint image to specify the version of conmon used on + // the host where the checkpoint was created. + CheckpointAnnotationConmonVersion = "io.podman.annotations.checkpoint.conmon.version" + + // CheckpointAnnotationHostArch is used by Container Checkpoint when + // creating a checkpoint image to specify the CPU architecture of the host + // on which the checkpoint was created. + CheckpointAnnotationHostArch = "io.podman.annotations.checkpoint.host.arch" + + // CheckpointAnnotationHostKernel is used by Container Checkpoint when + // creating a checkpoint image to specify the kernel version used by the + // host where the checkpoint was created. + CheckpointAnnotationHostKernel = "io.podman.annotations.checkpoint.host.kernel" + + // CheckpointAnnotationCgroupVersion is used by Container Checkpoint when + // creating a checkpoint image to specify the cgroup version used by the + // host where the checkpoint was created. + CheckpointAnnotationCgroupVersion = "io.podman.annotations.checkpoint.cgroups.version" + + // CheckpointAnnotationDistributionVersion is used by Container Checkpoint + // when creating a checkpoint image to specify the version of host + // distribution on which the checkpoint was created. + CheckpointAnnotationDistributionVersion = "io.podman.annotations.checkpoint.distribution.version" + + // CheckpointAnnotationDistributionName is used by Container Checkpoint when + // creating a checkpoint image to specify the name of host distribution on + // which the checkpoint was created. + CheckpointAnnotationDistributionName = "io.podman.annotations.checkpoint.distribution.name" ) // IsReservedAnnotation returns true if the specified value corresponds to an diff --git a/libpod/define/container_inspect.go b/libpod/define/container_inspect.go index 444fbff62..ae2ce9724 100644 --- a/libpod/define/container_inspect.go +++ b/libpod/define/container_inspect.go @@ -44,7 +44,7 @@ type InspectContainerConfig struct { // Container working directory WorkingDir string `json:"WorkingDir"` // Container entrypoint - Entrypoint []string `json:"Entrypoint"` + Entrypoint string `json:"Entrypoint"` // On-build arguments - presently unused. More of Buildah's domain. OnBuild *string `json:"OnBuild"` // Container labels diff --git a/libpod/events.go b/libpod/events.go index 3908536a1..39f5786a4 100644 --- a/libpod/events.go +++ b/libpod/events.go @@ -15,7 +15,7 @@ func (r *Runtime) newEventer() (events.Eventer, error) { options := events.EventerOptions{ EventerType: r.config.Engine.EventsLogger, LogFilePath: r.config.Engine.EventsLogFilePath, - LogFileMaxSize: r.config.Engine.EventsLogFileMaxSize, + LogFileMaxSize: r.config.Engine.EventsLogMaxSize(), } return events.NewEventer(options) } diff --git a/libpod/events/logfile.go b/libpod/events/logfile.go index 5091f3723..21fdd8027 100644 --- a/libpod/events/logfile.go +++ b/libpod/events/logfile.go @@ -158,6 +158,10 @@ func rotateLog(logfile string, content string, limit uint64) (bool, error) { } file, err := os.Stat(logfile) if err != nil { + if errors.Is(err, os.ErrNotExist) { + // The logfile does not exist yet. + return false, nil + } return false, err } var filesize = uint64(file.Size()) diff --git a/pkg/api/handlers/libpod/containers.go b/pkg/api/handlers/libpod/containers.go index dfa09b8b8..03dd436f6 100644 --- a/pkg/api/handlers/libpod/containers.go +++ b/pkg/api/handlers/libpod/containers.go @@ -209,15 +209,16 @@ func Checkpoint(w http.ResponseWriter, r *http.Request) { decoder := r.Context().Value(api.DecoderKey).(*schema.Decoder) query := struct { - Keep bool `schema:"keep"` - LeaveRunning bool `schema:"leaveRunning"` - TCPEstablished bool `schema:"tcpEstablished"` - Export bool `schema:"export"` - IgnoreRootFS bool `schema:"ignoreRootFS"` - PrintStats bool `schema:"printStats"` - PreCheckpoint bool `schema:"preCheckpoint"` - WithPrevious bool `schema:"withPrevious"` - FileLocks bool `schema:"fileLocks"` + Keep bool `schema:"keep"` + LeaveRunning bool `schema:"leaveRunning"` + TCPEstablished bool `schema:"tcpEstablished"` + Export bool `schema:"export"` + IgnoreRootFS bool `schema:"ignoreRootFS"` + PrintStats bool `schema:"printStats"` + PreCheckpoint bool `schema:"preCheckpoint"` + WithPrevious bool `schema:"withPrevious"` + FileLocks bool `schema:"fileLocks"` + CreateImage string `schema:"createImage"` }{ // override any golang type defaults } @@ -243,6 +244,7 @@ func Checkpoint(w http.ResponseWriter, r *http.Request) { PreCheckPoint: query.PreCheckpoint, WithPrevious: query.WithPrevious, FileLocks: query.FileLocks, + CreateImage: query.CreateImage, } if query.Export { @@ -341,8 +343,17 @@ func Restore(w http.ResponseWriter, r *http.Request) { } else { name := utils.GetName(r) if _, err := runtime.LookupContainer(name); err != nil { - utils.ContainerNotFound(w, name, err) - return + // If container was not found, check if this is a checkpoint image + ir := abi.ImageEngine{Libpod: runtime} + report, err := ir.Exists(r.Context(), name) + if err != nil { + utils.Error(w, http.StatusNotFound, errors.Wrapf(err, "failed to find container or checkpoint image %s", name)) + return + } + if !report.Value { + utils.Error(w, http.StatusNotFound, errors.Errorf("failed to find container or checkpoint image %s", name)) + return + } } names = []string{name} } diff --git a/pkg/api/server/server.go b/pkg/api/server/server.go index 73740a6f9..a906a01f1 100644 --- a/pkg/api/server/server.go +++ b/pkg/api/server/server.go @@ -60,11 +60,11 @@ func NewServer(runtime *libpod.Runtime) (*APIServer, error) { } // NewServerWithSettings will create and configure a new API server using provided settings -func NewServerWithSettings(runtime *libpod.Runtime, listener *net.Listener, opts entities.ServiceOptions) (*APIServer, error) { +func NewServerWithSettings(runtime *libpod.Runtime, listener net.Listener, opts entities.ServiceOptions) (*APIServer, error) { return newServer(runtime, listener, opts) } -func newServer(runtime *libpod.Runtime, listener *net.Listener, opts entities.ServiceOptions) (*APIServer, error) { +func newServer(runtime *libpod.Runtime, listener net.Listener, opts entities.ServiceOptions) (*APIServer, error) { // If listener not provided try socket activation protocol if listener == nil { if _, found := os.LookupEnv("LISTEN_PID"); !found { @@ -78,7 +78,11 @@ func newServer(runtime *libpod.Runtime, listener *net.Listener, opts entities.Se if len(listeners) != 1 { return nil, fmt.Errorf("wrong number of file descriptors for socket activation protocol (%d != 1)", len(listeners)) } - listener = &listeners[0] + listener = listeners[0] + // note that activation.Listeners() return nil when it cannot listen on the fd (i.e. udp connection) + if listener == nil { + return nil, fmt.Errorf("unexpected fd received from systemd: cannot listen on it") + } } if opts.CorsHeaders == "" { logrus.Debug("CORS Headers were not set") @@ -86,7 +90,7 @@ func newServer(runtime *libpod.Runtime, listener *net.Listener, opts entities.Se logrus.Debugf("CORS Headers were set to %q", opts.CorsHeaders) } - logrus.Infof("API service listening on %q", (*listener).Addr()) + logrus.Infof("API service listening on %q", listener.Addr()) router := mux.NewRouter().UseEncodedPath() tracker := idle.NewTracker(opts.Timeout) @@ -101,7 +105,7 @@ func newServer(runtime *libpod.Runtime, listener *net.Listener, opts entities.Se IdleTimeout: opts.Timeout * 2, }, CorsHeaders: opts.CorsHeaders, - Listener: *listener, + Listener: listener, PProfAddr: opts.PProfAddr, idleTracker: tracker, } diff --git a/pkg/bindings/containers/types.go b/pkg/bindings/containers/types.go index c87f82bf4..81d491bb7 100644 --- a/pkg/bindings/containers/types.go +++ b/pkg/bindings/containers/types.go @@ -47,6 +47,7 @@ type AttachOptions struct { // CheckpointOptions are optional options for checkpointing containers type CheckpointOptions struct { Export *string + CreateImage *string IgnoreRootfs *bool Keep *bool LeaveRunning *bool diff --git a/pkg/bindings/containers/types_checkpoint_options.go b/pkg/bindings/containers/types_checkpoint_options.go index e717daf9f..d5f6e541d 100644 --- a/pkg/bindings/containers/types_checkpoint_options.go +++ b/pkg/bindings/containers/types_checkpoint_options.go @@ -32,6 +32,21 @@ func (o *CheckpointOptions) GetExport() string { return *o.Export } +// WithCreateImage set field CreateImage to given value +func (o *CheckpointOptions) WithCreateImage(value string) *CheckpointOptions { + o.CreateImage = &value + return o +} + +// GetCreateImage returns value of field CreateImage +func (o *CheckpointOptions) GetCreateImage() string { + if o.CreateImage == nil { + var z string + return z + } + return *o.CreateImage +} + // WithIgnoreRootfs set field IgnoreRootfs to given value func (o *CheckpointOptions) WithIgnoreRootfs(value bool) *CheckpointOptions { o.IgnoreRootfs = &value diff --git a/pkg/checkpoint/checkpoint_restore.go b/pkg/checkpoint/checkpoint_restore.go index 270b5b6c4..396b521a1 100644 --- a/pkg/checkpoint/checkpoint_restore.go +++ b/pkg/checkpoint/checkpoint_restore.go @@ -22,9 +22,7 @@ import ( // Prefixing the checkpoint/restore related functions with 'cr' -// CRImportCheckpoint it the function which imports the information -// from checkpoint tarball and re-creates the container from that information -func CRImportCheckpoint(ctx context.Context, runtime *libpod.Runtime, restoreOptions entities.RestoreOptions) ([]*libpod.Container, error) { +func CRImportCheckpointTar(ctx context.Context, runtime *libpod.Runtime, restoreOptions entities.RestoreOptions) ([]*libpod.Container, error) { // First get the container definition from the // tarball to a temporary directory dir, err := ioutil.TempDir("", "checkpoint") @@ -39,7 +37,12 @@ func CRImportCheckpoint(ctx context.Context, runtime *libpod.Runtime, restoreOpt if err := crutils.CRImportCheckpointConfigOnly(dir, restoreOptions.Import); err != nil { return nil, err } + return CRImportCheckpoint(ctx, runtime, restoreOptions, dir) +} +// CRImportCheckpoint it the function which imports the information +// from checkpoint tarball and re-creates the container from that information +func CRImportCheckpoint(ctx context.Context, runtime *libpod.Runtime, restoreOptions entities.RestoreOptions, dir string) ([]*libpod.Container, error) { // Load spec.dump from temporary directory dumpSpec := new(spec.Spec) if _, err := metadata.ReadJSONFile(dumpSpec, dir, metadata.SpecDumpFile); err != nil { @@ -48,7 +51,7 @@ func CRImportCheckpoint(ctx context.Context, runtime *libpod.Runtime, restoreOpt // Load config.dump from temporary directory ctrConfig := new(libpod.ContainerConfig) - if _, err = metadata.ReadJSONFile(ctrConfig, dir, metadata.ConfigDumpFile); err != nil { + if _, err := metadata.ReadJSONFile(ctrConfig, dir, metadata.ConfigDumpFile); err != nil { return nil, err } diff --git a/pkg/checkpoint/crutils/checkpoint_restore_utils.go b/pkg/checkpoint/crutils/checkpoint_restore_utils.go index 6a8a7894a..76c868cee 100644 --- a/pkg/checkpoint/crutils/checkpoint_restore_utils.go +++ b/pkg/checkpoint/crutils/checkpoint_restore_utils.go @@ -54,7 +54,6 @@ func CRImportCheckpointConfigOnly(destination, input string) error { options := &archive.TarOptions{ // Here we only need the files config.dump and spec.dump ExcludePatterns: []string{ - "volumes", "ctr.log", "artifacts", stats.StatsDump, @@ -62,6 +61,7 @@ func CRImportCheckpointConfigOnly(destination, input string) error { metadata.DeletedFilesFile, metadata.NetworkStatusFile, metadata.CheckpointDirectory, + metadata.CheckpointVolumesDirectory, }, } if err = archive.Untar(archiveFile, destination, options); err != nil { diff --git a/pkg/criu/criu.go b/pkg/criu/criu.go index b54870abc..6570159d7 100644 --- a/pkg/criu/criu.go +++ b/pkg/criu/criu.go @@ -28,6 +28,11 @@ func CheckForCriu(version int) bool { return result } +func GetCriuVestion() (int, error) { + c := criu.MakeCriu() + return c.GetCriuVersion() +} + func MemTrack() bool { features, err := criu.MakeCriu().FeatureCheck( &rpc.CriuFeatures{ diff --git a/pkg/domain/entities/containers.go b/pkg/domain/entities/containers.go index 99b000c2c..ae60e5b96 100644 --- a/pkg/domain/entities/containers.go +++ b/pkg/domain/entities/containers.go @@ -178,6 +178,7 @@ type ContainerExportOptions struct { type CheckpointOptions struct { All bool Export string + CreateImage string IgnoreRootFS bool IgnoreVolumes bool Keep bool @@ -205,6 +206,7 @@ type RestoreOptions struct { IgnoreStaticIP bool IgnoreStaticMAC bool Import string + CheckpointImage bool Keep bool Latest bool Name string diff --git a/pkg/domain/infra/abi/containers.go b/pkg/domain/infra/abi/containers.go index db58569c5..b56c36015 100644 --- a/pkg/domain/infra/abi/containers.go +++ b/pkg/domain/infra/abi/containers.go @@ -563,6 +563,7 @@ func (ic *ContainerEngine) ContainerCheckpoint(ctx context.Context, namesOrIds [ Compression: options.Compression, PrintStats: options.PrintStats, FileLocks: options.FileLocks, + CreateImage: options.CreateImage, } if options.All { @@ -592,8 +593,9 @@ func (ic *ContainerEngine) ContainerCheckpoint(ctx context.Context, namesOrIds [ func (ic *ContainerEngine) ContainerRestore(ctx context.Context, namesOrIds []string, options entities.RestoreOptions) ([]*entities.RestoreReport, error) { var ( - cons []*libpod.Container - err error + containers []*libpod.Container + checkpointImageImportErrors []error + err error ) restoreOptions := libpod.ContainerCheckpointOptions{ @@ -619,17 +621,49 @@ func (ic *ContainerEngine) ContainerRestore(ctx context.Context, namesOrIds []st switch { case options.Import != "": - cons, err = checkpoint.CRImportCheckpoint(ctx, ic.Libpod, options) + containers, err = checkpoint.CRImportCheckpointTar(ctx, ic.Libpod, options) case options.All: - cons, err = ic.Libpod.GetContainers(filterFuncs...) + containers, err = ic.Libpod.GetContainers(filterFuncs...) + case options.Latest: + containers, err = getContainersByContext(false, options.Latest, namesOrIds, ic.Libpod) default: - cons, err = getContainersByContext(false, options.Latest, namesOrIds, ic.Libpod) + for _, nameOrID := range namesOrIds { + logrus.Debugf("lookup container: %q", nameOrID) + ctr, err := ic.Libpod.LookupContainer(nameOrID) + if err == nil { + containers = append(containers, ctr) + } else { + // If container was not found, check if this is a checkpoint image + logrus.Debugf("lookup image: %q", nameOrID) + img, _, err := ic.Libpod.LibimageRuntime().LookupImage(nameOrID, nil) + if err != nil { + return nil, fmt.Errorf("no such container or image: %s", nameOrID) + } + restoreOptions.CheckpointImageID = img.ID() + mountPoint, err := img.Mount(ctx, nil, "") + defer img.Unmount(true) + if err != nil { + return nil, err + } + importedContainers, err := checkpoint.CRImportCheckpoint(ctx, ic.Libpod, options, mountPoint) + if err != nil { + // CRImportCheckpoint is expected to import exactly one container from checkpoint image + checkpointImageImportErrors = append( + checkpointImageImportErrors, + errors.Errorf("unable to import checkpoint from image: %q: %v", nameOrID, err), + ) + } else { + containers = append(containers, importedContainers[0]) + } + } + } } if err != nil { return nil, err } - reports := make([]*entities.RestoreReport, 0, len(cons)) - for _, con := range cons { + + reports := make([]*entities.RestoreReport, 0, len(containers)) + for _, con := range containers { criuStatistics, runtimeRestoreDuration, err := con.Restore(ctx, restoreOptions) reports = append(reports, &entities.RestoreReport{ Err: err, @@ -638,6 +672,13 @@ func (ic *ContainerEngine) ContainerRestore(ctx context.Context, namesOrIds []st CRIUStatistics: criuStatistics, }) } + + for _, importErr := range checkpointImageImportErrors { + reports = append(reports, &entities.RestoreReport{ + Err: importErr, + }) + } + return reports, nil } diff --git a/pkg/domain/infra/tunnel/containers.go b/pkg/domain/infra/tunnel/containers.go index 10bfb3984..82e8fbb5b 100644 --- a/pkg/domain/infra/tunnel/containers.go +++ b/pkg/domain/infra/tunnel/containers.go @@ -16,6 +16,7 @@ import ( "github.com/containers/podman/v4/libpod/events" "github.com/containers/podman/v4/pkg/api/handlers" "github.com/containers/podman/v4/pkg/bindings/containers" + "github.com/containers/podman/v4/pkg/bindings/images" "github.com/containers/podman/v4/pkg/domain/entities" "github.com/containers/podman/v4/pkg/domain/entities/reports" "github.com/containers/podman/v4/pkg/errorhandling" @@ -331,6 +332,7 @@ func (ic *ContainerEngine) ContainerCheckpoint(ctx context.Context, namesOrIds [ options.WithIgnoreRootfs(opts.IgnoreRootFS) options.WithKeep(opts.Keep) options.WithExport(opts.Export) + options.WithCreateImage(opts.CreateImage) options.WithTCPEstablished(opts.TCPEstablished) options.WithPrintStats(opts.PrintStats) options.WithPreCheckpoint(opts.PreCheckPoint) @@ -396,8 +398,7 @@ func (ic *ContainerEngine) ContainerRestore(ctx context.Context, namesOrIds []st } var ( - err error - ctrs = []entities.ListContainer{} + ids = []string{} ) if opts.All { allCtrs, err := getContainersByContext(ic.ClientCtx, true, false, []string{}) @@ -407,20 +408,42 @@ func (ic *ContainerEngine) ContainerRestore(ctx context.Context, namesOrIds []st // narrow the list to exited only for _, c := range allCtrs { if c.State == define.ContainerStateExited.String() { - ctrs = append(ctrs, c) + ids = append(ids, c.ID) } } } else { - ctrs, err = getContainersByContext(ic.ClientCtx, false, false, namesOrIds) + getImageOptions := new(images.GetOptions).WithSize(false) + hostInfo, err := ic.Info(context.Background()) if err != nil { return nil, err } + + for _, nameOrID := range namesOrIds { + ctrData, _, err := ic.ContainerInspect(ic.ClientCtx, []string{nameOrID}, entities.InspectOptions{}) + if err == nil && len(ctrData) > 0 { + ids = append(ids, ctrData[0].ID) + } else { + // If container was not found, check if this is a checkpoint image + inspectReport, err := images.GetImage(ic.ClientCtx, nameOrID, getImageOptions) + if err != nil { + return nil, fmt.Errorf("no such container or image: %s", nameOrID) + } + checkpointRuntimeName, found := inspectReport.Annotations[define.CheckpointAnnotationRuntimeName] + if !found { + return nil, fmt.Errorf("image is not a checkpoint: %s", nameOrID) + } + if hostInfo.Host.OCIRuntime.Name != checkpointRuntimeName { + return nil, fmt.Errorf("container image \"%s\" requires runtime: \"%s\"", nameOrID, checkpointRuntimeName) + } + ids = append(ids, inspectReport.ID) + } + } } - reports := make([]*entities.RestoreReport, 0, len(ctrs)) - for _, c := range ctrs { - report, err := containers.Restore(ic.ClientCtx, c.ID, options) + reports := make([]*entities.RestoreReport, 0, len(ids)) + for _, id := range ids { + report, err := containers.Restore(ic.ClientCtx, id, options) if err != nil { - reports = append(reports, &entities.RestoreReport{Id: c.ID, Err: err}) + reports = append(reports, &entities.RestoreReport{Id: id, Err: err}) } reports = append(reports, report) } diff --git a/test/e2e/checkpoint_image_test.go b/test/e2e/checkpoint_image_test.go new file mode 100644 index 000000000..8274dfc80 --- /dev/null +++ b/test/e2e/checkpoint_image_test.go @@ -0,0 +1,296 @@ +package integration + +import ( + "os" + "os/exec" + "strconv" + "strings" + + "github.com/containers/podman/v4/pkg/criu" + . "github.com/containers/podman/v4/test/utils" + . "github.com/onsi/ginkgo" + . "github.com/onsi/gomega" + . "github.com/onsi/gomega/gexec" +) + +var _ = Describe("Podman checkpoint", func() { + var ( + tempdir string + err error + podmanTest *PodmanTestIntegration + ) + + BeforeEach(func() { + SkipIfRootless("checkpoint not supported in rootless mode") + tempdir, err = CreateTempDirInTempDir() + if err != nil { + os.Exit(1) + } + podmanTest = PodmanTestCreate(tempdir) + podmanTest.Setup() + podmanTest.SeedImages() + // Check if the runtime implements checkpointing. Currently only + // runc's checkpoint/restore implementation is supported. + cmd := exec.Command(podmanTest.OCIRuntime, "checkpoint", "--help") + if err := cmd.Start(); err != nil { + Skip("OCI runtime does not support checkpoint/restore") + } + if err := cmd.Wait(); err != nil { + Skip("OCI runtime does not support checkpoint/restore") + } + + if !criu.CheckForCriu(criu.MinCriuVersion) { + Skip("CRIU is missing or too old.") + } + }) + + AfterEach(func() { + podmanTest.Cleanup() + f := CurrentGinkgoTestDescription() + processTestResult(f) + }) + + It("podman checkpoint --create-image with bogus container", func() { + checkpointImage := "foobar-checkpoint" + session := podmanTest.Podman([]string{"container", "checkpoint", "--create-image", checkpointImage, "foobar"}) + session.WaitWithDefaultTimeout() + Expect(session).To(ExitWithError()) + Expect(session.ErrorToString()).To(ContainSubstring("no container with name or ID \"foobar\" found")) + }) + + It("podman checkpoint --create-image with running container", func() { + // Container image must be lowercase + checkpointImage := "alpine-checkpoint-" + strings.ToLower(RandomString(6)) + containerName := "alpine-container-" + RandomString(6) + + localRunString := []string{ + "run", + "-it", + "-d", + "--ip", GetRandomIPAddress(), + "--name", containerName, + ALPINE, + "top", + } + session := podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + containerID := session.OutputToString() + + // Checkpoint image should not exist + session = podmanTest.Podman([]string{"images"}) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + Expect(session.LineInOutputContainsTag("localhost/"+checkpointImage, "latest")).To(BeFalse()) + + // Check if none of the checkpoint/restore specific information is displayed + // for newly started containers. + inspect := podmanTest.Podman([]string{"inspect", containerID}) + inspect.WaitWithDefaultTimeout() + Expect(inspect).Should(Exit(0)) + inspectOut := inspect.InspectContainerToJSON() + Expect(inspectOut[0].State.Checkpointed).To(BeFalse(), ".State.Checkpointed") + Expect(inspectOut[0].State.Restored).To(BeFalse(), ".State.Restored") + Expect(inspectOut[0].State.CheckpointPath).To(Equal("")) + Expect(inspectOut[0].State.CheckpointLog).To(Equal("")) + Expect(inspectOut[0].State.RestoreLog).To(Equal("")) + + result := podmanTest.Podman([]string{"container", "checkpoint", "--create-image", checkpointImage, "--keep", containerID}) + result.WaitWithDefaultTimeout() + + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Exited")) + + inspect = podmanTest.Podman([]string{"inspect", containerID}) + inspect.WaitWithDefaultTimeout() + Expect(inspect).Should(Exit(0)) + inspectOut = inspect.InspectContainerToJSON() + Expect(inspectOut[0].State.Checkpointed).To(BeTrue(), ".State.Checkpointed") + Expect(inspectOut[0].State.CheckpointPath).To(ContainSubstring("userdata/checkpoint")) + Expect(inspectOut[0].State.CheckpointLog).To(ContainSubstring("userdata/dump.log")) + + // Check if checkpoint image has been created + session = podmanTest.Podman([]string{"images"}) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + Expect(session.LineInOutputContainsTag("localhost/"+checkpointImage, "latest")).To(BeTrue()) + + // Check if the checkpoint image contains annotations + inspect = podmanTest.Podman([]string{"inspect", checkpointImage}) + inspect.WaitWithDefaultTimeout() + Expect(inspect).Should(Exit(0)) + inspectImageOut := inspect.InspectImageJSON() + Expect(inspectImageOut[0].Annotations["io.podman.annotations.checkpoint.name"]).To( + BeEquivalentTo(containerName), + "io.podman.annotations.checkpoint.name", + ) + + ociRuntimeName := "" + if strings.Contains(podmanTest.OCIRuntime, "runc") { + ociRuntimeName = "runc" + } else if strings.Contains(podmanTest.OCIRuntime, "crun") { + ociRuntimeName = "crun" + } + if ociRuntimeName != "" { + Expect(inspectImageOut[0].Annotations["io.podman.annotations.checkpoint.runtime.name"]).To( + BeEquivalentTo(ociRuntimeName), + "io.podman.annotations.checkpoint.runtime.name", + ) + } + + // Remove existing container + result = podmanTest.Podman([]string{"rm", "-t", "1", "-f", containerID}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + + // Restore container from checkpoint image + result = podmanTest.Podman([]string{"container", "restore", checkpointImage}) + result.WaitWithDefaultTimeout() + + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Clean-up + result = podmanTest.Podman([]string{"rm", "-t", "0", "-fa"}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + + result = podmanTest.Podman([]string{"rmi", checkpointImage}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + }) + + It("podman restore multiple containers from single checkpint image", func() { + // Container image must be lowercase + checkpointImage := "alpine-checkpoint-" + strings.ToLower(RandomString(6)) + containerName := "alpine-container-" + RandomString(6) + + localRunString := []string{"run", "-d", "--name", containerName, ALPINE, "top"} + session := podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + containerID := session.OutputToString() + + // Checkpoint image should not exist + session = podmanTest.Podman([]string{"images"}) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + Expect(session.LineInOutputContainsTag("localhost/"+checkpointImage, "latest")).To(BeFalse()) + + result := podmanTest.Podman([]string{"container", "checkpoint", "--create-image", checkpointImage, "--keep", containerID}) + result.WaitWithDefaultTimeout() + + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Exited")) + + // Check if checkpoint image has been created + session = podmanTest.Podman([]string{"images"}) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + Expect(session.LineInOutputContainsTag("localhost/"+checkpointImage, "latest")).To(BeTrue()) + + // Remove existing container + result = podmanTest.Podman([]string{"rm", "-t", "1", "-f", containerID}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + + for i := 1; i < 5; i++ { + // Restore container from checkpoint image + name := containerName + strconv.Itoa(i) + result = podmanTest.Podman([]string{"container", "restore", "--name", name, checkpointImage}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(i)) + + // Check that the container is running + status := podmanTest.Podman([]string{"inspect", name, "--format={{.State.Status}}"}) + status.WaitWithDefaultTimeout() + Expect(status).Should(Exit(0)) + Expect(status.OutputToString()).To(Equal("running")) + } + + // Clean-up + result = podmanTest.Podman([]string{"rm", "-t", "0", "-fa"}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + + result = podmanTest.Podman([]string{"rmi", checkpointImage}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + }) + + It("podman restore multiple containers from multiple checkpint images", func() { + // Container image must be lowercase + checkpointImage1 := "alpine-checkpoint-" + strings.ToLower(RandomString(6)) + checkpointImage2 := "alpine-checkpoint-" + strings.ToLower(RandomString(6)) + containerName1 := "alpine-container-" + RandomString(6) + containerName2 := "alpine-container-" + RandomString(6) + + // Create first container + localRunString := []string{"run", "-d", "--name", containerName1, ALPINE, "top"} + session := podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + containerID1 := session.OutputToString() + + // Create second container + localRunString = []string{"run", "-d", "--name", containerName2, ALPINE, "top"} + session = podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session).Should(Exit(0)) + containerID2 := session.OutputToString() + + // Checkpoint first container + result := podmanTest.Podman([]string{"container", "checkpoint", "--create-image", checkpointImage1, "--keep", containerID1}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + + // Checkpoint second container + result = podmanTest.Podman([]string{"container", "checkpoint", "--create-image", checkpointImage2, "--keep", containerID2}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + + // Remove existing containers + result = podmanTest.Podman([]string{"rm", "-t", "1", "-f", containerName1, containerName2}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + + // Restore both containers from images + result = podmanTest.Podman([]string{"container", "restore", checkpointImage1, checkpointImage2}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(2)) + + // Check if first container is running + status := podmanTest.Podman([]string{"inspect", containerName1, "--format={{.State.Status}}"}) + status.WaitWithDefaultTimeout() + Expect(status).Should(Exit(0)) + Expect(status.OutputToString()).To(Equal("running")) + + // Check if second container is running + status = podmanTest.Podman([]string{"inspect", containerName2, "--format={{.State.Status}}"}) + status.WaitWithDefaultTimeout() + Expect(status).Should(Exit(0)) + Expect(status.OutputToString()).To(Equal("running")) + + // Clean-up + result = podmanTest.Podman([]string{"rm", "-t", "0", "-fa"}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + + result = podmanTest.Podman([]string{"rmi", checkpointImage1, checkpointImage2}) + result.WaitWithDefaultTimeout() + Expect(result).Should(Exit(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + }) +}) diff --git a/test/e2e/common_test.go b/test/e2e/common_test.go index 9580230b5..2f4146bba 100644 --- a/test/e2e/common_test.go +++ b/test/e2e/common_test.go @@ -858,18 +858,15 @@ func (p *PodmanTestIntegration) makeOptions(args []string, noEvents, noCache boo eventsType = "none" } - networkBackend := p.NetworkBackend.ToString() - networkDir := p.NetworkConfigDir - if p.NetworkBackend == Netavark { - networkDir = p.NetworkConfigDir - } podmanOptions := strings.Split(fmt.Sprintf("%s--root %s --runroot %s --runtime %s --conmon %s --network-config-dir %s --cgroup-manager %s --tmpdir %s --events-backend %s", - debug, p.Root, p.RunRoot, p.OCIRuntime, p.ConmonBinary, networkDir, p.CgroupManager, p.TmpDir, eventsType), " ") + debug, p.Root, p.RunRoot, p.OCIRuntime, p.ConmonBinary, p.NetworkConfigDir, p.CgroupManager, p.TmpDir, eventsType), " ") if os.Getenv("HOOK_OPTION") != "" { podmanOptions = append(podmanOptions, os.Getenv("HOOK_OPTION")) } - podmanOptions = append(podmanOptions, "--network-backend", networkBackend) + if !p.RemoteTest { + podmanOptions = append(podmanOptions, "--network-backend", p.NetworkBackend.ToString()) + } podmanOptions = append(podmanOptions, strings.Split(p.StorageOptions, " ")...) if !noCache { diff --git a/test/e2e/create_test.go b/test/e2e/create_test.go index 11f8b5abf..4c3b5604a 100644 --- a/test/e2e/create_test.go +++ b/test/e2e/create_test.go @@ -118,7 +118,7 @@ var _ = Describe("Podman create", func() { result := podmanTest.Podman([]string{"inspect", "entrypoint_test", "--format", "{{.Config.Entrypoint}}"}) result.WaitWithDefaultTimeout() Expect(result).Should(Exit(0)) - Expect(result.OutputToString()).To(Equal("[/bin/foobar]")) + Expect(result.OutputToString()).To(Equal("/bin/foobar")) }) It("podman create --entrypoint \"\"", func() { @@ -130,7 +130,7 @@ var _ = Describe("Podman create", func() { result := podmanTest.Podman([]string{"inspect", session.OutputToString(), "--format", "{{.Config.Entrypoint}}"}) result.WaitWithDefaultTimeout() Expect(result).Should(Exit(0)) - Expect(result.OutputToString()).To(Equal("[]")) + Expect(result.OutputToString()).To(Equal("")) }) It("podman create --entrypoint json", func() { @@ -143,7 +143,7 @@ var _ = Describe("Podman create", func() { result := podmanTest.Podman([]string{"inspect", "entrypoint_json", "--format", "{{.Config.Entrypoint}}"}) result.WaitWithDefaultTimeout() Expect(result).Should(Exit(0)) - Expect(result.OutputToString()).To(Equal("[/bin/foo -c]")) + Expect(result.OutputToString()).To(Equal("/bin/foo -c")) }) It("podman create --mount flag with multiple mounts", func() { @@ -281,8 +281,7 @@ var _ = Describe("Podman create", func() { Expect(ctrJSON).To(HaveLen(1)) Expect(ctrJSON[0].Config.Cmd).To(HaveLen(1)) Expect(ctrJSON[0].Config.Cmd[0]).To(Equal("redis-server")) - Expect(ctrJSON[0].Config.Entrypoint).To(HaveLen(1)) - Expect(ctrJSON[0].Config.Entrypoint[0]).To(Equal("docker-entrypoint.sh")) + Expect(ctrJSON[0].Config.Entrypoint).To(Equal("docker-entrypoint.sh")) }) It("podman create --pull", func() { diff --git a/test/e2e/libpod_suite_remote_test.go b/test/e2e/libpod_suite_remote_test.go index dddcf5c14..9ad2bf7b9 100644 --- a/test/e2e/libpod_suite_remote_test.go +++ b/test/e2e/libpod_suite_remote_test.go @@ -137,6 +137,9 @@ func getRemoteOptions(p *PodmanTestIntegration, args []string) []string { if os.Getenv("HOOK_OPTION") != "" { podmanOptions = append(podmanOptions, os.Getenv("HOOK_OPTION")) } + if p.NetworkBackend.ToString() == "netavark" { + podmanOptions = append(podmanOptions, "--network-backend", "netavark") + } podmanOptions = append(podmanOptions, strings.Split(p.StorageOptions, " ")...) podmanOptions = append(podmanOptions, args...) return podmanOptions diff --git a/test/e2e/pod_create_test.go b/test/e2e/pod_create_test.go index dc43ce6fd..8def80213 100644 --- a/test/e2e/pod_create_test.go +++ b/test/e2e/pod_create_test.go @@ -368,7 +368,7 @@ var _ = Describe("Podman pod create", func() { check1 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Config.Entrypoint}}", data.Containers[0].ID}) check1.WaitWithDefaultTimeout() Expect(check1).Should(Exit(0)) - Expect(check1.OutputToString()).To(Equal("[/catatonit -P]")) + Expect(check1.OutputToString()).To(Equal("/catatonit -P")) // check the Path and Args check2 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Path}}:{{.Args}}", data.Containers[0].ID}) @@ -391,7 +391,7 @@ var _ = Describe("Podman pod create", func() { check1 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Config.Entrypoint}}", data.Containers[0].ID}) check1.WaitWithDefaultTimeout() Expect(check1).Should(Exit(0)) - Expect(check1.OutputToString()).To(Equal("[/pause1]")) + Expect(check1.OutputToString()).To(Equal("/pause1")) // check the Path and Args check2 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Path}}:{{.Args}}", data.Containers[0].ID}) @@ -418,7 +418,7 @@ entrypoint ["/fromimage"] check1 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Config.Entrypoint}}", data.Containers[0].ID}) check1.WaitWithDefaultTimeout() Expect(check1).Should(Exit(0)) - Expect(check1.OutputToString()).To(Equal("[/fromimage]")) + Expect(check1.OutputToString()).To(Equal("/fromimage")) // check the Path and Args check2 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Path}}:{{.Args}}", data.Containers[0].ID}) @@ -445,7 +445,7 @@ entrypoint ["/fromimage"] check1 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Config.Entrypoint}}", data.Containers[0].ID}) check1.WaitWithDefaultTimeout() Expect(check1).Should(Exit(0)) - Expect(check1.OutputToString()).To(Equal("[/fromcommand]")) + Expect(check1.OutputToString()).To(Equal("/fromcommand")) // check the Path and Args check2 := podmanTest.Podman([]string{"container", "inspect", "--format", "{{.Path}}:{{.Args}}", data.Containers[0].ID}) diff --git a/test/system/200-pod.bats b/test/system/200-pod.bats index e6f4ecdbc..f5fe41924 100644 --- a/test/system/200-pod.bats +++ b/test/system/200-pod.bats @@ -234,7 +234,7 @@ EOF local infra_cid="$output" # confirm that entrypoint is what we set run_podman container inspect --format '{{.Config.Entrypoint}}' $infra_cid - is "$output" "[$infra_command]" "infra-command took effect" + is "$output" "$infra_command" "infra-command took effect" # confirm that infra container name is set run_podman container inspect --format '{{.Name}}' $infra_cid is "$output" "$infra_name" "infra-name took effect" diff --git a/test/system/520-checkpoint.bats b/test/system/520-checkpoint.bats index 046dfd126..b41d460a4 100644 --- a/test/system/520-checkpoint.bats +++ b/test/system/520-checkpoint.bats @@ -47,7 +47,8 @@ function teardown() { # Checkpoint, and confirm via inspect run_podman container checkpoint $cid - is "$output" "$cid" "podman container checkpoint" + # FIXME: remove the `.*` prefix after fix packaged for https://github.com/checkpoint-restore/criu/pull/1706 + is "$output" ".*$cid" "podman container checkpoint" run_podman container inspect \ --format '{{.State.Status}}:{{.State.Running}}:{{.State.Paused}}:{{.State.Checkpointed}}' $cid diff --git a/test/upgrade/test-upgrade.bats b/test/upgrade/test-upgrade.bats index 198d8a169..5efe05d49 100644 --- a/test/upgrade/test-upgrade.bats +++ b/test/upgrade/test-upgrade.bats @@ -146,6 +146,12 @@ EOF # cause connectivity issues since cni and netavark should never be mixed. mkdir -p /run/netns /run/cni /run/containers /var/lib/cni /etc/cni/net.d + # Containers-common around release 1-55 no-longer supplies this file + sconf=/etc/containers/storage.conf + v_sconf= + if [[ -e "$sconf" ]]; then + v_sconf="-v $sconf:$sconf" + fi # # Use new-podman to run the above script under old-podman. @@ -165,7 +171,7 @@ EOF --net=host \ --cgroupns=host \ --pid=host \ - -v /etc/containers/storage.conf:/etc/containers/storage.conf \ + $v_sconf \ -v /dev/fuse:/dev/fuse \ -v /run/crun:/run/crun \ -v /run/netns:/run/netns:rshared \ diff --git a/test/utils/utils.go b/test/utils/utils.go index da56a3a2e..39a0ac875 100644 --- a/test/utils/utils.go +++ b/test/utils/utils.go @@ -110,7 +110,7 @@ func (p *PodmanTest) PodmanAsUserBase(args []string, uid, gid uint32, cwd string } runCmd := append(wrapper, podmanBinary) - if p.NetworkBackend == Netavark { + if !p.RemoteTest && p.NetworkBackend == Netavark { runCmd = append(runCmd, []string{"--network-backend", "netavark"}...) } diff --git a/vendor/github.com/checkpoint-restore/checkpointctl/lib/metadata.go b/vendor/github.com/checkpoint-restore/checkpointctl/lib/metadata.go index 712fd2d50..3b1abd928 100644 --- a/vendor/github.com/checkpoint-restore/checkpointctl/lib/metadata.go +++ b/vendor/github.com/checkpoint-restore/checkpointctl/lib/metadata.go @@ -48,13 +48,16 @@ const ( // kubelet archive CheckpointedPodsFile = "checkpointed.pods" // container archive - ConfigDumpFile = "config.dump" - SpecDumpFile = "spec.dump" - NetworkStatusFile = "network.status" - CheckpointDirectory = "checkpoint" - DevShmCheckpointTar = "devshm-checkpoint.tar" - RootFsDiffTar = "rootfs-diff.tar" - DeletedFilesFile = "deleted.files" + ConfigDumpFile = "config.dump" + SpecDumpFile = "spec.dump" + NetworkStatusFile = "network.status" + CheckpointDirectory = "checkpoint" + CheckpointVolumesDirectory = "volumes" + DevShmCheckpointTar = "devshm-checkpoint.tar" + RootFsDiffTar = "rootfs-diff.tar" + DeletedFilesFile = "deleted.files" + DumpLogFile = "dump.log" + RestoreLogFile = "restore.log" // pod archive PodOptionsFile = "pod.options" PodDumpFile = "pod.dump" diff --git a/vendor/github.com/containers/common/libimage/copier.go b/vendor/github.com/containers/common/libimage/copier.go index 2a8f47f7f..01cedc7ed 100644 --- a/vendor/github.com/containers/common/libimage/copier.go +++ b/vendor/github.com/containers/common/libimage/copier.go @@ -147,15 +147,13 @@ type copier struct { destinationLookup LookupReferenceFunc } -var ( - // storageAllowedPolicyScopes overrides the policy for local storage - // to ensure that we can read images from it. - storageAllowedPolicyScopes = signature.PolicyTransportScopes{ - "": []signature.PolicyRequirement{ - signature.NewPRInsecureAcceptAnything(), - }, - } -) +// storageAllowedPolicyScopes overrides the policy for local storage +// to ensure that we can read images from it. +var storageAllowedPolicyScopes = signature.PolicyTransportScopes{ + "": []signature.PolicyRequirement{ + signature.NewPRInsecureAcceptAnything(), + }, +} // getDockerAuthConfig extracts a docker auth config from the CopyOptions. Returns // nil if no credentials are set. diff --git a/vendor/github.com/containers/common/libimage/image_config.go b/vendor/github.com/containers/common/libimage/image_config.go index 140202440..683a2dc98 100644 --- a/vendor/github.com/containers/common/libimage/image_config.go +++ b/vendor/github.com/containers/common/libimage/image_config.go @@ -95,9 +95,7 @@ func ImageConfigFromChanges(changes []string) (*ImageConfig, error) { // nolint: // For now: we only support key=value // We will attempt to strip quotation marks if present. - var ( - key, val string - ) + var key, val string splitEnv := strings.SplitN(value, "=", 2) key = splitEnv[0] diff --git a/vendor/github.com/containers/common/libimage/inspect.go b/vendor/github.com/containers/common/libimage/inspect.go index d44ebf46e..05d60edfc 100644 --- a/vendor/github.com/containers/common/libimage/inspect.go +++ b/vendor/github.com/containers/common/libimage/inspect.go @@ -213,7 +213,6 @@ func (i *Image) inspectInfo(ctx context.Context) (*types.ImageInspectInfo, error ref, err := i.StorageReference() if err != nil { - return nil, err } diff --git a/vendor/github.com/containers/common/libimage/manifests/copy.go b/vendor/github.com/containers/common/libimage/manifests/copy.go index 7e651a46c..578b64ca8 100644 --- a/vendor/github.com/containers/common/libimage/manifests/copy.go +++ b/vendor/github.com/containers/common/libimage/manifests/copy.go @@ -4,12 +4,10 @@ import ( "github.com/containers/image/v5/signature" ) -var ( - // storageAllowedPolicyScopes overrides the policy for local storage - // to ensure that we can read images from it. - storageAllowedPolicyScopes = signature.PolicyTransportScopes{ - "": []signature.PolicyRequirement{ - signature.NewPRInsecureAcceptAnything(), - }, - } -) +// storageAllowedPolicyScopes overrides the policy for local storage +// to ensure that we can read images from it. +var storageAllowedPolicyScopes = signature.PolicyTransportScopes{ + "": []signature.PolicyRequirement{ + signature.NewPRInsecureAcceptAnything(), + }, +} diff --git a/vendor/github.com/containers/common/libimage/manifests/manifests.go b/vendor/github.com/containers/common/libimage/manifests/manifests.go index ccff908c9..2624dee78 100644 --- a/vendor/github.com/containers/common/libimage/manifests/manifests.go +++ b/vendor/github.com/containers/common/libimage/manifests/manifests.go @@ -384,10 +384,8 @@ func (l *list) Add(ctx context.Context, sys *types.SystemContext, ref types.Imag } instanceInfo.instanceDigest = &manifestDigest instanceInfo.Size = int64(len(manifestBytes)) - } else { - if manifestDigest == "" { - manifestDigest = *instanceInfo.instanceDigest - } + } else if manifestDigest == "" { + manifestDigest = *instanceInfo.instanceDigest } err = l.List.AddInstance(*instanceInfo.instanceDigest, instanceInfo.Size, manifestType, instanceInfo.OS, instanceInfo.Architecture, instanceInfo.OSVersion, instanceInfo.OSFeatures, instanceInfo.Variant, instanceInfo.Features, instanceInfo.Annotations) if err != nil { @@ -405,9 +403,7 @@ func (l *list) Add(ctx context.Context, sys *types.SystemContext, ref types.Imag func (l *list) Remove(instanceDigest digest.Digest) error { err := l.List.Remove(instanceDigest) if err == nil { - if _, needToDelete := l.instances[instanceDigest]; needToDelete { - delete(l.instances, instanceDigest) - } + delete(l.instances, instanceDigest) } return err } diff --git a/vendor/github.com/containers/common/libimage/runtime.go b/vendor/github.com/containers/common/libimage/runtime.go index 2191e3c4a..974b50b50 100644 --- a/vendor/github.com/containers/common/libimage/runtime.go +++ b/vendor/github.com/containers/common/libimage/runtime.go @@ -74,7 +74,7 @@ func (r *Runtime) SystemContext() *types.SystemContext { // Returns a copy of the runtime's system context. func (r *Runtime) systemContextCopy() *types.SystemContext { var sys types.SystemContext - deepcopy.Copy(&sys, &r.systemContext) + _ = deepcopy.Copy(&sys, &r.systemContext) return &sys } diff --git a/vendor/github.com/containers/common/libimage/save.go b/vendor/github.com/containers/common/libimage/save.go index e1b8c3f75..fed86d4ef 100644 --- a/vendor/github.com/containers/common/libimage/save.go +++ b/vendor/github.com/containers/common/libimage/save.go @@ -68,7 +68,6 @@ func (r *Runtime) Save(ctx context.Context, names []string, format, path string, } return errors.Errorf("unsupported format %q for saving images", format) - } // saveSingleImage saves the specified image name to the specified path. diff --git a/vendor/github.com/containers/common/libnetwork/cni/cni_conversion.go b/vendor/github.com/containers/common/libnetwork/cni/cni_conversion.go index 36ac468de..bda7ed7d0 100644 --- a/vendor/github.com/containers/common/libnetwork/cni/cni_conversion.go +++ b/vendor/github.com/containers/common/libnetwork/cni/cni_conversion.go @@ -11,7 +11,6 @@ import ( "path/filepath" "strconv" "strings" - "syscall" "time" "github.com/containernetworking/cni/libcni" @@ -21,6 +20,7 @@ import ( pkgutil "github.com/containers/common/pkg/util" "github.com/pkg/errors" "github.com/sirupsen/logrus" + "golang.org/x/sys/unix" ) func createNetworkFromCNIConfigList(conf *libcni.NetworkConfigList, confPath string) (*types.Network, error) { @@ -45,12 +45,11 @@ func createNetworkFromCNIConfigList(conf *libcni.NetworkConfigList, confPath str } } - f, err := os.Stat(confPath) + t, err := fileTime(confPath) if err != nil { return nil, err } - stat := f.Sys().(*syscall.Stat_t) - network.Created = time.Unix(int64(stat.Ctim.Sec), int64(stat.Ctim.Nsec)) + network.Created = t firstPlugin := conf.Plugins[0] network.Driver = firstPlugin.Network.Type @@ -316,16 +315,15 @@ func (n *cniNetwork) createCNIConfigListFromNetwork(network *types.Network, writ cniPathName := "" if writeToDisk { cniPathName = filepath.Join(n.cniConfigDir, network.Name+".conflist") - err = ioutil.WriteFile(cniPathName, b, 0644) + err = ioutil.WriteFile(cniPathName, b, 0o644) if err != nil { return nil, "", err } - f, err := os.Stat(cniPathName) + t, err := fileTime(cniPathName) if err != nil { return nil, "", err } - stat := f.Sys().(*syscall.Stat_t) - network.Created = time.Unix(int64(stat.Ctim.Sec), int64(stat.Ctim.Nsec)) + network.Created = t } else { network.Created = time.Now() } @@ -424,3 +422,17 @@ func parseOptions(networkOptions map[string]string, networkDriver string) (*opti } return opt, nil } + +func fileTime(file string) (time.Time, error) { + var st unix.Stat_t + for { + err := unix.Stat(file, &st) + if err == nil { + break + } + if err != unix.EINTR { //nolint:errorlint // unix errors are bare + return time.Time{}, &os.PathError{Path: file, Op: "stat", Err: err} + } + } + return time.Unix(int64(st.Ctim.Sec), int64(st.Ctim.Nsec)), nil //nolint:unconvert // On some platforms Sec and Nsec are int32. +} diff --git a/vendor/github.com/containers/common/libnetwork/cni/config.go b/vendor/github.com/containers/common/libnetwork/cni/config.go index e94a53db6..c6967b600 100644 --- a/vendor/github.com/containers/common/libnetwork/cni/config.go +++ b/vendor/github.com/containers/common/libnetwork/cni/config.go @@ -17,7 +17,6 @@ import ( // NetworkCreate will take a partial filled Network and fill the // missing fields. It creates the Network and returns the full Network. -// nolint:gocritic func (n *cniNetwork) NetworkCreate(net types.Network) (types.Network, error) { n.lock.Lock() defer n.lock.Unlock() diff --git a/vendor/github.com/containers/common/libnetwork/internal/util/util.go b/vendor/github.com/containers/common/libnetwork/internal/util/util.go index 0ae8b6443..6b76a700b 100644 --- a/vendor/github.com/containers/common/libnetwork/internal/util/util.go +++ b/vendor/github.com/containers/common/libnetwork/internal/util/util.go @@ -109,7 +109,6 @@ func GetFreeIPv4NetworkSubnet(usedNetworks []*net.IPNet, subnetPools []config.Su return nil, err } return nil, errors.New("could not find free subnet from subnet pools") - } // GetFreeIPv6NetworkSubnet returns a unused ipv6 subnet diff --git a/vendor/github.com/containers/common/libnetwork/netavark/config.go b/vendor/github.com/containers/common/libnetwork/netavark/config.go index 6a08de55c..0bb0539b2 100644 --- a/vendor/github.com/containers/common/libnetwork/netavark/config.go +++ b/vendor/github.com/containers/common/libnetwork/netavark/config.go @@ -19,7 +19,6 @@ import ( // NetworkCreate will take a partial filled Network and fill the // missing fields. It creates the Network and returns the full Network. -// nolint:gocritic func (n *netavarkNetwork) NetworkCreate(net types.Network) (types.Network, error) { n.lock.Lock() defer n.lock.Unlock() diff --git a/vendor/github.com/containers/common/libnetwork/netavark/ipam.go b/vendor/github.com/containers/common/libnetwork/netavark/ipam.go index c0535515a..861854351 100644 --- a/vendor/github.com/containers/common/libnetwork/netavark/ipam.go +++ b/vendor/github.com/containers/common/libnetwork/netavark/ipam.go @@ -59,9 +59,7 @@ func newIPAMError(cause error, msg string, args ...interface{}) *ipamError { // openDB will open the ipam database // Note that the caller has to Close it. func (n *netavarkNetwork) openDB() (*bbolt.DB, error) { - // linter complains about the octal value - // nolint:gocritic - db, err := bbolt.Open(n.ipamDBPath, 0600, nil) + db, err := bbolt.Open(n.ipamDBPath, 0o600, nil) if err != nil { return nil, newIPAMError(err, "failed to open database %s", n.ipamDBPath) } diff --git a/vendor/github.com/containers/common/libnetwork/netavark/network.go b/vendor/github.com/containers/common/libnetwork/netavark/network.go index 15d1f03eb..9c8c4bfb4 100644 --- a/vendor/github.com/containers/common/libnetwork/netavark/network.go +++ b/vendor/github.com/containers/common/libnetwork/netavark/network.go @@ -108,11 +108,11 @@ func NewNetworkInterface(conf *InitConfig) (types.ContainerNetwork, error) { return nil, errors.Wrap(err, "failed to parse default subnet") } - if err := os.MkdirAll(conf.NetworkConfigDir, 0755); err != nil { + if err := os.MkdirAll(conf.NetworkConfigDir, 0o755); err != nil { return nil, err } - if err := os.MkdirAll(conf.NetworkRunDir, 0755); err != nil { + if err := os.MkdirAll(conf.NetworkRunDir, 0o755); err != nil { return nil, err } diff --git a/vendor/github.com/containers/common/libnetwork/network/interface.go b/vendor/github.com/containers/common/libnetwork/network/interface.go index 9278d7773..e452e6cd5 100644 --- a/vendor/github.com/containers/common/libnetwork/network/interface.go +++ b/vendor/github.com/containers/common/libnetwork/network/interface.go @@ -121,8 +121,7 @@ func defaultNetworkBackend(store storage.Store, conf *config.Config) (backend ty defer func() { // only write when there is no error if err == nil { - // nolint:gocritic - if err := ioutils.AtomicWriteFile(file, []byte(backend), 0644); err != nil { + if err := ioutils.AtomicWriteFile(file, []byte(backend), 0o644); err != nil { logrus.Errorf("could not write network backend to file: %v", err) } } diff --git a/vendor/github.com/containers/common/pkg/apparmor/apparmor_linux.go b/vendor/github.com/containers/common/pkg/apparmor/apparmor_linux.go index c864a189e..35f79a1ad 100644 --- a/vendor/github.com/containers/common/pkg/apparmor/apparmor_linux.go +++ b/vendor/github.com/containers/common/pkg/apparmor/apparmor_linux.go @@ -233,7 +233,6 @@ func parseAAParserVersion(output string) (int, error) { // major*10^5 + minor*10^3 + patch*10^0 numericVersion := majorVersion*1e5 + minorVersion*1e3 + patchLevel return numericVersion, nil - } // CheckProfileAndLoadDefault checks if the specified profile is loaded and diff --git a/vendor/github.com/containers/common/pkg/cgroups/blkio.go b/vendor/github.com/containers/common/pkg/cgroups/blkio.go index bacd4eb93..0fb61c757 100644 --- a/vendor/github.com/containers/common/pkg/cgroups/blkio.go +++ b/vendor/github.com/containers/common/pkg/cgroups/blkio.go @@ -12,8 +12,7 @@ import ( "github.com/pkg/errors" ) -type blkioHandler struct { -} +type blkioHandler struct{} func getBlkioHandler() *blkioHandler { return &blkioHandler{} diff --git a/vendor/github.com/containers/common/pkg/cgroups/cgroups.go b/vendor/github.com/containers/common/pkg/cgroups/cgroups.go index 0bf275f38..57997d652 100644 --- a/vendor/github.com/containers/common/pkg/cgroups/cgroups.go +++ b/vendor/github.com/containers/common/pkg/cgroups/cgroups.go @@ -265,7 +265,7 @@ func createCgroupv2Path(path string) (deferredError error) { for i, e := range elements[3:] { current = filepath.Join(current, e) if i > 0 { - if err := os.Mkdir(current, 0755); err != nil { + if err := os.Mkdir(current, 0o755); err != nil { if !os.IsExist(err) { return err } @@ -281,7 +281,7 @@ func createCgroupv2Path(path string) (deferredError error) { // We enable the controllers for all the path components except the last one. It is not allowed to add // PIDs if there are already enabled controllers. if i < len(elements[3:])-1 { - if err := ioutil.WriteFile(filepath.Join(current, "cgroup.subtree_control"), res, 0755); err != nil { + if err := ioutil.WriteFile(filepath.Join(current, "cgroup.subtree_control"), res, 0o755); err != nil { return err } } @@ -323,7 +323,7 @@ func (c *CgroupControl) initialize() (err error) { continue } path := c.getCgroupv1Path(ctr.name) - if err := os.MkdirAll(path, 0755); err != nil { + if err := os.MkdirAll(path, 0o755); err != nil { return errors.Wrapf(err, "error creating cgroup path for %s", ctr.name) } } @@ -343,7 +343,7 @@ func (c *CgroupControl) createCgroupDirectory(controller string) (bool, error) { return false, err } - if err := os.MkdirAll(cPath, 0755); err != nil { + if err := os.MkdirAll(cPath, 0o755); err != nil { return false, errors.Wrapf(err, "error creating cgroup for %s", controller) } return true, nil @@ -589,7 +589,7 @@ func (c *CgroupControl) AddPid(pid int) error { if c.cgroup2 { p := filepath.Join(cgroupRoot, c.path, "cgroup.procs") - if err := ioutil.WriteFile(p, pidString, 0644); err != nil { + if err := ioutil.WriteFile(p, pidString, 0o644); err != nil { return errors.Wrapf(err, "write %s", p) } return nil @@ -612,7 +612,7 @@ func (c *CgroupControl) AddPid(pid int) error { continue } p := filepath.Join(c.getCgroupv1Path(n), "tasks") - if err := ioutil.WriteFile(p, pidString, 0644); err != nil { + if err := ioutil.WriteFile(p, pidString, 0o644); err != nil { return errors.Wrapf(err, "write %s", p) } } diff --git a/vendor/github.com/containers/common/pkg/cgroups/cpu.go b/vendor/github.com/containers/common/pkg/cgroups/cpu.go index 23539757d..c9e94f269 100644 --- a/vendor/github.com/containers/common/pkg/cgroups/cpu.go +++ b/vendor/github.com/containers/common/pkg/cgroups/cpu.go @@ -12,8 +12,7 @@ import ( "github.com/pkg/errors" ) -type cpuHandler struct { -} +type cpuHandler struct{} func getCPUHandler() *cpuHandler { return &cpuHandler{} diff --git a/vendor/github.com/containers/common/pkg/cgroups/cpuset.go b/vendor/github.com/containers/common/pkg/cgroups/cpuset.go index 22ac0a079..2bfeb80db 100644 --- a/vendor/github.com/containers/common/pkg/cgroups/cpuset.go +++ b/vendor/github.com/containers/common/pkg/cgroups/cpuset.go @@ -10,8 +10,7 @@ import ( "github.com/pkg/errors" ) -type cpusetHandler struct { -} +type cpusetHandler struct{} func cpusetCopyFileFromParent(dir, file string, cgroupv2 bool) ([]byte, error) { if dir == cgroupRoot { @@ -33,7 +32,7 @@ func cpusetCopyFileFromParent(dir, file string, cgroupv2 bool) ([]byte, error) { if err != nil { return nil, err } - if err := ioutil.WriteFile(path, data, 0644); err != nil { + if err := ioutil.WriteFile(path, data, 0o644); err != nil { return nil, errors.Wrapf(err, "write %s", path) } return data, nil diff --git a/vendor/github.com/containers/common/pkg/cgroups/pids.go b/vendor/github.com/containers/common/pkg/cgroups/pids.go index 58cb32b3b..650120a56 100644 --- a/vendor/github.com/containers/common/pkg/cgroups/pids.go +++ b/vendor/github.com/containers/common/pkg/cgroups/pids.go @@ -8,8 +8,7 @@ import ( spec "github.com/opencontainers/runtime-spec/specs-go" ) -type pidHandler struct { -} +type pidHandler struct{} func getPidsHandler() *pidHandler { return &pidHandler{} @@ -29,7 +28,7 @@ func (c *pidHandler) Apply(ctr *CgroupControl, res *spec.LinuxResources) error { } p := filepath.Join(PIDRoot, "pids.max") - return ioutil.WriteFile(p, []byte(fmt.Sprintf("%d\n", res.Pids.Limit)), 0644) + return ioutil.WriteFile(p, []byte(fmt.Sprintf("%d\n", res.Pids.Limit)), 0o644) } // Create the cgroup diff --git a/vendor/github.com/containers/common/pkg/chown/chown_unix.go b/vendor/github.com/containers/common/pkg/chown/chown_unix.go index ea8f5963e..61327fd24 100644 --- a/vendor/github.com/containers/common/pkg/chown/chown_unix.go +++ b/vendor/github.com/containers/common/pkg/chown/chown_unix.go @@ -41,7 +41,6 @@ func ChangeHostPathOwnership(path string, recursive bool, uid, gid int) error { return nil }) - if err != nil { return errors.Wrap(err, "failed to chown recursively host path") } diff --git a/vendor/github.com/containers/common/pkg/config/config.go b/vendor/github.com/containers/common/pkg/config/config.go index 77654406a..319b8d153 100644 --- a/vendor/github.com/containers/common/pkg/config/config.go +++ b/vendor/github.com/containers/common/pkg/config/config.go @@ -95,6 +95,13 @@ type ContainersConfig struct { // Annotation to add to all containers Annotations []string `toml:"annotations,omitempty"` + // BaseHostsFile is the path to a hosts file, the entries from this file + // are added to the containers hosts file. As special value "image" is + // allowed which uses the /etc/hosts file from within the image and "none" + // which uses no base file at all. If it is empty we should default + // to /etc/hosts. + BaseHostsFile string `toml:"base_hosts_file,omitempty"` + // Default way to create a cgroup namespace for the container CgroupNS string `toml:"cgroupns,omitempty"` @@ -136,6 +143,9 @@ type ContainersConfig struct { // EnvHost Pass all host environment variables into the container. EnvHost bool `toml:"env_host,omitempty"` + // HostContainersInternalIP is used to set a specific host.containers.internal ip. + HostContainersInternalIP string `toml:"host_containers_internal_ip,omitempty"` + // HTTPProxy is the proxy environment variable list to apply to container process HTTPProxy bool `toml:"http_proxy,omitempty"` @@ -252,7 +262,7 @@ type EngineConfig struct { // EventsLogFileMaxSize sets the maximum size for the events log. When the limit is exceeded, // the logfile is rotated and the old one is deleted. - EventsLogFileMaxSize uint64 `toml:"events_logfile_max_size,omitempty,omitzero"` + EventsLogFileMaxSize eventsLogMaxSize `toml:"events_logfile_max_size,omitzero"` // EventsLogger determines where events should be logged. EventsLogger string `toml:"events_logger,omitempty"` @@ -581,7 +591,6 @@ type Destination struct { // with cgroupv2v2. Other OCI runtimes are not yet supporting cgroupv2v2. This // might change in the future. func NewConfig(userConfigPath string) (*Config, error) { - // Generate the default config for the system config, err := DefaultConfig() if err != nil { @@ -765,7 +774,6 @@ func (c *Config) addCAPPrefix() { // Validate is the main entry point for library configuration validation. func (c *Config) Validate() error { - if err := c.Containers.Validate(); err != nil { return errors.Wrap(err, "validating containers config") } @@ -822,7 +830,6 @@ func (c *EngineConfig) Validate() error { // It returns an `error` on validation failure, otherwise // `nil`. func (c *ContainersConfig) Validate() error { - if err := c.validateUlimits(); err != nil { return err } @@ -954,7 +961,6 @@ func (c *Config) GetDefaultEnvEx(envHost, httpProxy bool) []string { // Capabilities returns the capabilities parses the Add and Drop capability // list from the default capabiltiies for the container func (c *Config) Capabilities(user string, addCapabilities, dropCapabilities []string) ([]string, error) { - userNotRoot := func(user string) bool { if user == "" || user == "root" || user == "0" { return false @@ -1014,7 +1020,7 @@ func Device(device string) (src, dst, permissions string, err error) { // IsValidDeviceMode checks if the mode for device is valid or not. // IsValid mode is a composition of r (read), w (write), and m (mknod). func IsValidDeviceMode(mode string) bool { - var legalDeviceMode = map[rune]bool{ + legalDeviceMode := map[rune]bool{ 'r': true, 'w': true, 'm': true, @@ -1065,7 +1071,6 @@ func rootlessConfigPath() (string, error) { } func stringsEq(a, b []string) bool { - if len(a) != len(b) { return false } @@ -1150,10 +1155,10 @@ func (c *Config) Write() error { if err != nil { return err } - if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { return err } - configFile, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0644) + configFile, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0o644) if err != nil { return err } @@ -1266,3 +1271,33 @@ func (c *Config) setupEnv() error { } return nil } + +// eventsLogMaxSize is the type used by EventsLogFileMaxSize +type eventsLogMaxSize uint64 + +// UnmarshalText parses the JSON encoding of eventsLogMaxSize and +// stores it in a value. +func (e *eventsLogMaxSize) UnmarshalText(text []byte) error { + // REMOVE once writing works + if string(text) == "" { + return nil + } + val, err := units.FromHumanSize((string(text))) + if err != nil { + return err + } + if val < 0 { + return fmt.Errorf("events log file max size cannot be negative: %s", string(text)) + } + *e = eventsLogMaxSize(uint64(val)) + return nil +} + +// MarshalText returns the JSON encoding of eventsLogMaxSize. +func (e eventsLogMaxSize) MarshalText() ([]byte, error) { + if uint64(e) == DefaultEventsLogSizeMax || e == 0 { + v := []byte{} + return v, nil + } + return []byte(fmt.Sprintf("%d", e)), nil +} diff --git a/vendor/github.com/containers/common/pkg/config/containers.conf b/vendor/github.com/containers/common/pkg/config/containers.conf index 923b668bb..429b254bc 100644 --- a/vendor/github.com/containers/common/pkg/config/containers.conf +++ b/vendor/github.com/containers/common/pkg/config/containers.conf @@ -26,6 +26,13 @@ # #apparmor_profile = "container-default" +# The hosts entries from the base hosts file are added to the containers hosts +# file. This must be either an absolute path or as special values "image" which +# uses the hosts file from the container image or "none" which means +# no base hosts file is used. The default is "" which will use /etc/hosts. +# +#base_hosts_file = "" + # Default way to to create a cgroup namespace for the container # Options are: # `private` Create private Cgroup Namespace for the container. @@ -114,6 +121,16 @@ default_sysctls = [ # #env_host = false +# Set the ip for the host.containers.internal entry in the containers /etc/hosts +# file. This can be set to "none" to disable adding this entry. By default it +# will automatically choose the host ip. +# +# NOTE: When using podman machine this entry will never be added to the containers +# hosts file instead the gvproxy dns resolver will resolve this hostname. Therefore +# it is not possible to disable the entry in this case. +# +#host_containers_internal_ip = "" + # Default proxy environment variables passed into the container. # The environment variables passed in include: # http_proxy, https_proxy, ftp_proxy, no_proxy, and the upper case versions of @@ -373,11 +390,14 @@ default_sysctls = [ # Define where event logs will be stored, when events_logger is "file". #events_logfile_path="" -# Sets the maximum size for events_logfile_path in bytes. When the limit is exceeded, -# the logfile will be rotated and the old one will be deleted. +# Sets the maximum size for events_logfile_path. +# The size can be b (bytes), k (kilobytes), m (megabytes), or g (gigabytes). +# The format for the size is `<number><unit>`, e.g., `1b` or `3g`. +# If no unit is included then the size will be read in bytes. +# When the limit is exceeded, the logfile will be rotated and the old one will be deleted. # If the maximum size is set to 0, then no limit will be applied, # and the logfile will not be rotated. -#events_logfile_max_size = 0 +#events_logfile_max_size = "1m" # Selects which logging mechanism to use for container engine events. # Valid values are `journald`, `file` and `none`. @@ -461,9 +481,26 @@ default_sysctls = [ #network_cmd_path = "" # Default options to pass to the slirp4netns binary. -# For example "allow_host_loopback=true" -# -#network_cmd_options = ["enable_ipv6=true",] +# Valid options values are: +# +# - allow_host_loopback=true|false: Allow the slirp4netns to reach the host loopback IP (`10.0.2.2`). +# Default is false. +# - mtu=MTU: Specify the MTU to use for this network. (Default is `65520`). +# - cidr=CIDR: Specify ip range to use for this network. (Default is `10.0.2.0/24`). +# - enable_ipv6=true|false: Enable IPv6. Default is true. (Required for `outbound_addr6`). +# - outbound_addr=INTERFACE: Specify the outbound interface slirp should bind to (ipv4 traffic only). +# - outbound_addr=IPv4: Specify the outbound ipv4 address slirp should bind to. +# - outbound_addr6=INTERFACE: Specify the outbound interface slirp should bind to (ipv6 traffic only). +# - outbound_addr6=IPv6: Specify the outbound ipv6 address slirp should bind to. +# - port_handler=rootlesskit: Use rootlesskit for port forwarding. Default. +# Note: Rootlesskit changes the source IP address of incoming packets to a IP address in the container +# network namespace, usually `10.0.2.100`. If your application requires the real source IP address, +# e.g. web server logs, use the slirp4netns port handler. The rootlesskit port handler is also used for +# rootless containers when connected to user-defined networks. +# - port_handler=slirp4netns: Use the slirp4netns port forwarding, it is slower than rootlesskit but +# preserves the correct source IP address. This port handler cannot be used for user-defined networks. +# +#network_cmd_options = [] # Whether to use chroot instead of pivot_root in the runtime # @@ -629,7 +666,7 @@ default_sysctls = [ # Host directories to be mounted as volumes into the VM by default. # Environment variables like $HOME as well as complete paths are supported for -# the source and destination. An optional third field `:ro` can be used to +# the source and destination. An optional third field `:ro` can be used to # tell the container engines to mount the volume readonly. # # volumes = [ diff --git a/vendor/github.com/containers/common/pkg/config/default.go b/vendor/github.com/containers/common/pkg/config/default.go index 1a1da3fcd..62b348d6e 100644 --- a/vendor/github.com/containers/common/pkg/config/default.go +++ b/vendor/github.com/containers/common/pkg/config/default.go @@ -109,7 +109,6 @@ func parseSubnetPool(subnet string, size int) SubnetPool { Base: &nettypes.IPNet{IPNet: *n}, Size: size, } - } const ( @@ -123,11 +122,16 @@ const ( CgroupfsCgroupsManager = "cgroupfs" // DefaultApparmorProfile specifies the default apparmor profile for the container. DefaultApparmorProfile = apparmor.Profile + // DefaultHostsFile is the default path to the hosts file + DefaultHostsFile = "/etc/hosts" // SystemdCgroupsManager represents systemd native cgroup manager SystemdCgroupsManager = "systemd" // DefaultLogSizeMax is the default value for the maximum log size // allowed for a container. Negative values mean that no limit is imposed. DefaultLogSizeMax = -1 + // DefaultEventsLogSize is the default value for the maximum events log size + // before rotation. + DefaultEventsLogSizeMax = uint64(1000000) // DefaultPidsLimit is the default value for maximum number of processes // allowed inside a container DefaultPidsLimit = 2048 @@ -156,7 +160,6 @@ const ( // DefaultConfig defines the default values from containers.conf func DefaultConfig() (*Config, error) { - defaultEngineConfig, err := defaultConfigFromMemory() if err != nil { return nil, err @@ -188,6 +191,7 @@ func DefaultConfig() (*Config, error) { Volumes: []string{}, Annotations: []string{}, ApparmorProfile: DefaultApparmorProfile, + BaseHostsFile: "", CgroupNS: cgroupNS, Cgroups: "enabled", DefaultCapabilities: DefaultCapabilities, @@ -263,6 +267,8 @@ func defaultConfigFromMemory() (*EngineConfig, error) { c.EventsLogFilePath = filepath.Join(c.TmpDir, "events", "events.log") + c.EventsLogFileMaxSize = eventsLogMaxSize(DefaultEventsLogSizeMax) + c.CompatAPIEnforceDockerHub = true if path, ok := os.LookupEnv("CONTAINERS_STORAGE_CONF"); ok { @@ -296,9 +302,6 @@ func defaultConfigFromMemory() (*EngineConfig, error) { c.ServiceTimeout = uint(5) c.StopTimeout = uint(10) c.ExitCommandDelay = uint(5 * 60) - c.NetworkCmdOptions = []string{ - "enable_ipv6=true", - } c.Remote = isRemote() c.OCIRuntimes = map[string][]string{ "crun": { @@ -399,10 +402,10 @@ func defaultTmpDir() (string, error) { } libpodRuntimeDir := filepath.Join(runtimeDir, "libpod") - if err := os.Mkdir(libpodRuntimeDir, 0700|os.ModeSticky); err != nil { + if err := os.Mkdir(libpodRuntimeDir, 0o700|os.ModeSticky); err != nil { if !os.IsExist(err) { return "", err - } else if err := os.Chmod(libpodRuntimeDir, 0700|os.ModeSticky); err != nil { + } else if err := os.Chmod(libpodRuntimeDir, 0o700|os.ModeSticky); err != nil { // The directory already exist, just set the sticky bit return "", errors.Wrap(err, "set sticky bit on") } @@ -466,6 +469,10 @@ func (c *Config) NetNS() string { return c.Containers.NetNS } +func (c EngineConfig) EventsLogMaxSize() uint64 { + return uint64(c.EventsLogFileMaxSize) +} + // SecurityOptions returns the default security options func (c *Config) SecurityOptions() []string { securityOpts := []string{} diff --git a/vendor/github.com/containers/common/pkg/config/systemd.go b/vendor/github.com/containers/common/pkg/config/systemd.go index f17a84304..03d19a12f 100644 --- a/vendor/github.com/containers/common/pkg/config/systemd.go +++ b/vendor/github.com/containers/common/pkg/config/systemd.go @@ -58,7 +58,6 @@ func useSystemd() bool { val := strings.TrimSuffix(string(dat), "\n") usesSystemd = (val == "systemd") } - return }) return usesSystemd } @@ -82,7 +81,6 @@ func useJournald() bool { } } } - return }) return usesJournald } diff --git a/vendor/github.com/containers/common/pkg/manifests/manifests.go b/vendor/github.com/containers/common/pkg/manifests/manifests.go index 5c2836893..75ffac06c 100644 --- a/vendor/github.com/containers/common/pkg/manifests/manifests.go +++ b/vendor/github.com/containers/common/pkg/manifests/manifests.go @@ -16,31 +16,22 @@ import ( type List interface { AddInstance(manifestDigest digest.Digest, manifestSize int64, manifestType, os, architecture, osVersion string, osFeatures []string, variant string, features []string, annotations []string) error Remove(instanceDigest digest.Digest) error - SetURLs(instanceDigest digest.Digest, urls []string) error URLs(instanceDigest digest.Digest) ([]string, error) - SetAnnotations(instanceDigest *digest.Digest, annotations map[string]string) error Annotations(instanceDigest *digest.Digest) (map[string]string, error) - SetOS(instanceDigest digest.Digest, os string) error OS(instanceDigest digest.Digest) (string, error) - SetArchitecture(instanceDigest digest.Digest, arch string) error Architecture(instanceDigest digest.Digest) (string, error) - SetOSVersion(instanceDigest digest.Digest, osVersion string) error OSVersion(instanceDigest digest.Digest) (string, error) - SetVariant(instanceDigest digest.Digest, variant string) error Variant(instanceDigest digest.Digest) (string, error) - SetFeatures(instanceDigest digest.Digest, features []string) error Features(instanceDigest digest.Digest) ([]string, error) - SetOSFeatures(instanceDigest digest.Digest, osFeatures []string) error OSFeatures(instanceDigest digest.Digest) ([]string, error) - Serialize(mimeType string) ([]byte, error) Instances() []digest.Digest OCIv1() *v1.Index @@ -81,7 +72,7 @@ func Create() List { // AddInstance adds an entry for the specified manifest digest, with assorted // additional information specified in parameters, to the list or index. -func (l *list) AddInstance(manifestDigest digest.Digest, manifestSize int64, manifestType, osName, architecture, osVersion string, osFeatures []string, variant string, features []string, annotations []string) error { +func (l *list) AddInstance(manifestDigest digest.Digest, manifestSize int64, manifestType, osName, architecture, osVersion string, osFeatures []string, variant string, features, annotations []string) error { if err := l.Remove(manifestDigest); err != nil && !os.IsNotExist(errors.Cause(err)) { return err } @@ -451,38 +442,37 @@ func (l *list) preferOCI() bool { // Serialize encodes the list using the specified format, or by selecting one // which it thinks is appropriate. func (l *list) Serialize(mimeType string) ([]byte, error) { - var manifestBytes []byte + var ( + res []byte + err error + ) switch mimeType { case "": if l.preferOCI() { - manifest, err := json.Marshal(&l.oci) + res, err = json.Marshal(&l.oci) if err != nil { return nil, errors.Wrapf(err, "error marshalling OCI image index") } - manifestBytes = manifest } else { - manifest, err := json.Marshal(&l.docker) + res, err = json.Marshal(&l.docker) if err != nil { return nil, errors.Wrapf(err, "error marshalling Docker manifest list") } - manifestBytes = manifest } case v1.MediaTypeImageIndex: - manifest, err := json.Marshal(&l.oci) + res, err = json.Marshal(&l.oci) if err != nil { return nil, errors.Wrapf(err, "error marshalling OCI image index") } - manifestBytes = manifest case manifest.DockerV2ListMediaType: - manifest, err := json.Marshal(&l.docker) + res, err = json.Marshal(&l.docker) if err != nil { return nil, errors.Wrapf(err, "error marshalling Docker manifest list") } - manifestBytes = manifest default: return nil, errors.Wrapf(ErrManifestTypeNotSupported, "serializing list to type %q not implemented", mimeType) } - return manifestBytes, nil + return res, nil } // Instances returns the list of image instances mentioned in this list. diff --git a/vendor/github.com/containers/common/pkg/netns/netns_linux.go b/vendor/github.com/containers/common/pkg/netns/netns_linux.go index 9f85e910d..b3459b213 100644 --- a/vendor/github.com/containers/common/pkg/netns/netns_linux.go +++ b/vendor/github.com/containers/common/pkg/netns/netns_linux.go @@ -71,7 +71,7 @@ func NewNSWithName(name string) (ns.NetNS, error) { // Create the directory for mounting network namespaces // This needs to be a shared mountpoint in case it is mounted in to // other namespaces (containers) - err = os.MkdirAll(nsRunDir, 0755) + err = os.MkdirAll(nsRunDir, 0o755) if err != nil { return nil, err } diff --git a/vendor/github.com/containers/common/pkg/parse/parse.go b/vendor/github.com/containers/common/pkg/parse/parse.go index e73d44287..6c4958cc2 100644 --- a/vendor/github.com/containers/common/pkg/parse/parse.go +++ b/vendor/github.com/containers/common/pkg/parse/parse.go @@ -141,7 +141,7 @@ func Device(device string) (src, dest, permissions string, err error) { // isValidDeviceMode checks if the mode for device is valid or not. // isValid mode is a composition of r (read), w (write), and m (mknod). func isValidDeviceMode(mode string) bool { - var legalDeviceMode = map[rune]bool{ + legalDeviceMode := map[rune]bool{ 'r': true, 'w': true, 'm': true, diff --git a/vendor/github.com/containers/common/pkg/report/template.go b/vendor/github.com/containers/common/pkg/report/template.go index 95c04424d..29963099e 100644 --- a/vendor/github.com/containers/common/pkg/report/template.go +++ b/vendor/github.com/containers/common/pkg/report/template.go @@ -40,14 +40,14 @@ var DefaultFuncs = FuncMap{ buf := new(bytes.Buffer) enc := json.NewEncoder(buf) enc.SetEscapeHTML(false) - enc.Encode(v) + _ = enc.Encode(v) // Remove the trailing new line added by the encoder return strings.TrimSpace(buf.String()) }, "lower": strings.ToLower, "pad": padWithSpace, "split": strings.Split, - "title": strings.Title, + "title": strings.Title, //nolint:staticcheck "truncate": truncateWithLength, "upper": strings.ToUpper, } diff --git a/vendor/github.com/containers/common/pkg/seccomp/seccomp_linux.go b/vendor/github.com/containers/common/pkg/seccomp/seccomp_linux.go index d2498747c..f7adde8ab 100644 --- a/vendor/github.com/containers/common/pkg/seccomp/seccomp_linux.go +++ b/vendor/github.com/containers/common/pkg/seccomp/seccomp_linux.go @@ -112,7 +112,7 @@ func setupSeccomp(config *Seccomp, rs *specs.Spec) (*specs.LinuxSeccomp, error) newConfig := &specs.LinuxSeccomp{} var arch string - var native, err = libseccomp.GetNativeArch() + native, err := libseccomp.GetNativeArch() if err == nil { arch = native.String() } diff --git a/vendor/github.com/containers/common/pkg/secrets/filedriver/filedriver.go b/vendor/github.com/containers/common/pkg/secrets/filedriver/filedriver.go index 80fcf5458..e0c275851 100644 --- a/vendor/github.com/containers/common/pkg/secrets/filedriver/filedriver.go +++ b/vendor/github.com/containers/common/pkg/secrets/filedriver/filedriver.go @@ -34,7 +34,7 @@ func NewDriver(rootPath string) (*Driver, error) { fileDriver := new(Driver) fileDriver.secretsDataFilePath = filepath.Join(rootPath, secretsDataFile) // the lockfile functions require that the rootPath dir is executable - if err := os.MkdirAll(rootPath, 0700); err != nil { + if err := os.MkdirAll(rootPath, 0o700); err != nil { return nil, err } @@ -95,7 +95,7 @@ func (d *Driver) Store(id string, data []byte) error { if err != nil { return err } - err = ioutil.WriteFile(d.secretsDataFilePath, marshalled, 0600) + err = ioutil.WriteFile(d.secretsDataFilePath, marshalled, 0o600) if err != nil { return err } @@ -119,7 +119,7 @@ func (d *Driver) Delete(id string) error { if err != nil { return err } - err = ioutil.WriteFile(d.secretsDataFilePath, marshalled, 0600) + err = ioutil.WriteFile(d.secretsDataFilePath, marshalled, 0o600) if err != nil { return err } diff --git a/vendor/github.com/containers/common/pkg/secrets/secrets.go b/vendor/github.com/containers/common/pkg/secrets/secrets.go index aea983cb1..4a04e2b2f 100644 --- a/vendor/github.com/containers/common/pkg/secrets/secrets.go +++ b/vendor/github.com/containers/common/pkg/secrets/secrets.go @@ -102,7 +102,7 @@ func NewManager(rootPath string) (*SecretsManager, error) { return nil, errors.Wrapf(errInvalidPath, "path must be absolute: %s", rootPath) } // the lockfile functions require that the rootPath dir is executable - if err := os.MkdirAll(rootPath, 0700); err != nil { + if err := os.MkdirAll(rootPath, 0o700); err != nil { return nil, err } @@ -237,7 +237,6 @@ func (s *SecretsManager) List() ([]Secret, error) { var ls []Secret for _, v := range secrets { ls = append(ls, v) - } return ls, nil } diff --git a/vendor/github.com/containers/common/pkg/secrets/secretsdb.go b/vendor/github.com/containers/common/pkg/secrets/secretsdb.go index 0c4929995..4d2ca0fca 100644 --- a/vendor/github.com/containers/common/pkg/secrets/secretsdb.go +++ b/vendor/github.com/containers/common/pkg/secrets/secretsdb.go @@ -177,7 +177,7 @@ func (s *SecretsManager) store(entry *Secret) error { if err != nil { return err } - err = ioutil.WriteFile(s.secretsDBPath, marshalled, 0600) + err = ioutil.WriteFile(s.secretsDBPath, marshalled, 0o600) if err != nil { return err } @@ -203,7 +203,7 @@ func (s *SecretsManager) delete(nameOrID string) error { if err != nil { return err } - err = ioutil.WriteFile(s.secretsDBPath, marshalled, 0600) + err = ioutil.WriteFile(s.secretsDBPath, marshalled, 0o600) if err != nil { return err } diff --git a/vendor/github.com/containers/common/pkg/secrets/shelldriver/shelldriver.go b/vendor/github.com/containers/common/pkg/secrets/shelldriver/shelldriver.go index 846bd5c17..8eac200f7 100644 --- a/vendor/github.com/containers/common/pkg/secrets/shelldriver/shelldriver.go +++ b/vendor/github.com/containers/common/pkg/secrets/shelldriver/shelldriver.go @@ -3,12 +3,12 @@ package shelldriver import ( "bytes" "context" + "fmt" "os" "os/exec" "sort" "strings" - "github.com/mitchellh/mapstructure" "github.com/pkg/errors" ) @@ -27,22 +27,33 @@ var ( type driverConfig struct { // DeleteCommand contains a shell command that deletes a secret. // The secret id is provided as environment variable SECRET_ID - DeleteCommand string `mapstructure:"delete"` + DeleteCommand string // ListCommand contains a shell command that lists all secrets. // The output is expected to be one id per line - ListCommand string `mapstructure:"list"` + ListCommand string // LookupCommand contains a shell command that retrieves a secret. // The secret id is provided as environment variable SECRET_ID - LookupCommand string `mapstructure:"lookup"` + LookupCommand string // StoreCommand contains a shell command that stores a secret. // The secret id is provided as environment variable SECRET_ID // The secret value itself is provided over stdin - StoreCommand string `mapstructure:"store"` + StoreCommand string } func (cfg *driverConfig) ParseOpts(opts map[string]string) error { - if err := mapstructure.Decode(opts, cfg); err != nil { - return err + for key, value := range opts { + switch key { + case "delete": + cfg.DeleteCommand = value + case "list": + cfg.ListCommand = value + case "lookup": + cfg.LookupCommand = value + case "store": + cfg.StoreCommand = value + default: + return fmt.Errorf("invalid shell driver option: %q", key) + } } if cfg.DeleteCommand == "" || cfg.ListCommand == "" || diff --git a/vendor/github.com/containers/common/pkg/subscriptions/subscriptions.go b/vendor/github.com/containers/common/pkg/subscriptions/subscriptions.go index 3c0d2b237..4410292a7 100644 --- a/vendor/github.com/containers/common/pkg/subscriptions/subscriptions.go +++ b/vendor/github.com/containers/common/pkg/subscriptions/subscriptions.go @@ -262,7 +262,6 @@ func addSubscriptionsFromMountsFile(filePath, mountLabel, containerRunDir string data, err := readFileOrDir("", hostDirOrFile, mode.Perm()) if err != nil { return nil, err - } for _, s := range data { if err := os.MkdirAll(filepath.Dir(ctrDirOrFileOnHost), s.dirMode); err != nil { @@ -313,7 +312,7 @@ func addFIPSModeSubscription(mounts *[]rspec.Mount, containerRunDir, mountPoint, subscriptionsDir := "/run/secrets" ctrDirOnHost := filepath.Join(containerRunDir, subscriptionsDir) if _, err := os.Stat(ctrDirOnHost); os.IsNotExist(err) { - if err = idtools.MkdirAllAs(ctrDirOnHost, 0755, uid, gid); err != nil { //nolint + if err = idtools.MkdirAllAs(ctrDirOnHost, 0o755, uid, gid); err != nil { //nolint return err } if err = label.Relabel(ctrDirOnHost, mountLabel, false); err != nil { diff --git a/vendor/github.com/containers/common/pkg/sysinfo/sysinfo_solaris.go b/vendor/github.com/containers/common/pkg/sysinfo/sysinfo_solaris.go index 801db8c80..af1c77d60 100644 --- a/vendor/github.com/containers/common/pkg/sysinfo/sysinfo_solaris.go +++ b/vendor/github.com/containers/common/pkg/sysinfo/sysinfo_solaris.go @@ -46,7 +46,7 @@ func IsCPUSharesAvailable() bool { // New returns a new SysInfo, using the filesystem to detect which features // the kernel supports. -//NOTE Solaris: If we change the below capabilities be sure +// NOTE Solaris: If we change the below capabilities be sure // to update verifyPlatformContainerSettings() in daemon_solaris.go func New(quiet bool) *SysInfo { sysInfo := &SysInfo{} @@ -64,7 +64,6 @@ func New(quiet bool) *SysInfo { // setCgroupMem reads the memory information for Solaris. func setCgroupMem(quiet bool) cgroupMemInfo { - return cgroupMemInfo{ MemoryLimit: true, SwapLimit: true, @@ -77,7 +76,6 @@ func setCgroupMem(quiet bool) cgroupMemInfo { // setCgroupCPU reads the cpu information for Solaris. func setCgroupCPU(quiet bool) cgroupCPUInfo { - return cgroupCPUInfo{ CPUShares: true, CPUCfsPeriod: false, @@ -89,7 +87,6 @@ func setCgroupCPU(quiet bool) cgroupCPUInfo { // blkio switches are not supported in Solaris. func setCgroupBlkioInfo(quiet bool) cgroupBlkioInfo { - return cgroupBlkioInfo{ BlkioWeight: false, BlkioWeightDevice: false, @@ -98,7 +95,6 @@ func setCgroupBlkioInfo(quiet bool) cgroupBlkioInfo { // setCgroupCPUsetInfo reads the cpuset information for Solaris. func setCgroupCPUsetInfo(quiet bool) cgroupCpusetInfo { - return cgroupCpusetInfo{ Cpuset: true, Cpus: getCPUCount(), diff --git a/vendor/github.com/containers/common/pkg/timetype/timestamp.go b/vendor/github.com/containers/common/pkg/timetype/timestamp.go index ce2cb64f2..3cbfe4098 100644 --- a/vendor/github.com/containers/common/pkg/timetype/timestamp.go +++ b/vendor/github.com/containers/common/pkg/timetype/timestamp.go @@ -34,13 +34,14 @@ func GetTimestamp(value string, reference time.Time) (string, error) { // if the string has a Z or a + or three dashes use parse otherwise use parseinlocation parseInLocation := !(strings.ContainsAny(value, "zZ+") || strings.Count(value, "-") == 3) - if strings.Contains(value, ".") { // nolint:gocritic + switch { + case strings.Contains(value, "."): if parseInLocation { format = rFC3339NanoLocal } else { format = time.RFC3339Nano } - } else if strings.Contains(value, "T") { + case strings.Contains(value, "T"): // we want the number of colons in the T portion of the timestamp tcolons := strings.Count(value, ":") // if parseInLocation is off and we have a +/- zone offset (not Z) then @@ -68,9 +69,9 @@ func GetTimestamp(value string, reference time.Time) (string, error) { format = time.RFC3339 } } - } else if parseInLocation { + case parseInLocation: format = dateLocal - } else { + default: format = dateWithZone } @@ -112,7 +113,7 @@ func ParseTimestamps(value string, def int64) (secs, nanoSecs int64, err error) return parseTimestamp(value) } -func parseTimestamp(value string) (int64, int64, error) { // nolint:gocritic +func parseTimestamp(value string) (int64, int64, error) { sa := strings.SplitN(value, ".", 2) s, err := strconv.ParseInt(sa[0], 10, 64) if err != nil { diff --git a/vendor/github.com/containers/common/pkg/umask/umask_unix.go b/vendor/github.com/containers/common/pkg/umask/umask_unix.go index e59d7bea7..4f5527cb6 100644 --- a/vendor/github.com/containers/common/pkg/umask/umask_unix.go +++ b/vendor/github.com/containers/common/pkg/umask/umask_unix.go @@ -10,8 +10,8 @@ import ( ) func Check() { - oldUmask := syscall.Umask(0022) //nolint - if (oldUmask & ^0022) != 0 { + oldUmask := syscall.Umask(0o022) //nolint + if (oldUmask & ^0o022) != 0 { logrus.Debugf("umask value too restrictive. Forcing it to 022") } } diff --git a/vendor/github.com/containers/common/pkg/util/util_supported.go b/vendor/github.com/containers/common/pkg/util/util_supported.go index 284f3ffdd..35201f932 100644 --- a/vendor/github.com/containers/common/pkg/util/util_supported.go +++ b/vendor/github.com/containers/common/pkg/util/util_supported.go @@ -1,5 +1,5 @@ -//go:build linux || darwin -// +build linux darwin +//go:build linux || darwin || freebsd +// +build linux darwin freebsd package util @@ -23,7 +23,7 @@ var ( // isWriteableOnlyByOwner checks that the specified permission mask allows write // access only to the owner. func isWriteableOnlyByOwner(perm os.FileMode) bool { - return (perm & 0722) == 0700 + return (perm & 0o722) == 0o700 } // GetRuntimeDir returns the runtime directory @@ -46,7 +46,7 @@ func GetRuntimeDir() (string, error) { uid := fmt.Sprintf("%d", unshare.GetRootlessUID()) if runtimeDir == "" { tmpDir := filepath.Join("/run", "user", uid) - if err := os.MkdirAll(tmpDir, 0700); err != nil { + if err := os.MkdirAll(tmpDir, 0o700); err != nil { logrus.Debugf("unable to make temp dir: %v", err) } st, err := os.Stat(tmpDir) @@ -56,7 +56,7 @@ func GetRuntimeDir() (string, error) { } if runtimeDir == "" { tmpDir := filepath.Join(os.TempDir(), fmt.Sprintf("podman-run-%s", uid)) - if err := os.MkdirAll(tmpDir, 0700); err != nil { + if err := os.MkdirAll(tmpDir, 0o700); err != nil { logrus.Debugf("unable to make temp dir %v", err) } st, err := os.Stat(tmpDir) diff --git a/vendor/github.com/containers/image/v5/copy/copy.go b/vendor/github.com/containers/image/v5/copy/copy.go index b616e566c..644f82615 100644 --- a/vendor/github.com/containers/image/v5/copy/copy.go +++ b/vendor/github.com/containers/image/v5/copy/copy.go @@ -5,7 +5,6 @@ import ( "context" "fmt" "io" - "io/ioutil" "os" "reflect" "strings" @@ -199,7 +198,7 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef, return nil, err } - reportWriter := ioutil.Discard + reportWriter := io.Discard if options.ReportWriter != nil { reportWriter = options.ReportWriter @@ -232,7 +231,7 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef, // createProgressBar() will print a single line instead. progressOutput := reportWriter if !isTTY(reportWriter) { - progressOutput = ioutil.Discard + progressOutput = io.Discard } c := &copier{ @@ -1091,7 +1090,7 @@ func customPartialBlobDecorFunc(s decor.Statistics) string { } // createProgressBar creates a mpb.Bar in pool. Note that if the copier's reportWriter -// is ioutil.Discard, the progress bar's output will be discarded +// is io.Discard, the progress bar's output will be discarded // NOTE: Every progress bar created within a progress pool must either successfully // complete or be aborted, or pool.Wait() will hang. That is typically done // using "defer bar.Abort(false)", which must happen BEFORE pool.Wait() is called. @@ -1143,7 +1142,7 @@ func (c *copier) createProgressBar(pool *mpb.Progress, partial bool, info types. ), ) } - if c.progressOutput == ioutil.Discard { + if c.progressOutput == io.Discard { c.Printf("Copying %s %s\n", kind, info.Digest) } return bar @@ -1669,7 +1668,7 @@ func (c *copier) copyBlobFromStream(ctx context.Context, srcStream io.Reader, sr // sent there if we are not already at EOF. if getOriginalLayerCopyWriter != nil { logrus.Debugf("Consuming rest of the original blob to satisfy getOriginalLayerCopyWriter") - _, err := io.Copy(ioutil.Discard, originalLayerReader) + _, err := io.Copy(io.Discard, originalLayerReader) if err != nil { return types.BlobInfo{}, errors.Wrapf(err, "reading input blob %s", srcInfo.Digest) } diff --git a/vendor/github.com/containers/image/v5/directory/directory_dest.go b/vendor/github.com/containers/image/v5/directory/directory_dest.go index ea20e7c5e..3b135e68e 100644 --- a/vendor/github.com/containers/image/v5/directory/directory_dest.go +++ b/vendor/github.com/containers/image/v5/directory/directory_dest.go @@ -3,7 +3,6 @@ package directory import ( "context" "io" - "io/ioutil" "os" "path/filepath" "runtime" @@ -62,7 +61,7 @@ func newImageDestination(sys *types.SystemContext, ref dirReference) (types.Imag return nil, errors.Wrapf(err, "checking if path exists %q", d.ref.versionPath()) } if versionExists { - contents, err := ioutil.ReadFile(d.ref.versionPath()) + contents, err := os.ReadFile(d.ref.versionPath()) if err != nil { return nil, err } @@ -86,7 +85,7 @@ func newImageDestination(sys *types.SystemContext, ref dirReference) (types.Imag } } // create version file - err = ioutil.WriteFile(d.ref.versionPath(), []byte(version), 0644) + err = os.WriteFile(d.ref.versionPath(), []byte(version), 0644) if err != nil { return nil, errors.Wrapf(err, "creating version file %q", d.ref.versionPath()) } @@ -149,7 +148,7 @@ func (d *dirImageDestination) HasThreadSafePutBlob() bool { // to any other readers for download using the supplied digest. // If stream.Read() at any time, ESPECIALLY at end of input, returns an error, PutBlob MUST 1) fail, and 2) delete any data stored so far. func (d *dirImageDestination) PutBlob(ctx context.Context, stream io.Reader, inputInfo types.BlobInfo, cache types.BlobInfoCache, isConfig bool) (types.BlobInfo, error) { - blobFile, err := ioutil.TempFile(d.ref.path, "dir-put-blob") + blobFile, err := os.CreateTemp(d.ref.path, "dir-put-blob") if err != nil { return types.BlobInfo{}, err } @@ -232,7 +231,7 @@ func (d *dirImageDestination) TryReusingBlob(ctx context.Context, info types.Blo // If the destination is in principle available, refuses this manifest type (e.g. it does not recognize the schema), // but may accept a different manifest type, the returned error must be an ManifestTypeRejectedError. func (d *dirImageDestination) PutManifest(ctx context.Context, manifest []byte, instanceDigest *digest.Digest) error { - return ioutil.WriteFile(d.ref.manifestPath(instanceDigest), manifest, 0644) + return os.WriteFile(d.ref.manifestPath(instanceDigest), manifest, 0644) } // PutSignatures writes a set of signatures to the destination. @@ -240,7 +239,7 @@ func (d *dirImageDestination) PutManifest(ctx context.Context, manifest []byte, // (when the primary manifest is a manifest list); this should always be nil if the primary manifest is not a manifest list. func (d *dirImageDestination) PutSignatures(ctx context.Context, signatures [][]byte, instanceDigest *digest.Digest) error { for i, sig := range signatures { - if err := ioutil.WriteFile(d.ref.signaturePath(i, instanceDigest), sig, 0644); err != nil { + if err := os.WriteFile(d.ref.signaturePath(i, instanceDigest), sig, 0644); err != nil { return err } } @@ -272,7 +271,7 @@ func pathExists(path string) (bool, error) { // returns true if directory is empty func isDirEmpty(path string) (bool, error) { - files, err := ioutil.ReadDir(path) + files, err := os.ReadDir(path) if err != nil { return false, err } @@ -281,7 +280,7 @@ func isDirEmpty(path string) (bool, error) { // deletes the contents of a directory func removeDirContents(path string) error { - files, err := ioutil.ReadDir(path) + files, err := os.ReadDir(path) if err != nil { return err } diff --git a/vendor/github.com/containers/image/v5/directory/directory_src.go b/vendor/github.com/containers/image/v5/directory/directory_src.go index ad9129d40..8b509112a 100644 --- a/vendor/github.com/containers/image/v5/directory/directory_src.go +++ b/vendor/github.com/containers/image/v5/directory/directory_src.go @@ -3,7 +3,6 @@ package directory import ( "context" "io" - "io/ioutil" "os" "github.com/containers/image/v5/manifest" @@ -37,7 +36,7 @@ func (s *dirImageSource) Close() error { // If instanceDigest is not nil, it contains a digest of the specific manifest instance to retrieve (when the primary manifest is a manifest list); // this never happens if the primary manifest is not a manifest list (e.g. if the source never returns manifest lists). func (s *dirImageSource) GetManifest(ctx context.Context, instanceDigest *digest.Digest) ([]byte, string, error) { - m, err := ioutil.ReadFile(s.ref.manifestPath(instanceDigest)) + m, err := os.ReadFile(s.ref.manifestPath(instanceDigest)) if err != nil { return nil, "", err } @@ -71,7 +70,7 @@ func (s *dirImageSource) GetBlob(ctx context.Context, info types.BlobInfo, cache func (s *dirImageSource) GetSignatures(ctx context.Context, instanceDigest *digest.Digest) ([][]byte, error) { signatures := [][]byte{} for i := 0; ; i++ { - signature, err := ioutil.ReadFile(s.ref.signaturePath(i, instanceDigest)) + signature, err := os.ReadFile(s.ref.signaturePath(i, instanceDigest)) if err != nil { if os.IsNotExist(err) { break diff --git a/vendor/github.com/containers/image/v5/docker/docker_client.go b/vendor/github.com/containers/image/v5/docker/docker_client.go index 9837235d8..d984db718 100644 --- a/vendor/github.com/containers/image/v5/docker/docker_client.go +++ b/vendor/github.com/containers/image/v5/docker/docker_client.go @@ -7,7 +7,6 @@ import ( "encoding/json" "fmt" "io" - "io/ioutil" "net/http" "net/url" "os" @@ -654,7 +653,7 @@ func (c *dockerClient) getBearerTokenOAuth2(ctx context.Context, challenge chall params.Add("refresh_token", c.auth.IdentityToken) params.Add("client_id", "containers/image") - authReq.Body = ioutil.NopCloser(bytes.NewBufferString(params.Encode())) + authReq.Body = io.NopCloser(bytes.NewBufferString(params.Encode())) authReq.Header.Add("User-Agent", c.userAgent) authReq.Header.Add("Content-Type", "application/x-www-form-urlencoded") logrus.Debugf("%s %s", authReq.Method, authReq.URL.Redacted()) diff --git a/vendor/github.com/containers/image/v5/docker/docker_image_dest.go b/vendor/github.com/containers/image/v5/docker/docker_image_dest.go index e3275aa45..d02100cf8 100644 --- a/vendor/github.com/containers/image/v5/docker/docker_image_dest.go +++ b/vendor/github.com/containers/image/v5/docker/docker_image_dest.go @@ -7,7 +7,6 @@ import ( "encoding/json" "fmt" "io" - "io/ioutil" "net/http" "net/url" "os" @@ -592,7 +591,7 @@ func (d *dockerImageDestination) putOneSignature(url *url.URL, signature []byte) if err != nil { return err } - err = ioutil.WriteFile(url.Path, signature, 0644) + err = os.WriteFile(url.Path, signature, 0644) if err != nil { return err } diff --git a/vendor/github.com/containers/image/v5/docker/docker_image_src.go b/vendor/github.com/containers/image/v5/docker/docker_image_src.go index c08e5538a..c8e176f90 100644 --- a/vendor/github.com/containers/image/v5/docker/docker_image_src.go +++ b/vendor/github.com/containers/image/v5/docker/docker_image_src.go @@ -4,7 +4,6 @@ import ( "context" "fmt" "io" - "io/ioutil" "mime" "mime/multipart" "net/http" @@ -308,7 +307,7 @@ func splitHTTP200ResponseToPartial(streams chan io.ReadCloser, errs chan error, break } toSkip := c.Offset - currentOffset - if _, err := io.Copy(ioutil.Discard, io.LimitReader(body, int64(toSkip))); err != nil { + if _, err := io.Copy(io.Discard, io.LimitReader(body, int64(toSkip))); err != nil { errs <- err break } @@ -316,7 +315,7 @@ func splitHTTP200ResponseToPartial(streams chan io.ReadCloser, errs chan error, } s := signalCloseReader{ closed: make(chan interface{}), - stream: ioutil.NopCloser(io.LimitReader(body, int64(c.Length))), + stream: io.NopCloser(io.LimitReader(body, int64(c.Length))), consumeStream: true, } streams <- s @@ -515,7 +514,7 @@ func (s *dockerImageSource) getOneSignature(ctx context.Context, url *url.URL) ( switch url.Scheme { case "file": logrus.Debugf("Reading %s", url.Path) - sig, err := ioutil.ReadFile(url.Path) + sig, err := os.ReadFile(url.Path) if err != nil { if os.IsNotExist(err) { return nil, true, nil @@ -765,7 +764,7 @@ func (s signalCloseReader) Read(p []byte) (int, error) { func (s signalCloseReader) Close() error { defer close(s.closed) if s.consumeStream { - if _, err := io.Copy(ioutil.Discard, s.stream); err != nil { + if _, err := io.Copy(io.Discard, s.stream); err != nil { s.stream.Close() return err } diff --git a/vendor/github.com/containers/image/v5/docker/internal/tarfile/reader.go b/vendor/github.com/containers/image/v5/docker/internal/tarfile/reader.go index 6164ceb66..c77c002d1 100644 --- a/vendor/github.com/containers/image/v5/docker/internal/tarfile/reader.go +++ b/vendor/github.com/containers/image/v5/docker/internal/tarfile/reader.go @@ -4,7 +4,6 @@ import ( "archive/tar" "encoding/json" "io" - "io/ioutil" "os" "path" @@ -53,7 +52,7 @@ func NewReaderFromFile(sys *types.SystemContext, path string) (*Reader, error) { // The caller should call .Close() on the returned archive when done. func NewReaderFromStream(sys *types.SystemContext, inputStream io.Reader) (*Reader, error) { // Save inputStream to a temporary file - tarCopyFile, err := ioutil.TempFile(tmpdir.TemporaryDirectoryForBigFiles(sys), "docker-tar") + tarCopyFile, err := os.CreateTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "docker-tar") if err != nil { return nil, errors.Wrap(err, "creating temporary file") } diff --git a/vendor/github.com/containers/image/v5/docker/internal/tarfile/src.go b/vendor/github.com/containers/image/v5/docker/internal/tarfile/src.go index b8d84d245..8e9be17c1 100644 --- a/vendor/github.com/containers/image/v5/docker/internal/tarfile/src.go +++ b/vendor/github.com/containers/image/v5/docker/internal/tarfile/src.go @@ -6,7 +6,6 @@ import ( "context" "encoding/json" "io" - "io/ioutil" "os" "path" "sync" @@ -170,7 +169,7 @@ func (s *Source) prepareLayerData(tarManifest *ManifestItem, parsedConfig *manif uncompressedSize := h.Size if isCompressed { - uncompressedSize, err = io.Copy(ioutil.Discard, uncompressedStream) + uncompressedSize, err = io.Copy(io.Discard, uncompressedStream) if err != nil { return nil, errors.Wrapf(err, "reading %s to find its size", layerPath) } @@ -263,7 +262,7 @@ func (s *Source) GetBlob(ctx context.Context, info types.BlobInfo, cache types.B } if info.Digest == s.configDigest { // FIXME? Implement a more general algorithm matching instead of assuming sha256. - return ioutil.NopCloser(bytes.NewReader(s.configBytes)), int64(len(s.configBytes)), nil + return io.NopCloser(bytes.NewReader(s.configBytes)), int64(len(s.configBytes)), nil } if li, ok := s.knownLayers[info.Digest]; ok { // diffID is a digest of the uncompressed tarball, diff --git a/vendor/github.com/containers/image/v5/docker/lookaside.go b/vendor/github.com/containers/image/v5/docker/lookaside.go index 22d84931c..d0a3f1be0 100644 --- a/vendor/github.com/containers/image/v5/docker/lookaside.go +++ b/vendor/github.com/containers/image/v5/docker/lookaside.go @@ -2,7 +2,6 @@ package docker import ( "fmt" - "io/ioutil" "net/url" "os" "path" @@ -146,7 +145,7 @@ func loadAndMergeConfig(dirPath string) (*registryConfiguration, error) { continue } configPath := filepath.Join(dirPath, configName) - configBytes, err := ioutil.ReadFile(configPath) + configBytes, err := os.ReadFile(configPath) if err != nil { return nil, err } diff --git a/vendor/github.com/containers/image/v5/internal/iolimits/iolimits.go b/vendor/github.com/containers/image/v5/internal/iolimits/iolimits.go index 3fed1995c..49fa410e9 100644 --- a/vendor/github.com/containers/image/v5/internal/iolimits/iolimits.go +++ b/vendor/github.com/containers/image/v5/internal/iolimits/iolimits.go @@ -2,7 +2,6 @@ package iolimits import ( "io" - "io/ioutil" "github.com/pkg/errors" ) @@ -47,7 +46,7 @@ const ( func ReadAtMost(reader io.Reader, limit int) ([]byte, error) { limitedReader := io.LimitReader(reader, int64(limit+1)) - res, err := ioutil.ReadAll(limitedReader) + res, err := io.ReadAll(limitedReader) if err != nil { return nil, err } diff --git a/vendor/github.com/containers/image/v5/internal/streamdigest/stream_digest.go b/vendor/github.com/containers/image/v5/internal/streamdigest/stream_digest.go index 306220585..84bb656ac 100644 --- a/vendor/github.com/containers/image/v5/internal/streamdigest/stream_digest.go +++ b/vendor/github.com/containers/image/v5/internal/streamdigest/stream_digest.go @@ -3,7 +3,6 @@ package streamdigest import ( "fmt" "io" - "io/ioutil" "os" "github.com/containers/image/v5/internal/putblobdigest" @@ -16,7 +15,7 @@ import ( // It is the caller's responsibility to call the cleanup function, which closes and removes the temporary file. // If an error occurs, inputInfo is not modified. func ComputeBlobInfo(sys *types.SystemContext, stream io.Reader, inputInfo *types.BlobInfo) (io.Reader, func(), error) { - diskBlob, err := ioutil.TempFile(tmpdir.TemporaryDirectoryForBigFiles(sys), "stream-blob") + diskBlob, err := os.CreateTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "stream-blob") if err != nil { return nil, nil, fmt.Errorf("creating temporary on-disk layer: %w", err) } diff --git a/vendor/github.com/containers/image/v5/oci/archive/oci_transport.go b/vendor/github.com/containers/image/v5/oci/archive/oci_transport.go index 54d325d34..4fa912765 100644 --- a/vendor/github.com/containers/image/v5/oci/archive/oci_transport.go +++ b/vendor/github.com/containers/image/v5/oci/archive/oci_transport.go @@ -3,7 +3,6 @@ package archive import ( "context" "fmt" - "io/ioutil" "os" "strings" @@ -161,7 +160,7 @@ func (t *tempDirOCIRef) deleteTempDir() error { // createOCIRef creates the oci reference of the image // If SystemContext.BigFilesTemporaryDir not "", overrides the temporary directory to use for storing big files func createOCIRef(sys *types.SystemContext, image string) (tempDirOCIRef, error) { - dir, err := ioutil.TempDir(tmpdir.TemporaryDirectoryForBigFiles(sys), "oci") + dir, err := os.MkdirTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "oci") if err != nil { return tempDirOCIRef{}, errors.Wrapf(err, "creating temp directory") } diff --git a/vendor/github.com/containers/image/v5/oci/layout/oci_dest.go b/vendor/github.com/containers/image/v5/oci/layout/oci_dest.go index c8156cc3a..77e8fd876 100644 --- a/vendor/github.com/containers/image/v5/oci/layout/oci_dest.go +++ b/vendor/github.com/containers/image/v5/oci/layout/oci_dest.go @@ -4,7 +4,6 @@ import ( "context" "encoding/json" "io" - "io/ioutil" "os" "path/filepath" "runtime" @@ -124,7 +123,7 @@ func (d *ociImageDestination) HasThreadSafePutBlob() bool { // to any other readers for download using the supplied digest. // If stream.Read() at any time, ESPECIALLY at end of input, returns an error, PutBlob MUST 1) fail, and 2) delete any data stored so far. func (d *ociImageDestination) PutBlob(ctx context.Context, stream io.Reader, inputInfo types.BlobInfo, cache types.BlobInfoCache, isConfig bool) (types.BlobInfo, error) { - blobFile, err := ioutil.TempFile(d.ref.dir, "oci-put-blob") + blobFile, err := os.CreateTemp(d.ref.dir, "oci-put-blob") if err != nil { return types.BlobInfo{}, err } @@ -238,7 +237,7 @@ func (d *ociImageDestination) PutManifest(ctx context.Context, m []byte, instanc if err := ensureParentDirectoryExists(blobPath); err != nil { return err } - if err := ioutil.WriteFile(blobPath, m, 0644); err != nil { + if err := os.WriteFile(blobPath, m, 0644); err != nil { return err } @@ -309,14 +308,14 @@ func (d *ociImageDestination) PutSignatures(ctx context.Context, signatures [][] // - Uploaded data MAY be visible to others before Commit() is called // - Uploaded data MAY be removed or MAY remain around if Close() is called without Commit() (i.e. rollback is allowed but not guaranteed) func (d *ociImageDestination) Commit(context.Context, types.UnparsedImage) error { - if err := ioutil.WriteFile(d.ref.ociLayoutPath(), []byte(`{"imageLayoutVersion": "1.0.0"}`), 0644); err != nil { + if err := os.WriteFile(d.ref.ociLayoutPath(), []byte(`{"imageLayoutVersion": "1.0.0"}`), 0644); err != nil { return err } indexJSON, err := json.Marshal(d.index) if err != nil { return err } - return ioutil.WriteFile(d.ref.indexPath(), indexJSON, 0644) + return os.WriteFile(d.ref.indexPath(), indexJSON, 0644) } func ensureDirectoryExists(path string) error { diff --git a/vendor/github.com/containers/image/v5/oci/layout/oci_src.go b/vendor/github.com/containers/image/v5/oci/layout/oci_src.go index 9d8ab689b..8973f461c 100644 --- a/vendor/github.com/containers/image/v5/oci/layout/oci_src.go +++ b/vendor/github.com/containers/image/v5/oci/layout/oci_src.go @@ -3,7 +3,6 @@ package layout import ( "context" "io" - "io/ioutil" "net/http" "net/url" "os" @@ -93,7 +92,7 @@ func (s *ociImageSource) GetManifest(ctx context.Context, instanceDigest *digest return nil, "", err } - m, err := ioutil.ReadFile(manifestPath) + m, err := os.ReadFile(manifestPath) if err != nil { return nil, "", err } diff --git a/vendor/github.com/containers/image/v5/openshift/openshift-copies.go b/vendor/github.com/containers/image/v5/openshift/openshift-copies.go index 4ffbced6b..a6473ae68 100644 --- a/vendor/github.com/containers/image/v5/openshift/openshift-copies.go +++ b/vendor/github.com/containers/image/v5/openshift/openshift-copies.go @@ -5,7 +5,6 @@ import ( "crypto/x509" "encoding/json" "fmt" - "io/ioutil" "net" "net/http" "net/url" @@ -625,7 +624,7 @@ func (rules *clientConfigLoadingRules) Load() (*clientcmdConfig, error) { // loadFromFile is a modified copy of k8s.io/kubernetes/pkg/client/unversioned/clientcmd.LoadFromFile // LoadFromFile takes a filename and deserializes the contents into Config object func loadFromFile(filename string) (*clientcmdConfig, error) { - kubeconfigBytes, err := ioutil.ReadFile(filename) + kubeconfigBytes, err := os.ReadFile(filename) if err != nil { return nil, err } @@ -1013,7 +1012,7 @@ func dataFromSliceOrFile(data []byte, file string) ([]byte, error) { return data, nil } if len(file) > 0 { - fileData, err := ioutil.ReadFile(file) + fileData, err := os.ReadFile(file) if err != nil { return []byte{}, err } diff --git a/vendor/github.com/containers/image/v5/ostree/ostree_dest.go b/vendor/github.com/containers/image/v5/ostree/ostree_dest.go index 3eb2a2cba..011118fa5 100644 --- a/vendor/github.com/containers/image/v5/ostree/ostree_dest.go +++ b/vendor/github.com/containers/image/v5/ostree/ostree_dest.go @@ -10,7 +10,6 @@ import ( "encoding/json" "fmt" "io" - "io/ioutil" "os" "os/exec" "path/filepath" @@ -148,7 +147,7 @@ func (d *ostreeImageDestination) HasThreadSafePutBlob() bool { // to any other readers for download using the supplied digest. // If stream.Read() at any time, ESPECIALLY at end of input, returns an error, PutBlob MUST 1) fail, and 2) delete any data stored so far. func (d *ostreeImageDestination) PutBlob(ctx context.Context, stream io.Reader, inputInfo types.BlobInfo, cache types.BlobInfoCache, isConfig bool) (types.BlobInfo, error) { - tmpDir, err := ioutil.TempDir(d.tmpDirPath, "blob") + tmpDir, err := os.MkdirTemp(d.tmpDirPath, "blob") if err != nil { return types.BlobInfo{}, err } @@ -180,20 +179,24 @@ func (d *ostreeImageDestination) PutBlob(ctx context.Context, stream io.Reader, } func fixFiles(selinuxHnd *C.struct_selabel_handle, root string, dir string, usermode bool) error { - entries, err := ioutil.ReadDir(dir) + entries, err := os.ReadDir(dir) if err != nil { return err } - for _, info := range entries { - fullpath := filepath.Join(dir, info.Name()) - if info.Mode()&(os.ModeNamedPipe|os.ModeSocket|os.ModeDevice) != 0 { + for _, entry := range entries { + fullpath := filepath.Join(dir, entry.Name()) + if entry.Type()&(os.ModeNamedPipe|os.ModeSocket|os.ModeDevice) != 0 { if err := os.Remove(fullpath); err != nil { return err } continue } + info, err := entry.Info() + if err != nil { + return err + } if selinuxHnd != nil { relPath, err := filepath.Rel(root, fullpath) if err != nil { @@ -223,7 +226,7 @@ func fixFiles(selinuxHnd *C.struct_selabel_handle, root string, dir string, user } } - if info.IsDir() { + if entry.IsDir() { if usermode { if err := os.Chmod(fullpath, info.Mode()|0700); err != nil { return err @@ -233,7 +236,7 @@ func fixFiles(selinuxHnd *C.struct_selabel_handle, root string, dir string, user if err != nil { return err } - } else if usermode && (info.Mode().IsRegular()) { + } else if usermode && (entry.Type().IsRegular()) { if err := os.Chmod(fullpath, info.Mode()|0600); err != nil { return err } @@ -405,7 +408,7 @@ func (d *ostreeImageDestination) PutManifest(ctx context.Context, manifestBlob [ } d.digest = digest - return ioutil.WriteFile(manifestPath, manifestBlob, 0644) + return os.WriteFile(manifestPath, manifestBlob, 0644) } // PutSignatures writes signatures to the destination. @@ -423,7 +426,7 @@ func (d *ostreeImageDestination) PutSignatures(ctx context.Context, signatures [ for i, sig := range signatures { signaturePath := filepath.Join(d.tmpDirPath, d.ref.signaturePath(i)) - if err := ioutil.WriteFile(signaturePath, sig, 0644); err != nil { + if err := os.WriteFile(signaturePath, sig, 0644); err != nil { return err } } diff --git a/vendor/github.com/containers/image/v5/ostree/ostree_src.go b/vendor/github.com/containers/image/v5/ostree/ostree_src.go index d30c764a6..1e1f2be03 100644 --- a/vendor/github.com/containers/image/v5/ostree/ostree_src.go +++ b/vendor/github.com/containers/image/v5/ostree/ostree_src.go @@ -9,7 +9,6 @@ import ( "encoding/base64" "fmt" "io" - "io/ioutil" "strconv" "strings" "unsafe" @@ -369,7 +368,7 @@ func (s *ostreeImageSource) GetSignatures(ctx context.Context, instanceDigest *d } defer sigReader.Close() - sig, err := ioutil.ReadAll(sigReader) + sig, err := os.ReadAll(sigReader) if err != nil { return nil, err } diff --git a/vendor/github.com/containers/image/v5/pkg/blobcache/blobcache.go b/vendor/github.com/containers/image/v5/pkg/blobcache/blobcache.go index b67a83f33..8b22733ac 100644 --- a/vendor/github.com/containers/image/v5/pkg/blobcache/blobcache.go +++ b/vendor/github.com/containers/image/v5/pkg/blobcache/blobcache.go @@ -4,7 +4,6 @@ import ( "bytes" "context" "io" - "io/ioutil" "os" "path/filepath" "sync" @@ -196,7 +195,7 @@ func (s *blobCacheSource) Close() error { func (s *blobCacheSource) GetManifest(ctx context.Context, instanceDigest *digest.Digest) ([]byte, string, error) { if instanceDigest != nil { filename := filepath.Join(s.reference.directory, makeFilename(*instanceDigest, false)) - manifestBytes, err := ioutil.ReadFile(filename) + manifestBytes, err := os.ReadFile(filename) if err == nil { s.cacheHits++ return manifestBytes, manifest.GuessMIMEType(manifestBytes), nil @@ -280,10 +279,10 @@ func (s *blobCacheSource) LayerInfosForCopy(ctx context.Context, instanceDigest switch s.reference.compress { case types.Compress: alternate = blobFile + compressedNote - replaceDigest, err = ioutil.ReadFile(alternate) + replaceDigest, err = os.ReadFile(alternate) case types.Decompress: alternate = blobFile + decompressedNote - replaceDigest, err = ioutil.ReadFile(alternate) + replaceDigest, err = os.ReadFile(alternate) } if err == nil && digest.Digest(replaceDigest).Validate() == nil { alternate = filepath.Join(filepath.Dir(alternate), makeFilename(digest.Digest(replaceDigest), false)) @@ -373,7 +372,7 @@ func saveStream(wg *sync.WaitGroup, decompressReader io.ReadCloser, tempFile *os _, err3 = io.Copy(io.MultiWriter(tempFile, digester.Hash()), decompressed) } else { // Drain the pipe to keep from stalling the PutBlob() thread. - if _, err := io.Copy(ioutil.Discard, decompressReader); err != nil { + if _, err := io.Copy(io.Discard, decompressReader); err != nil { logrus.Debugf("error draining the pipe: %v", err) } } @@ -423,7 +422,7 @@ func (d *blobCacheDestination) PutBlob(ctx context.Context, stream io.Reader, in compression := archive.Uncompressed if inputInfo.Digest != "" { filename := filepath.Join(d.reference.directory, makeFilename(inputInfo.Digest, isConfig)) - tempfile, err = ioutil.TempFile(d.reference.directory, makeFilename(inputInfo.Digest, isConfig)) + tempfile, err = os.CreateTemp(d.reference.directory, makeFilename(inputInfo.Digest, isConfig)) if err == nil { stream = io.TeeReader(stream, tempfile) defer func() { @@ -457,7 +456,7 @@ func (d *blobCacheDestination) PutBlob(ctx context.Context, stream io.Reader, in if compression == archive.Gzip { // The stream is compressed, so create a file which we'll // use to store a decompressed copy. - decompressedTemp, err2 := ioutil.TempFile(d.reference.directory, makeFilename(inputInfo.Digest, isConfig)) + decompressedTemp, err2 := os.CreateTemp(d.reference.directory, makeFilename(inputInfo.Digest, isConfig)) if err2 != nil { logrus.Debugf("error while creating a temporary file under %q to hold decompressed blob %q: %v", d.reference.directory, inputInfo.Digest.String(), err2) decompressedTemp.Close() diff --git a/vendor/github.com/containers/image/v5/pkg/compression/compression.go b/vendor/github.com/containers/image/v5/pkg/compression/compression.go index c28e81792..34c90dd77 100644 --- a/vendor/github.com/containers/image/v5/pkg/compression/compression.go +++ b/vendor/github.com/containers/image/v5/pkg/compression/compression.go @@ -5,7 +5,6 @@ import ( "compress/bzip2" "fmt" "io" - "io/ioutil" "github.com/containers/image/v5/pkg/compression/internal" "github.com/containers/image/v5/pkg/compression/types" @@ -65,7 +64,7 @@ func GzipDecompressor(r io.Reader) (io.ReadCloser, error) { // Bzip2Decompressor is a DecompressorFunc for the bzip2 compression algorithm. func Bzip2Decompressor(r io.Reader) (io.ReadCloser, error) { - return ioutil.NopCloser(bzip2.NewReader(r)), nil + return io.NopCloser(bzip2.NewReader(r)), nil } // XzDecompressor is a DecompressorFunc for the xz compression algorithm. @@ -74,7 +73,7 @@ func XzDecompressor(r io.Reader) (io.ReadCloser, error) { if err != nil { return nil, err } - return ioutil.NopCloser(r), nil + return io.NopCloser(r), nil } // gzipCompressor is a CompressorFunc for the gzip compression algorithm. @@ -161,7 +160,7 @@ func AutoDecompress(stream io.Reader) (io.ReadCloser, bool, error) { return nil, false, errors.Wrapf(err, "initializing decompression") } } else { - res = ioutil.NopCloser(stream) + res = io.NopCloser(stream) } return res, decompressor != nil, nil } diff --git a/vendor/github.com/containers/image/v5/pkg/docker/config/config.go b/vendor/github.com/containers/image/v5/pkg/docker/config/config.go index 52734bead..d0bdd08e9 100644 --- a/vendor/github.com/containers/image/v5/pkg/docker/config/config.go +++ b/vendor/github.com/containers/image/v5/pkg/docker/config/config.go @@ -4,7 +4,6 @@ import ( "encoding/base64" "encoding/json" "fmt" - "io/ioutil" "os" "os/exec" "path/filepath" @@ -544,7 +543,7 @@ func getPathToAuthWithOS(sys *types.SystemContext, goOS string) (string, bool, e func readJSONFile(path string, legacyFormat bool) (dockerConfigFile, error) { var auths dockerConfigFile - raw, err := ioutil.ReadFile(path) + raw, err := os.ReadFile(path) if err != nil { if os.IsNotExist(err) { auths.AuthConfigs = map[string]dockerAuthConfig{} diff --git a/vendor/github.com/containers/image/v5/pkg/sysregistriesv2/system_registries_v2.go b/vendor/github.com/containers/image/v5/pkg/sysregistriesv2/system_registries_v2.go index c5df241b7..c1753c845 100644 --- a/vendor/github.com/containers/image/v5/pkg/sysregistriesv2/system_registries_v2.go +++ b/vendor/github.com/containers/image/v5/pkg/sysregistriesv2/system_registries_v2.go @@ -2,6 +2,7 @@ package sysregistriesv2 import ( "fmt" + "io/fs" "os" "path/filepath" "reflect" @@ -643,17 +644,17 @@ func dropInConfigs(wrapper configWrapper) ([]string, error) { dirPaths = append(dirPaths, wrapper.userConfigDirPath) } for _, dirPath := range dirPaths { - err := filepath.Walk(dirPath, + err := filepath.WalkDir(dirPath, // WalkFunc to read additional configs - func(path string, info os.FileInfo, err error) error { + func(path string, d fs.DirEntry, err error) error { switch { case err != nil: // return error (could be a permission problem) return err - case info == nil: + case d == nil: // this should only happen when err != nil but let's be sure return nil - case info.IsDir(): + case d.IsDir(): if path != dirPath { // make sure to not recurse into sub-directories return filepath.SkipDir diff --git a/vendor/github.com/containers/image/v5/pkg/tlsclientconfig/tlsclientconfig.go b/vendor/github.com/containers/image/v5/pkg/tlsclientconfig/tlsclientconfig.go index 7e2142b1f..c766417d0 100644 --- a/vendor/github.com/containers/image/v5/pkg/tlsclientconfig/tlsclientconfig.go +++ b/vendor/github.com/containers/image/v5/pkg/tlsclientconfig/tlsclientconfig.go @@ -2,7 +2,6 @@ package tlsclientconfig import ( "crypto/tls" - "io/ioutil" "net" "net/http" "os" @@ -19,7 +18,7 @@ import ( // SetupCertificates opens all .crt, .cert, and .key files in dir and appends / loads certs and key pairs as appropriate to tlsc func SetupCertificates(dir string, tlsc *tls.Config) error { logrus.Debugf("Looking for TLS certificates and private keys in %s", dir) - fs, err := ioutil.ReadDir(dir) + fs, err := os.ReadDir(dir) if err != nil { if os.IsNotExist(err) { return nil @@ -35,7 +34,7 @@ func SetupCertificates(dir string, tlsc *tls.Config) error { fullPath := filepath.Join(dir, f.Name()) if strings.HasSuffix(f.Name(), ".crt") { logrus.Debugf(" crt: %s", fullPath) - data, err := ioutil.ReadFile(fullPath) + data, err := os.ReadFile(fullPath) if err != nil { if os.IsNotExist(err) { // Dangling symbolic link? @@ -81,7 +80,7 @@ func SetupCertificates(dir string, tlsc *tls.Config) error { return nil } -func hasFile(files []os.FileInfo, name string) bool { +func hasFile(files []os.DirEntry, name string) bool { for _, f := range files { if f.Name() == name { return true diff --git a/vendor/github.com/containers/image/v5/sif/load.go b/vendor/github.com/containers/image/v5/sif/load.go index ba6d875ba..70758ad43 100644 --- a/vendor/github.com/containers/image/v5/sif/load.go +++ b/vendor/github.com/containers/image/v5/sif/load.go @@ -5,7 +5,6 @@ import ( "context" "fmt" "io" - "io/ioutil" "os" "os/exec" "path/filepath" @@ -103,7 +102,7 @@ func writeInjectedScript(extractedRootPath string, injectedScript []byte) error if err := os.MkdirAll(parentDirPath, 0755); err != nil { return fmt.Errorf("creating %s: %w", parentDirPath, err) } - if err := ioutil.WriteFile(filePath, injectedScript, 0755); err != nil { + if err := os.WriteFile(filePath, injectedScript, 0755); err != nil { return fmt.Errorf("writing %s to %s: %w", injectedScriptTargetPath, filePath, err) } return nil @@ -121,7 +120,7 @@ func createTarFromSIFInputs(ctx context.Context, tarPath, squashFSPath string, i conversionCommand := fmt.Sprintf("unsquashfs -d %s -f %s && tar --acls --xattrs -C %s -cpf %s ./", extractedRootPath, squashFSPath, extractedRootPath, tarPath) script := "#!/bin/sh\n" + conversionCommand + "\n" - if err := ioutil.WriteFile(scriptPath, []byte(script), 0755); err != nil { + if err := os.WriteFile(scriptPath, []byte(script), 0755); err != nil { return err } defer os.Remove(scriptPath) @@ -149,7 +148,7 @@ func createTarFromSIFInputs(ctx context.Context, tarPath, squashFSPath string, i // at start, and is exclusively used by the current process (i.e. it is safe // to use hard-coded relative paths within it). func convertSIFToElements(ctx context.Context, sifImage *sif.FileImage, tempDir string) (string, []string, error) { - // We could allocate unique names for all of these using ioutil.Temp*, but tempDir is exclusive, + // We could allocate unique names for all of these using os.{CreateTemp,MkdirTemp}, but tempDir is exclusive, // so we can just hard-code a set of unique values here. // We create and/or manage cleanup of these two paths. squashFSPath := filepath.Join(tempDir, "rootfs.squashfs") diff --git a/vendor/github.com/containers/image/v5/sif/src.go b/vendor/github.com/containers/image/v5/sif/src.go index ba95a469f..ccf125966 100644 --- a/vendor/github.com/containers/image/v5/sif/src.go +++ b/vendor/github.com/containers/image/v5/sif/src.go @@ -7,7 +7,6 @@ import ( "errors" "fmt" "io" - "io/ioutil" "os" "github.com/containers/image/v5/internal/tmpdir" @@ -65,7 +64,7 @@ func newImageSource(ctx context.Context, sys *types.SystemContext, ref sifRefere _ = sifImg.UnloadContainer() }() - workDir, err := ioutil.TempDir(tmpdir.TemporaryDirectoryForBigFiles(sys), "sif") + workDir, err := os.MkdirTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "sif") if err != nil { return nil, fmt.Errorf("creating temp directory: %w", err) } @@ -170,7 +169,7 @@ func (s *sifImageSource) HasThreadSafeGetBlob() bool { func (s *sifImageSource) GetBlob(ctx context.Context, info types.BlobInfo, cache types.BlobInfoCache) (io.ReadCloser, int64, error) { switch info.Digest { case s.configDigest: - return ioutil.NopCloser(bytes.NewBuffer(s.config)), int64(len(s.config)), nil + return io.NopCloser(bytes.NewBuffer(s.config)), int64(len(s.config)), nil case s.layerDigest: reader, err := os.Open(s.layerFile) if err != nil { diff --git a/vendor/github.com/containers/image/v5/signature/mechanism.go b/vendor/github.com/containers/image/v5/signature/mechanism.go index 961246147..249b5a1fe 100644 --- a/vendor/github.com/containers/image/v5/signature/mechanism.go +++ b/vendor/github.com/containers/image/v5/signature/mechanism.go @@ -6,7 +6,7 @@ import ( "bytes" "errors" "fmt" - "io/ioutil" + "io" "strings" // This code is used only to parse the data in an explicitly-untrusted @@ -82,7 +82,7 @@ func gpgUntrustedSignatureContents(untrustedSignature []byte) (untrustedContents if !md.IsSigned { return nil, "", errors.New("The input is not a signature") } - content, err := ioutil.ReadAll(md.UnverifiedBody) + content, err := io.ReadAll(md.UnverifiedBody) if err != nil { // Coverage: An error during reading the body can happen only if // 1) the message is encrypted, which is not our case (and we don’t give ReadMessage the key diff --git a/vendor/github.com/containers/image/v5/signature/mechanism_gpgme.go b/vendor/github.com/containers/image/v5/signature/mechanism_gpgme.go index c166fb32d..4c7968417 100644 --- a/vendor/github.com/containers/image/v5/signature/mechanism_gpgme.go +++ b/vendor/github.com/containers/image/v5/signature/mechanism_gpgme.go @@ -7,7 +7,6 @@ import ( "bytes" "errors" "fmt" - "io/ioutil" "os" "github.com/proglottis/gpgme" @@ -37,7 +36,7 @@ func newGPGSigningMechanismInDirectory(optionalDir string) (signingMechanismWith // of these keys. // The caller must call .Close() on the returned SigningMechanism. func newEphemeralGPGSigningMechanism(blob []byte) (signingMechanismWithPassphrase, []string, error) { - dir, err := ioutil.TempDir("", "containers-ephemeral-gpg-") + dir, err := os.MkdirTemp("", "containers-ephemeral-gpg-") if err != nil { return nil, nil, err } diff --git a/vendor/github.com/containers/image/v5/signature/mechanism_openpgp.go b/vendor/github.com/containers/image/v5/signature/mechanism_openpgp.go index ef4e70e7f..63cb7788b 100644 --- a/vendor/github.com/containers/image/v5/signature/mechanism_openpgp.go +++ b/vendor/github.com/containers/image/v5/signature/mechanism_openpgp.go @@ -7,7 +7,7 @@ import ( "bytes" "errors" "fmt" - "io/ioutil" + "io" "os" "path" "strings" @@ -44,7 +44,7 @@ func newGPGSigningMechanismInDirectory(optionalDir string) (signingMechanismWith } } - pubring, err := ioutil.ReadFile(path.Join(gpgHome, "pubring.gpg")) + pubring, err := os.ReadFile(path.Join(gpgHome, "pubring.gpg")) if err != nil { if !os.IsNotExist(err) { return nil, err @@ -130,7 +130,7 @@ func (m *openpgpSigningMechanism) Verify(unverifiedSignature []byte) (contents [ if !md.IsSigned { return nil, "", errors.New("not signed") } - content, err := ioutil.ReadAll(md.UnverifiedBody) + content, err := io.ReadAll(md.UnverifiedBody) if err != nil { // Coverage: md.UnverifiedBody.Read only fails if the body is encrypted // (and possibly also signed, but it _must_ be encrypted) and the signing diff --git a/vendor/github.com/containers/image/v5/signature/policy_config.go b/vendor/github.com/containers/image/v5/signature/policy_config.go index 82fbb68cb..bb91cae8c 100644 --- a/vendor/github.com/containers/image/v5/signature/policy_config.go +++ b/vendor/github.com/containers/image/v5/signature/policy_config.go @@ -16,7 +16,6 @@ package signature import ( "encoding/json" "fmt" - "io/ioutil" "os" "path/filepath" "regexp" @@ -80,7 +79,7 @@ func defaultPolicyPathWithHomeDir(sys *types.SystemContext, homeDir string) stri // NewPolicyFromFile returns a policy configured in the specified file. func NewPolicyFromFile(fileName string) (*Policy, error) { - contents, err := ioutil.ReadFile(fileName) + contents, err := os.ReadFile(fileName) if err != nil { return nil, err } diff --git a/vendor/github.com/containers/image/v5/signature/policy_eval_signedby.go b/vendor/github.com/containers/image/v5/signature/policy_eval_signedby.go index 26cca4759..65e825973 100644 --- a/vendor/github.com/containers/image/v5/signature/policy_eval_signedby.go +++ b/vendor/github.com/containers/image/v5/signature/policy_eval_signedby.go @@ -5,7 +5,7 @@ package signature import ( "context" "fmt" - "io/ioutil" + "os" "strings" "github.com/containers/image/v5/manifest" @@ -33,7 +33,7 @@ func (pr *prSignedBy) isSignatureAuthorAccepted(ctx context.Context, image types if pr.KeyData != nil { data = pr.KeyData } else { - d, err := ioutil.ReadFile(pr.KeyPath) + d, err := os.ReadFile(pr.KeyPath) if err != nil { return sarRejected, nil, err } diff --git a/vendor/github.com/containers/image/v5/storage/storage_image.go b/vendor/github.com/containers/image/v5/storage/storage_image.go index 08ae042ac..8071e3b32 100644 --- a/vendor/github.com/containers/image/v5/storage/storage_image.go +++ b/vendor/github.com/containers/image/v5/storage/storage_image.go @@ -10,7 +10,6 @@ import ( stderrors "errors" "fmt" "io" - "io/ioutil" "os" "path/filepath" "sync" @@ -155,7 +154,7 @@ func (s *storageImageSource) HasThreadSafeGetBlob() bool { // May update BlobInfoCache, preferably after it knows for certain that a blob truly exists at a specific location. func (s *storageImageSource) GetBlob(ctx context.Context, info types.BlobInfo, cache types.BlobInfoCache) (rc io.ReadCloser, n int64, err error) { if info.Digest == image.GzippedEmptyLayerDigest { - return ioutil.NopCloser(bytes.NewReader(image.GzippedEmptyLayer)), int64(len(image.GzippedEmptyLayer)), nil + return io.NopCloser(bytes.NewReader(image.GzippedEmptyLayer)), int64(len(image.GzippedEmptyLayer)), nil } // NOTE: the blob is first written to a temporary file and subsequently @@ -167,7 +166,7 @@ func (s *storageImageSource) GetBlob(ctx context.Context, info types.BlobInfo, c } defer rc.Close() - tmpFile, err := ioutil.TempFile(tmpdir.TemporaryDirectoryForBigFiles(s.systemContext), "") + tmpFile, err := os.CreateTemp(tmpdir.TemporaryDirectoryForBigFiles(s.systemContext), "") if err != nil { return nil, 0, err } @@ -210,7 +209,7 @@ func (s *storageImageSource) getBlobAndLayerID(info types.BlobInfo) (rc io.ReadC } r := bytes.NewReader(b) logrus.Debugf("exporting opaque data as blob %q", info.Digest.String()) - return ioutil.NopCloser(r), int64(r.Len()), "", nil + return io.NopCloser(r), int64(r.Len()), "", nil } // Step through the list of matching layers. Tests may want to verify that if we have multiple layers // which claim to have the same contents, that we actually do have multiple layers, otherwise we could @@ -395,7 +394,7 @@ func (s *storageImageSource) GetSignatures(ctx context.Context, instanceDigest * // newImageDestination sets us up to write a new image, caching blobs in a temporary directory until // it's time to Commit() the image func newImageDestination(sys *types.SystemContext, imageRef storageReference) (*storageImageDestination, error) { - directory, err := ioutil.TempDir(tmpdir.TemporaryDirectoryForBigFiles(sys), "storage") + directory, err := os.MkdirTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "storage") if err != nil { return nil, errors.Wrapf(err, "creating a temporary directory") } @@ -791,7 +790,7 @@ func (s *storageImageDestination) getConfigBlob(info types.BlobInfo) ([]byte, er } // Assume it's a file, since we're only calling this from a place that expects to read files. if filename, ok := s.filenames[info.Digest]; ok { - contents, err2 := ioutil.ReadFile(filename) + contents, err2 := os.ReadFile(filename) if err2 != nil { return nil, errors.Wrapf(err2, `reading blob from file %q`, filename) } @@ -1136,7 +1135,7 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t delete(dataBlobs, layerBlob.Digest) } for blob := range dataBlobs { - v, err := ioutil.ReadFile(s.filenames[blob]) + v, err := os.ReadFile(s.filenames[blob]) if err != nil { return errors.Wrapf(err, "copying non-layer blob %q to image", blob) } diff --git a/vendor/github.com/containers/image/v5/tarball/tarball_src.go b/vendor/github.com/containers/image/v5/tarball/tarball_src.go index 694ad17bd..aedfdf5de 100644 --- a/vendor/github.com/containers/image/v5/tarball/tarball_src.go +++ b/vendor/github.com/containers/image/v5/tarball/tarball_src.go @@ -6,7 +6,6 @@ import ( "encoding/json" "fmt" "io" - "io/ioutil" "os" "runtime" "strings" @@ -87,7 +86,7 @@ func (r *tarballReference) NewImageSource(ctx context.Context, sys *types.System uncompressed = nil } // TODO: This can take quite some time, and should ideally be cancellable using ctx.Done(). - n, err := io.Copy(ioutil.Discard, reader) + n, err := io.Copy(io.Discard, reader) if err != nil { return nil, fmt.Errorf("error reading %q: %v", filename, err) } @@ -217,14 +216,14 @@ func (is *tarballImageSource) HasThreadSafeGetBlob() bool { func (is *tarballImageSource) GetBlob(ctx context.Context, blobinfo types.BlobInfo, cache types.BlobInfoCache) (io.ReadCloser, int64, error) { // We should only be asked about things in the manifest. Maybe the configuration blob. if blobinfo.Digest == is.configID { - return ioutil.NopCloser(bytes.NewBuffer(is.config)), is.configSize, nil + return io.NopCloser(bytes.NewBuffer(is.config)), is.configSize, nil } // Maybe one of the layer blobs. for i := range is.blobIDs { if blobinfo.Digest == is.blobIDs[i] { // We want to read that layer: open the file or memory block and hand it back. if is.filenames[i] == "-" { - return ioutil.NopCloser(bytes.NewBuffer(is.reference.stdin)), int64(len(is.reference.stdin)), nil + return io.NopCloser(bytes.NewBuffer(is.reference.stdin)), int64(len(is.reference.stdin)), nil } reader, err := os.Open(is.filenames[i]) if err != nil { diff --git a/vendor/github.com/containers/image/v5/tarball/tarball_transport.go b/vendor/github.com/containers/image/v5/tarball/tarball_transport.go index d407c657f..63d835530 100644 --- a/vendor/github.com/containers/image/v5/tarball/tarball_transport.go +++ b/vendor/github.com/containers/image/v5/tarball/tarball_transport.go @@ -3,7 +3,7 @@ package tarball import ( "errors" "fmt" - "io/ioutil" + "io" "os" "strings" @@ -36,7 +36,7 @@ func (t *tarballTransport) ParseReference(reference string) (types.ImageReferenc filenames := strings.Split(reference, separator) for _, filename := range filenames { if filename == "-" { - stdin, err = ioutil.ReadAll(os.Stdin) + stdin, err = io.ReadAll(os.Stdin) if err != nil { return nil, fmt.Errorf("error buffering stdin: %v", err) } diff --git a/vendor/github.com/containers/storage/Makefile b/vendor/github.com/containers/storage/Makefile index d7ca0c1c4..2c1e4a185 100644 --- a/vendor/github.com/containers/storage/Makefile +++ b/vendor/github.com/containers/storage/Makefile @@ -69,44 +69,44 @@ local-cross: ## cross build the binaries for arm, darwin, and\nfreebsd done cross: ## cross build the binaries for arm, darwin, and\nfreebsd using VMs - $(RUNINVM) make local-$@ + $(RUNINVM) $(MAKE) local-$@ docs: install.tools ## build the docs on the host $(MAKE) -C docs docs gccgo: ## build using gccgo using VMs - $(RUNINVM) make local-$@ + $(RUNINVM) $(MAKE) local-$@ test: local-binary ## build the binaries and run the tests using VMs - $(RUNINVM) make local-binary local-cross local-test-unit local-test-integration + $(RUNINVM) $(MAKE) local-binary local-cross local-test-unit local-test-integration local-test-unit: local-binary ## run the unit tests on the host (requires\nsuperuser privileges) @$(GO) test $(MOD_VENDOR) $(BUILDFLAGS) $(TESTFLAGS) $(shell $(GO) list ./... | grep -v ^$(PACKAGE)/vendor) test-unit: local-binary ## run the unit tests using VMs - $(RUNINVM) make local-$@ + $(RUNINVM) $(MAKE) local-$@ local-test-integration: local-binary ## run the integration tests on the host (requires\nsuperuser privileges) @cd tests; ./test_runner.bash test-integration: local-binary ## run the integration tests using VMs - $(RUNINVM) make local-$@ + $(RUNINVM) $(MAKE) local-$@ local-validate: ## validate DCO and gofmt on the host @./hack/git-validation.sh @./hack/gofmt.sh validate: ## validate DCO, gofmt, ./pkg/ isolation, golint,\ngo vet and vendor using VMs - $(RUNINVM) make local-$@ + $(RUNINVM) $(MAKE) local-$@ install.tools: - make -C tests/tools + $(MAKE) -C tests/tools $(FFJSON): - make -C tests/tools + $(MAKE) -C tests/tools install.docs: docs - make -C docs install + $(MAKE) -C docs install install: install.docs diff --git a/vendor/github.com/containers/storage/drivers/driver_freebsd.go b/vendor/github.com/containers/storage/drivers/driver_freebsd.go index e1320ee07..79a591288 100644 --- a/vendor/github.com/containers/storage/drivers/driver_freebsd.go +++ b/vendor/github.com/containers/storage/drivers/driver_freebsd.go @@ -1,16 +1,45 @@ package graphdriver import ( + "fmt" "golang.org/x/sys/unix" + + "github.com/containers/storage/pkg/mount" +) + +const ( + // FsMagicZfs filesystem id for Zfs + FsMagicZfs = FsMagic(0x2fc12fc1) ) var ( // Slice of drivers that should be used in an order priority = []string{ "zfs", + "vfs", + } + + // FsNames maps filesystem id to name of the filesystem. + FsNames = map[FsMagic]string{ + FsMagicZfs: "zfs", } ) +// NewDefaultChecker returns a check that parses /proc/mountinfo to check +// if the specified path is mounted. +// No-op on FreeBSD. +func NewDefaultChecker() Checker { + return &defaultChecker{} +} + +type defaultChecker struct { +} + +func (c *defaultChecker) IsMounted(path string) bool { + m, _ := mount.Mounted(path) + return m +} + // Mounted checks if the given path is mounted as the fs type func Mounted(fsType FsMagic, mountPath string) (bool, error) { var buf unix.Statfs_t diff --git a/vendor/github.com/containers/storage/drivers/register/register_zfs.go b/vendor/github.com/containers/storage/drivers/register/register_zfs.go index c748468e5..4623e7f46 100644 --- a/vendor/github.com/containers/storage/drivers/register/register_zfs.go +++ b/vendor/github.com/containers/storage/drivers/register/register_zfs.go @@ -1,4 +1,4 @@ -// +build !exclude_graphdriver_zfs,linux !exclude_graphdriver_zfs,freebsd, solaris +// +build !exclude_graphdriver_zfs,linux !exclude_graphdriver_zfs,freebsd solaris package register diff --git a/vendor/github.com/containers/storage/drivers/zfs/zfs.go b/vendor/github.com/containers/storage/drivers/zfs/zfs.go index e034bf152..f29dc8f85 100644 --- a/vendor/github.com/containers/storage/drivers/zfs/zfs.go +++ b/vendor/github.com/containers/storage/drivers/zfs/zfs.go @@ -344,7 +344,7 @@ func (d *Driver) create(id, parent string, opts *graphdriver.CreateOpts) error { return errors.Wrap(err, "error creating zfs mount") } defer func() { - if err := unix.Unmount(mountpoint, unix.MNT_DETACH); err != nil { + if err := detachUnmount(mountpoint); err != nil { logrus.Warnf("Failed to unmount %s mount %s: %v", id, mountpoint, err) } }() @@ -483,7 +483,7 @@ func (d *Driver) Put(id string) error { logger.Debugf(`unmount("%s")`, mountpoint) - if err := unix.Unmount(mountpoint, unix.MNT_DETACH); err != nil { + if err := detachUnmount(mountpoint); err != nil { logger.Warnf("Failed to unmount %s mount %s: %v", id, mountpoint, err) } if err := unix.Rmdir(mountpoint); err != nil && !os.IsNotExist(err) { diff --git a/vendor/github.com/containers/storage/drivers/zfs/zfs_freebsd.go b/vendor/github.com/containers/storage/drivers/zfs/zfs_freebsd.go index bf6905159..fd98ad305 100644 --- a/vendor/github.com/containers/storage/drivers/zfs/zfs_freebsd.go +++ b/vendor/github.com/containers/storage/drivers/zfs/zfs_freebsd.go @@ -37,3 +37,8 @@ func getMountpoint(id string) string { return id[:maxlen] } + +func detachUnmount(mountpoint string) error { + // FreeBSD doesn't have an equivalent to MNT_DETACH + return unix.Unmount(mountpoint, 0) +} diff --git a/vendor/github.com/containers/storage/drivers/zfs/zfs_linux.go b/vendor/github.com/containers/storage/drivers/zfs/zfs_linux.go index edcb1da36..44c68f394 100644 --- a/vendor/github.com/containers/storage/drivers/zfs/zfs_linux.go +++ b/vendor/github.com/containers/storage/drivers/zfs/zfs_linux.go @@ -4,6 +4,7 @@ import ( graphdriver "github.com/containers/storage/drivers" "github.com/pkg/errors" "github.com/sirupsen/logrus" + "golang.org/x/sys/unix" ) func checkRootdirFs(rootDir string) error { @@ -27,3 +28,7 @@ func checkRootdirFs(rootDir string) error { func getMountpoint(id string) string { return id } + +func detachUnmount(mountpoint string) error { + return unix.Unmount(mountpoint, unix.MNT_DETACH) +} diff --git a/vendor/github.com/containers/storage/layers.go b/vendor/github.com/containers/storage/layers.go index 5e9930ea7..34d27ffa3 100644 --- a/vendor/github.com/containers/storage/layers.go +++ b/vendor/github.com/containers/storage/layers.go @@ -683,7 +683,7 @@ func (r *layerStore) PutAdditionalLayer(id string, parentLayer *Layer, names []s r.bycompressedsum[layer.CompressedDigest] = append(r.bycompressedsum[layer.CompressedDigest], layer.ID) } if layer.UncompressedDigest != "" { - r.byuncompressedsum[layer.CompressedDigest] = append(r.byuncompressedsum[layer.CompressedDigest], layer.ID) + r.byuncompressedsum[layer.UncompressedDigest] = append(r.byuncompressedsum[layer.UncompressedDigest], layer.ID) } if err := r.Save(); err != nil { r.driver.Remove(id) @@ -866,6 +866,14 @@ func (r *layerStore) Put(id string, parentLayer *Layer, names []string, mountLab return nil, -1, err } delete(layer.Flags, incompleteFlag) + } else { + // applyDiffWithOptions in the `diff != nil` case handles this bit for us + if layer.CompressedDigest != "" { + r.bycompressedsum[layer.CompressedDigest] = append(r.bycompressedsum[layer.CompressedDigest], layer.ID) + } + if layer.UncompressedDigest != "" { + r.byuncompressedsum[layer.UncompressedDigest] = append(r.byuncompressedsum[layer.UncompressedDigest], layer.ID) + } } err = r.Save() if err != nil { diff --git a/vendor/github.com/containers/storage/storage.conf-freebsd b/vendor/github.com/containers/storage/storage.conf-freebsd new file mode 100644 index 000000000..cc655c62e --- /dev/null +++ b/vendor/github.com/containers/storage/storage.conf-freebsd @@ -0,0 +1,205 @@ +# This file is is the configuration file for all tools +# that use the containers/storage library. The storage.conf file +# overrides all other storage.conf files. Container engines using the +# container/storage library do not inherit fields from other storage.conf +# files. +# +# Note: The storage.conf file overrides other storage.conf files based on this precedence: +# /usr/containers/storage.conf +# /etc/containers/storage.conf +# $HOME/.config/containers/storage.conf +# $XDG_CONFIG_HOME/containers/storage.conf (If XDG_CONFIG_HOME is set) +# See man 5 containers-storage.conf for more information +# The "container storage" table contains all of the server options. +[storage] + +# Default Storage Driver, Must be set for proper operation. +driver = "zfs" + +# Temporary storage location +runroot = "/var/run/containers/storage" + +# Primary Read/Write location of container storage +graphroot = "/var/db/containers/storage" + + +# Storage path for rootless users +# +# rootless_storage_path = "$HOME/.local/share/containers/storage" + +[storage.options] +# Storage options to be passed to underlying storage drivers + +# AdditionalImageStores is used to pass paths to additional Read/Only image stores +# Must be comma separated list. +additionalimagestores = [ +] + +# Remap-UIDs/GIDs is the mapping from UIDs/GIDs as they should appear inside of +# a container, to the UIDs/GIDs as they should appear outside of the container, +# and the length of the range of UIDs/GIDs. Additional mapped sets can be +# listed and will be heeded by libraries, but there are limits to the number of +# mappings which the kernel will allow when you later attempt to run a +# container. +# +# remap-uids = 0:1668442479:65536 +# remap-gids = 0:1668442479:65536 + +# Remap-User/Group is a user name which can be used to look up one or more UID/GID +# ranges in the /etc/subuid or /etc/subgid file. Mappings are set up starting +# with an in-container ID of 0 and then a host-level ID taken from the lowest +# range that matches the specified name, and using the length of that range. +# Additional ranges are then assigned, using the ranges which specify the +# lowest host-level IDs first, to the lowest not-yet-mapped in-container ID, +# until all of the entries have been used for maps. +# +# remap-user = "containers" +# remap-group = "containers" + +# Root-auto-userns-user is a user name which can be used to look up one or more UID/GID +# ranges in the /etc/subuid and /etc/subgid file. These ranges will be partitioned +# to containers configured to create automatically a user namespace. Containers +# configured to automatically create a user namespace can still overlap with containers +# having an explicit mapping set. +# This setting is ignored when running as rootless. +# root-auto-userns-user = "storage" +# +# Auto-userns-min-size is the minimum size for a user namespace created automatically. +# auto-userns-min-size=1024 +# +# Auto-userns-max-size is the minimum size for a user namespace created automatically. +# auto-userns-max-size=65536 + +[storage.options.overlay] +# ignore_chown_errors can be set to allow a non privileged user running with +# a single UID within a user namespace to run containers. The user can pull +# and use any image even those with multiple uids. Note multiple UIDs will be +# squashed down to the default uid in the container. These images will have no +# separation between the users in the container. Only supported for the overlay +# and vfs drivers. +#ignore_chown_errors = "false" + +# Inodes is used to set a maximum inodes of the container image. +# inodes = "" + +# Path to an helper program to use for mounting the file system instead of mounting it +# directly. +#mount_program = "/usr/bin/fuse-overlayfs" + +# mountopt specifies comma separated list of extra mount options +mountopt = "nodev" + +# Set to skip a PRIVATE bind mount on the storage home directory. +# skip_mount_home = "false" + +# Size is used to set a maximum size of the container image. +# size = "" + +# ForceMask specifies the permissions mask that is used for new files and +# directories. +# +# The values "shared" and "private" are accepted. +# Octal permission masks are also accepted. +# +# "": No value specified. +# All files/directories, get set with the permissions identified within the +# image. +# "private": it is equivalent to 0700. +# All files/directories get set with 0700 permissions. The owner has rwx +# access to the files. No other users on the system can access the files. +# This setting could be used with networked based homedirs. +# "shared": it is equivalent to 0755. +# The owner has rwx access to the files and everyone else can read, access +# and execute them. This setting is useful for sharing containers storage +# with other users. For instance have a storage owned by root but shared +# to rootless users as an additional store. +# NOTE: All files within the image are made readable and executable by any +# user on the system. Even /etc/shadow within your image is now readable by +# any user. +# +# OCTAL: Users can experiment with other OCTAL Permissions. +# +# Note: The force_mask Flag is an experimental feature, it could change in the +# future. When "force_mask" is set the original permission mask is stored in +# the "user.containers.override_stat" xattr and the "mount_program" option must +# be specified. Mount programs like "/usr/bin/fuse-overlayfs" present the +# extended attribute permissions to processes within containers rather then the +# "force_mask" permissions. +# +# force_mask = "" + +[storage.options.thinpool] +# Storage Options for thinpool + +# autoextend_percent determines the amount by which pool needs to be +# grown. This is specified in terms of % of pool size. So a value of 20 means +# that when threshold is hit, pool will be grown by 20% of existing +# pool size. +# autoextend_percent = "20" + +# autoextend_threshold determines the pool extension threshold in terms +# of percentage of pool size. For example, if threshold is 60, that means when +# pool is 60% full, threshold has been hit. +# autoextend_threshold = "80" + +# basesize specifies the size to use when creating the base device, which +# limits the size of images and containers. +# basesize = "10G" + +# blocksize specifies a custom blocksize to use for the thin pool. +# blocksize="64k" + +# directlvm_device specifies a custom block storage device to use for the +# thin pool. Required if you setup devicemapper. +# directlvm_device = "" + +# directlvm_device_force wipes device even if device already has a filesystem. +# directlvm_device_force = "True" + +# fs specifies the filesystem type to use for the base device. +# fs="xfs" + +# log_level sets the log level of devicemapper. +# 0: LogLevelSuppress 0 (Default) +# 2: LogLevelFatal +# 3: LogLevelErr +# 4: LogLevelWarn +# 5: LogLevelNotice +# 6: LogLevelInfo +# 7: LogLevelDebug +# log_level = "7" + +# min_free_space specifies the min free space percent in a thin pool require for +# new device creation to succeed. Valid values are from 0% - 99%. +# Value 0% disables +# min_free_space = "10%" + +# mkfsarg specifies extra mkfs arguments to be used when creating the base +# device. +# mkfsarg = "" + +# metadata_size is used to set the `pvcreate --metadatasize` options when +# creating thin devices. Default is 128k +# metadata_size = "" + +# Size is used to set a maximum size of the container image. +# size = "" + +# use_deferred_removal marks devicemapper block device for deferred removal. +# If the thinpool is in use when the driver attempts to remove it, the driver +# tells the kernel to remove it as soon as possible. Note this does not free +# up the disk space, use deferred deletion to fully remove the thinpool. +# use_deferred_removal = "True" + +# use_deferred_deletion marks thinpool device for deferred deletion. +# If the device is busy when the driver attempts to delete it, the driver +# will attempt to delete device every 30 seconds until successful. +# If the program using the driver exits, the driver will continue attempting +# to cleanup the next time the driver is used. Deferred deletion permanently +# deletes the device and all data stored in device will be lost. +# use_deferred_deletion = "True" + +# xfs_nospace_max_retries specifies the maximum number of retries XFS should +# attempt to complete IO when ENOSPC (no space) error is returned by +# underlying storage device. +# xfs_nospace_max_retries = "0" diff --git a/vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md b/vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md index 828a60b24..8a642563d 100644 --- a/vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md +++ b/vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md @@ -48,18 +48,6 @@ fsnotify uses build tags to compile different code on Linux, BSD, macOS, and Win Before doing a pull request, please do your best to test your changes on multiple platforms, and list which platforms you were able/unable to test on. -To aid in cross-platform testing there is a Vagrantfile for Linux and BSD. - -* Install [Vagrant](http://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/) -* Setup [Vagrant Gopher](https://github.com/nathany/vagrant-gopher) in your `src` folder. -* Run `vagrant up` from the project folder. You can also setup just one box with `vagrant up linux` or `vagrant up bsd` (note: the BSD box doesn't support Windows hosts at this time, and NFS may prompt for your host OS password) -* Once setup, you can run the test suite on a given OS with a single command `vagrant ssh linux -c 'cd fsnotify/fsnotify; go test'`. -* When you're done, you will want to halt or destroy the Vagrant boxes. - -Notice: fsnotify file system events won't trigger in shared folders. The tests get around this limitation by using the /tmp directory. - -Right now there is no equivalent solution for Windows and macOS, but there are Windows VMs [freely available from Microsoft](http://www.modern.ie/en-us/virtualization-tools#downloads). - ### Maintainers Help maintaining fsnotify is welcome. To be a maintainer: @@ -67,11 +55,6 @@ Help maintaining fsnotify is welcome. To be a maintainer: * Submit a pull request and sign the CLA as above. * You must be able to run the test suite on Mac, Windows, Linux and BSD. -To keep master clean, the fsnotify project uses the "apply mail" workflow outlined in Nathaniel Talbott's post ["Merge pull request" Considered Harmful][am]. This requires installing [hub][]. - All code changes should be internal pull requests. Releases are tagged using [Semantic Versioning](http://semver.org/). - -[hub]: https://github.com/github/hub -[am]: http://blog.spreedly.com/2014/06/24/merge-pull-request-considered-harmful/#.VGa5yZPF_Zs diff --git a/vendor/github.com/fsnotify/fsnotify/README.md b/vendor/github.com/fsnotify/fsnotify/README.md index df57b1b28..7797745da 100644 --- a/vendor/github.com/fsnotify/fsnotify/README.md +++ b/vendor/github.com/fsnotify/fsnotify/README.md @@ -1,12 +1,10 @@ # File system notifications for Go -[](https://godoc.org/github.com/fsnotify/fsnotify) [](https://goreportcard.com/report/github.com/fsnotify/fsnotify) +[](https://pkg.go.dev/github.com/fsnotify/fsnotify) [](https://goreportcard.com/report/github.com/fsnotify/fsnotify) [](https://github.com/fsnotify/fsnotify/issues/413) -fsnotify utilizes [golang.org/x/sys](https://godoc.org/golang.org/x/sys) rather than `syscall` from the standard library. Ensure you have the latest version installed by running: -```console -go get -u golang.org/x/sys/... -``` + +fsnotify utilizes [golang.org/x/sys](https://godoc.org/golang.org/x/sys) rather than `syscall` from the standard library. Cross platform: Windows, Linux, BSD and macOS. @@ -16,22 +14,20 @@ Cross platform: Windows, Linux, BSD and macOS. | kqueue | BSD, macOS, iOS\* | Supported | | ReadDirectoryChangesW | Windows | Supported | | FSEvents | macOS | [Planned](https://github.com/fsnotify/fsnotify/issues/11) | -| FEN | Solaris 11 | [In Progress](https://github.com/fsnotify/fsnotify/issues/12) | -| fanotify | Linux 2.6.37+ | [Planned](https://github.com/fsnotify/fsnotify/issues/114) | +| FEN | Solaris 11 | [In Progress](https://github.com/fsnotify/fsnotify/pull/371) | +| fanotify | Linux 2.6.37+ | [Maybe](https://github.com/fsnotify/fsnotify/issues/114) | | USN Journals | Windows | [Maybe](https://github.com/fsnotify/fsnotify/issues/53) | | Polling | *All* | [Maybe](https://github.com/fsnotify/fsnotify/issues/9) | \* Android and iOS are untested. -Please see [the documentation](https://godoc.org/github.com/fsnotify/fsnotify) and consult the [FAQ](#faq) for usage information. +Please see [the documentation](https://pkg.go.dev/github.com/fsnotify/fsnotify) and consult the [FAQ](#faq) for usage information. ## API stability -fsnotify is a fork of [howeyc/fsnotify](https://godoc.org/github.com/howeyc/fsnotify) with a new API as of v1.0. The API is based on [this design document](http://goo.gl/MrYxyA). - -All [releases](https://github.com/fsnotify/fsnotify/releases) are tagged based on [Semantic Versioning](http://semver.org/). Further API changes are [planned](https://github.com/fsnotify/fsnotify/milestones), and will be tagged with a new major revision number. +fsnotify is a fork of [howeyc/fsnotify](https://github.com/howeyc/fsnotify) with a new API as of v1.0. The API is based on [this design document](http://goo.gl/MrYxyA). -Go 1.6 supports dependencies located in the `vendor/` folder. Unless you are creating a library, it is recommended that you copy fsnotify into `vendor/github.com/fsnotify/fsnotify` within your project, and likewise for `golang.org/x/sys`. +All [releases](https://github.com/fsnotify/fsnotify/releases) are tagged based on [Semantic Versioning](http://semver.org/). ## Usage @@ -84,10 +80,6 @@ func main() { Please refer to [CONTRIBUTING][] before opening an issue or pull request. -## Example - -See [example_test.go](https://github.com/fsnotify/fsnotify/blob/master/example_test.go). - ## FAQ **When a file is moved to another directory is it still being watched?** diff --git a/vendor/github.com/fsnotify/fsnotify/fsnotify_unsupported.go b/vendor/github.com/fsnotify/fsnotify/fsnotify_unsupported.go new file mode 100644 index 000000000..eb25cb407 --- /dev/null +++ b/vendor/github.com/fsnotify/fsnotify/fsnotify_unsupported.go @@ -0,0 +1,36 @@ +// Copyright 2022 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +//go:build !darwin && !dragonfly && !freebsd && !linux && !netbsd && !solaris && !windows +// +build !darwin,!dragonfly,!freebsd,!linux,!netbsd,!solaris,!windows + +package fsnotify + +import ( + "fmt" + "runtime" +) + +// Watcher watches a set of files, delivering events to a channel. +type Watcher struct{} + +// NewWatcher establishes a new watcher with the underlying OS and begins waiting for events. +func NewWatcher() (*Watcher, error) { + return nil, fmt.Errorf("fsnotify not supported on %s", runtime.GOOS) +} + +// Close removes all watches and closes the events channel. +func (w *Watcher) Close() error { + return nil +} + +// Add starts watching the named file or directory (non-recursively). +func (w *Watcher) Add(name string) error { + return nil +} + +// Remove stops watching the the named file or directory (non-recursively). +func (w *Watcher) Remove(name string) error { + return nil +} diff --git a/vendor/github.com/fsnotify/fsnotify/go.mod b/vendor/github.com/fsnotify/fsnotify/go.mod index 54089e48b..8d1fc1295 100644 --- a/vendor/github.com/fsnotify/fsnotify/go.mod +++ b/vendor/github.com/fsnotify/fsnotify/go.mod @@ -1,6 +1,6 @@ module github.com/fsnotify/fsnotify -go 1.13 +go 1.16 require golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c diff --git a/vendor/github.com/fsnotify/fsnotify/inotify.go b/vendor/github.com/fsnotify/fsnotify/inotify.go index eb87699b5..a6d0e0ec8 100644 --- a/vendor/github.com/fsnotify/fsnotify/inotify.go +++ b/vendor/github.com/fsnotify/fsnotify/inotify.go @@ -163,6 +163,19 @@ func (w *Watcher) Remove(name string) error { return nil } +// WatchList returns the directories and files that are being monitered. +func (w *Watcher) WatchList() []string { + w.mu.Lock() + defer w.mu.Unlock() + + entries := make([]string, 0, len(w.watches)) + for pathname := range w.watches { + entries = append(entries, pathname) + } + + return entries +} + type watch struct { wd uint32 // Watch descriptor (as returned by the inotify_add_watch() syscall) flags uint32 // inotify flags of this watch (see inotify(7) for the list of valid flags) diff --git a/vendor/github.com/fsnotify/fsnotify/inotify_poller.go b/vendor/github.com/fsnotify/fsnotify/inotify_poller.go index e9ff9439f..b572a37c3 100644 --- a/vendor/github.com/fsnotify/fsnotify/inotify_poller.go +++ b/vendor/github.com/fsnotify/fsnotify/inotify_poller.go @@ -38,7 +38,6 @@ func newFdPoller(fd int) (*fdPoller, error) { poller.close() } }() - poller.fd = fd // Create epoll fd poller.epfd, errno = unix.EpollCreate1(unix.EPOLL_CLOEXEC) diff --git a/vendor/github.com/fsnotify/fsnotify/kqueue.go b/vendor/github.com/fsnotify/fsnotify/kqueue.go index 368f5b790..6fb8d8532 100644 --- a/vendor/github.com/fsnotify/fsnotify/kqueue.go +++ b/vendor/github.com/fsnotify/fsnotify/kqueue.go @@ -148,6 +148,19 @@ func (w *Watcher) Remove(name string) error { return nil } +// WatchList returns the directories and files that are being monitered. +func (w *Watcher) WatchList() []string { + w.mu.Lock() + defer w.mu.Unlock() + + entries := make([]string, 0, len(w.watches)) + for pathname := range w.watches { + entries = append(entries, pathname) + } + + return entries +} + // Watch all events (except NOTE_EXTEND, NOTE_LINK, NOTE_REVOKE) const noteAllEvents = unix.NOTE_DELETE | unix.NOTE_WRITE | unix.NOTE_ATTRIB | unix.NOTE_RENAME diff --git a/vendor/github.com/fsnotify/fsnotify/windows.go b/vendor/github.com/fsnotify/fsnotify/windows.go index c02b75f7c..ddc69ef87 100644 --- a/vendor/github.com/fsnotify/fsnotify/windows.go +++ b/vendor/github.com/fsnotify/fsnotify/windows.go @@ -12,6 +12,7 @@ import ( "fmt" "os" "path/filepath" + "reflect" "runtime" "sync" "syscall" @@ -96,6 +97,21 @@ func (w *Watcher) Remove(name string) error { return <-in.reply } +// WatchList returns the directories and files that are being monitered. +func (w *Watcher) WatchList() []string { + w.mu.Lock() + w.mu.Unlock() + + entries := make([]string, 0, len(w.watches)) + for _, entry := range w.watches { + for _, watchEntry := range entry { + entries = append(entries, watchEntry.path) + } + } + + return entries +} + const ( // Options for AddWatch sysFSONESHOT = 0x80000000 @@ -452,8 +468,16 @@ func (w *Watcher) readEvents() { // Point "raw" to the event in the buffer raw := (*syscall.FileNotifyInformation)(unsafe.Pointer(&watch.buf[offset])) - buf := (*[syscall.MAX_PATH]uint16)(unsafe.Pointer(&raw.FileName)) - name := syscall.UTF16ToString(buf[:raw.FileNameLength/2]) + // TODO: Consider using unsafe.Slice that is available from go1.17 + // https://stackoverflow.com/questions/51187973/how-to-create-an-array-or-a-slice-from-an-array-unsafe-pointer-in-golang + // instead of using a fixed syscall.MAX_PATH buf, we create a buf that is the size of the path name + size := int(raw.FileNameLength / 2) + var buf []uint16 + sh := (*reflect.SliceHeader)(unsafe.Pointer(&buf)) + sh.Data = uintptr(unsafe.Pointer(&raw.FileName)) + sh.Len = size + sh.Cap = size + name := syscall.UTF16ToString(buf) fullname := filepath.Join(watch.path, name) var mask uint64 diff --git a/vendor/github.com/mitchellh/mapstructure/CHANGELOG.md b/vendor/github.com/mitchellh/mapstructure/CHANGELOG.md deleted file mode 100644 index 38a099162..000000000 --- a/vendor/github.com/mitchellh/mapstructure/CHANGELOG.md +++ /dev/null @@ -1,83 +0,0 @@ -## 1.4.3 - -* Fix cases where `json.Number` didn't decode properly [GH-261] - -## 1.4.2 - -* Custom name matchers to support any sort of casing, formatting, etc. for - field names. [GH-250] -* Fix possible panic in ComposeDecodeHookFunc [GH-251] - -## 1.4.1 - -* Fix regression where `*time.Time` value would be set to empty and not be sent - to decode hooks properly [GH-232] - -## 1.4.0 - -* A new decode hook type `DecodeHookFuncValue` has been added that has - access to the full values. [GH-183] -* Squash is now supported with embedded fields that are struct pointers [GH-205] -* Empty strings will convert to 0 for all numeric types when weakly decoding [GH-206] - -## 1.3.3 - -* Decoding maps from maps creates a settable value for decode hooks [GH-203] - -## 1.3.2 - -* Decode into interface type with a struct value is supported [GH-187] - -## 1.3.1 - -* Squash should only squash embedded structs. [GH-194] - -## 1.3.0 - -* Added `",omitempty"` support. This will ignore zero values in the source - structure when encoding. [GH-145] - -## 1.2.3 - -* Fix duplicate entries in Keys list with pointer values. [GH-185] - -## 1.2.2 - -* Do not add unsettable (unexported) values to the unused metadata key - or "remain" value. [GH-150] - -## 1.2.1 - -* Go modules checksum mismatch fix - -## 1.2.0 - -* Added support to capture unused values in a field using the `",remain"` value - in the mapstructure tag. There is an example to showcase usage. -* Added `DecoderConfig` option to always squash embedded structs -* `json.Number` can decode into `uint` types -* Empty slices are preserved and not replaced with nil slices -* Fix panic that can occur in when decoding a map into a nil slice of structs -* Improved package documentation for godoc - -## 1.1.2 - -* Fix error when decode hook decodes interface implementation into interface - type. [GH-140] - -## 1.1.1 - -* Fix panic that can happen in `decodePtr` - -## 1.1.0 - -* Added `StringToIPHookFunc` to convert `string` to `net.IP` and `net.IPNet` [GH-133] -* Support struct to struct decoding [GH-137] -* If source map value is nil, then destination map value is nil (instead of empty) -* If source slice value is nil, then destination slice value is nil (instead of empty) -* If source pointer is nil, then destination pointer is set to nil (instead of - allocated zero value of type) - -## 1.0.0 - -* Initial tagged stable release. diff --git a/vendor/github.com/mitchellh/mapstructure/LICENSE b/vendor/github.com/mitchellh/mapstructure/LICENSE deleted file mode 100644 index f9c841a51..000000000 --- a/vendor/github.com/mitchellh/mapstructure/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -The MIT License (MIT) - -Copyright (c) 2013 Mitchell Hashimoto - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. diff --git a/vendor/github.com/mitchellh/mapstructure/README.md b/vendor/github.com/mitchellh/mapstructure/README.md deleted file mode 100644 index 0018dc7d9..000000000 --- a/vendor/github.com/mitchellh/mapstructure/README.md +++ /dev/null @@ -1,46 +0,0 @@ -# mapstructure [](https://godoc.org/github.com/mitchellh/mapstructure) - -mapstructure is a Go library for decoding generic map values to structures -and vice versa, while providing helpful error handling. - -This library is most useful when decoding values from some data stream (JSON, -Gob, etc.) where you don't _quite_ know the structure of the underlying data -until you read a part of it. You can therefore read a `map[string]interface{}` -and use this library to decode it into the proper underlying native Go -structure. - -## Installation - -Standard `go get`: - -``` -$ go get github.com/mitchellh/mapstructure -``` - -## Usage & Example - -For usage and examples see the [Godoc](http://godoc.org/github.com/mitchellh/mapstructure). - -The `Decode` function has examples associated with it there. - -## But Why?! - -Go offers fantastic standard libraries for decoding formats such as JSON. -The standard method is to have a struct pre-created, and populate that struct -from the bytes of the encoded format. This is great, but the problem is if -you have configuration or an encoding that changes slightly depending on -specific fields. For example, consider this JSON: - -```json -{ - "type": "person", - "name": "Mitchell" -} -``` - -Perhaps we can't populate a specific structure without first reading -the "type" field from the JSON. We could always do two passes over the -decoding of the JSON (reading the "type" first, and the rest later). -However, it is much simpler to just decode this into a `map[string]interface{}` -structure, read the "type" key, then use something like this library -to decode it into the proper structure. diff --git a/vendor/github.com/mitchellh/mapstructure/decode_hooks.go b/vendor/github.com/mitchellh/mapstructure/decode_hooks.go deleted file mode 100644 index 4d4bbc733..000000000 --- a/vendor/github.com/mitchellh/mapstructure/decode_hooks.go +++ /dev/null @@ -1,257 +0,0 @@ -package mapstructure - -import ( - "encoding" - "errors" - "fmt" - "net" - "reflect" - "strconv" - "strings" - "time" -) - -// typedDecodeHook takes a raw DecodeHookFunc (an interface{}) and turns -// it into the proper DecodeHookFunc type, such as DecodeHookFuncType. -func typedDecodeHook(h DecodeHookFunc) DecodeHookFunc { - // Create variables here so we can reference them with the reflect pkg - var f1 DecodeHookFuncType - var f2 DecodeHookFuncKind - var f3 DecodeHookFuncValue - - // Fill in the variables into this interface and the rest is done - // automatically using the reflect package. - potential := []interface{}{f1, f2, f3} - - v := reflect.ValueOf(h) - vt := v.Type() - for _, raw := range potential { - pt := reflect.ValueOf(raw).Type() - if vt.ConvertibleTo(pt) { - return v.Convert(pt).Interface() - } - } - - return nil -} - -// DecodeHookExec executes the given decode hook. This should be used -// since it'll naturally degrade to the older backwards compatible DecodeHookFunc -// that took reflect.Kind instead of reflect.Type. -func DecodeHookExec( - raw DecodeHookFunc, - from reflect.Value, to reflect.Value) (interface{}, error) { - - switch f := typedDecodeHook(raw).(type) { - case DecodeHookFuncType: - return f(from.Type(), to.Type(), from.Interface()) - case DecodeHookFuncKind: - return f(from.Kind(), to.Kind(), from.Interface()) - case DecodeHookFuncValue: - return f(from, to) - default: - return nil, errors.New("invalid decode hook signature") - } -} - -// ComposeDecodeHookFunc creates a single DecodeHookFunc that -// automatically composes multiple DecodeHookFuncs. -// -// The composed funcs are called in order, with the result of the -// previous transformation. -func ComposeDecodeHookFunc(fs ...DecodeHookFunc) DecodeHookFunc { - return func(f reflect.Value, t reflect.Value) (interface{}, error) { - var err error - data := f.Interface() - - newFrom := f - for _, f1 := range fs { - data, err = DecodeHookExec(f1, newFrom, t) - if err != nil { - return nil, err - } - newFrom = reflect.ValueOf(data) - } - - return data, nil - } -} - -// StringToSliceHookFunc returns a DecodeHookFunc that converts -// string to []string by splitting on the given sep. -func StringToSliceHookFunc(sep string) DecodeHookFunc { - return func( - f reflect.Kind, - t reflect.Kind, - data interface{}) (interface{}, error) { - if f != reflect.String || t != reflect.Slice { - return data, nil - } - - raw := data.(string) - if raw == "" { - return []string{}, nil - } - - return strings.Split(raw, sep), nil - } -} - -// StringToTimeDurationHookFunc returns a DecodeHookFunc that converts -// strings to time.Duration. -func StringToTimeDurationHookFunc() DecodeHookFunc { - return func( - f reflect.Type, - t reflect.Type, - data interface{}) (interface{}, error) { - if f.Kind() != reflect.String { - return data, nil - } - if t != reflect.TypeOf(time.Duration(5)) { - return data, nil - } - - // Convert it by parsing - return time.ParseDuration(data.(string)) - } -} - -// StringToIPHookFunc returns a DecodeHookFunc that converts -// strings to net.IP -func StringToIPHookFunc() DecodeHookFunc { - return func( - f reflect.Type, - t reflect.Type, - data interface{}) (interface{}, error) { - if f.Kind() != reflect.String { - return data, nil - } - if t != reflect.TypeOf(net.IP{}) { - return data, nil - } - - // Convert it by parsing - ip := net.ParseIP(data.(string)) - if ip == nil { - return net.IP{}, fmt.Errorf("failed parsing ip %v", data) - } - - return ip, nil - } -} - -// StringToIPNetHookFunc returns a DecodeHookFunc that converts -// strings to net.IPNet -func StringToIPNetHookFunc() DecodeHookFunc { - return func( - f reflect.Type, - t reflect.Type, - data interface{}) (interface{}, error) { - if f.Kind() != reflect.String { - return data, nil - } - if t != reflect.TypeOf(net.IPNet{}) { - return data, nil - } - - // Convert it by parsing - _, net, err := net.ParseCIDR(data.(string)) - return net, err - } -} - -// StringToTimeHookFunc returns a DecodeHookFunc that converts -// strings to time.Time. -func StringToTimeHookFunc(layout string) DecodeHookFunc { - return func( - f reflect.Type, - t reflect.Type, - data interface{}) (interface{}, error) { - if f.Kind() != reflect.String { - return data, nil - } - if t != reflect.TypeOf(time.Time{}) { - return data, nil - } - - // Convert it by parsing - return time.Parse(layout, data.(string)) - } -} - -// WeaklyTypedHook is a DecodeHookFunc which adds support for weak typing to -// the decoder. -// -// Note that this is significantly different from the WeaklyTypedInput option -// of the DecoderConfig. -func WeaklyTypedHook( - f reflect.Kind, - t reflect.Kind, - data interface{}) (interface{}, error) { - dataVal := reflect.ValueOf(data) - switch t { - case reflect.String: - switch f { - case reflect.Bool: - if dataVal.Bool() { - return "1", nil - } - return "0", nil - case reflect.Float32: - return strconv.FormatFloat(dataVal.Float(), 'f', -1, 64), nil - case reflect.Int: - return strconv.FormatInt(dataVal.Int(), 10), nil - case reflect.Slice: - dataType := dataVal.Type() - elemKind := dataType.Elem().Kind() - if elemKind == reflect.Uint8 { - return string(dataVal.Interface().([]uint8)), nil - } - case reflect.Uint: - return strconv.FormatUint(dataVal.Uint(), 10), nil - } - } - - return data, nil -} - -func RecursiveStructToMapHookFunc() DecodeHookFunc { - return func(f reflect.Value, t reflect.Value) (interface{}, error) { - if f.Kind() != reflect.Struct { - return f.Interface(), nil - } - - var i interface{} = struct{}{} - if t.Type() != reflect.TypeOf(&i).Elem() { - return f.Interface(), nil - } - - m := make(map[string]interface{}) - t.Set(reflect.ValueOf(m)) - - return f.Interface(), nil - } -} - -// TextUnmarshallerHookFunc returns a DecodeHookFunc that applies -// strings to the UnmarshalText function, when the target type -// implements the encoding.TextUnmarshaler interface -func TextUnmarshallerHookFunc() DecodeHookFuncType { - return func( - f reflect.Type, - t reflect.Type, - data interface{}) (interface{}, error) { - if f.Kind() != reflect.String { - return data, nil - } - result := reflect.New(t).Interface() - unmarshaller, ok := result.(encoding.TextUnmarshaler) - if !ok { - return data, nil - } - if err := unmarshaller.UnmarshalText([]byte(data.(string))); err != nil { - return nil, err - } - return result, nil - } -} diff --git a/vendor/github.com/mitchellh/mapstructure/error.go b/vendor/github.com/mitchellh/mapstructure/error.go deleted file mode 100644 index 47a99e5af..000000000 --- a/vendor/github.com/mitchellh/mapstructure/error.go +++ /dev/null @@ -1,50 +0,0 @@ -package mapstructure - -import ( - "errors" - "fmt" - "sort" - "strings" -) - -// Error implements the error interface and can represents multiple -// errors that occur in the course of a single decode. -type Error struct { - Errors []string -} - -func (e *Error) Error() string { - points := make([]string, len(e.Errors)) - for i, err := range e.Errors { - points[i] = fmt.Sprintf("* %s", err) - } - - sort.Strings(points) - return fmt.Sprintf( - "%d error(s) decoding:\n\n%s", - len(e.Errors), strings.Join(points, "\n")) -} - -// WrappedErrors implements the errwrap.Wrapper interface to make this -// return value more useful with the errwrap and go-multierror libraries. -func (e *Error) WrappedErrors() []error { - if e == nil { - return nil - } - - result := make([]error, len(e.Errors)) - for i, e := range e.Errors { - result[i] = errors.New(e) - } - - return result -} - -func appendErrors(errors []string, err error) []string { - switch e := err.(type) { - case *Error: - return append(errors, e.Errors...) - default: - return append(errors, e.Error()) - } -} diff --git a/vendor/github.com/mitchellh/mapstructure/go.mod b/vendor/github.com/mitchellh/mapstructure/go.mod deleted file mode 100644 index a03ae9730..000000000 --- a/vendor/github.com/mitchellh/mapstructure/go.mod +++ /dev/null @@ -1,3 +0,0 @@ -module github.com/mitchellh/mapstructure - -go 1.14 diff --git a/vendor/github.com/mitchellh/mapstructure/mapstructure.go b/vendor/github.com/mitchellh/mapstructure/mapstructure.go deleted file mode 100644 index 6b81b0067..000000000 --- a/vendor/github.com/mitchellh/mapstructure/mapstructure.go +++ /dev/null @@ -1,1467 +0,0 @@ -// Package mapstructure exposes functionality to convert one arbitrary -// Go type into another, typically to convert a map[string]interface{} -// into a native Go structure. -// -// The Go structure can be arbitrarily complex, containing slices, -// other structs, etc. and the decoder will properly decode nested -// maps and so on into the proper structures in the native Go struct. -// See the examples to see what the decoder is capable of. -// -// The simplest function to start with is Decode. -// -// Field Tags -// -// When decoding to a struct, mapstructure will use the field name by -// default to perform the mapping. For example, if a struct has a field -// "Username" then mapstructure will look for a key in the source value -// of "username" (case insensitive). -// -// type User struct { -// Username string -// } -// -// You can change the behavior of mapstructure by using struct tags. -// The default struct tag that mapstructure looks for is "mapstructure" -// but you can customize it using DecoderConfig. -// -// Renaming Fields -// -// To rename the key that mapstructure looks for, use the "mapstructure" -// tag and set a value directly. For example, to change the "username" example -// above to "user": -// -// type User struct { -// Username string `mapstructure:"user"` -// } -// -// Embedded Structs and Squashing -// -// Embedded structs are treated as if they're another field with that name. -// By default, the two structs below are equivalent when decoding with -// mapstructure: -// -// type Person struct { -// Name string -// } -// -// type Friend struct { -// Person -// } -// -// type Friend struct { -// Person Person -// } -// -// This would require an input that looks like below: -// -// map[string]interface{}{ -// "person": map[string]interface{}{"name": "alice"}, -// } -// -// If your "person" value is NOT nested, then you can append ",squash" to -// your tag value and mapstructure will treat it as if the embedded struct -// were part of the struct directly. Example: -// -// type Friend struct { -// Person `mapstructure:",squash"` -// } -// -// Now the following input would be accepted: -// -// map[string]interface{}{ -// "name": "alice", -// } -// -// When decoding from a struct to a map, the squash tag squashes the struct -// fields into a single map. Using the example structs from above: -// -// Friend{Person: Person{Name: "alice"}} -// -// Will be decoded into a map: -// -// map[string]interface{}{ -// "name": "alice", -// } -// -// DecoderConfig has a field that changes the behavior of mapstructure -// to always squash embedded structs. -// -// Remainder Values -// -// If there are any unmapped keys in the source value, mapstructure by -// default will silently ignore them. You can error by setting ErrorUnused -// in DecoderConfig. If you're using Metadata you can also maintain a slice -// of the unused keys. -// -// You can also use the ",remain" suffix on your tag to collect all unused -// values in a map. The field with this tag MUST be a map type and should -// probably be a "map[string]interface{}" or "map[interface{}]interface{}". -// See example below: -// -// type Friend struct { -// Name string -// Other map[string]interface{} `mapstructure:",remain"` -// } -// -// Given the input below, Other would be populated with the other -// values that weren't used (everything but "name"): -// -// map[string]interface{}{ -// "name": "bob", -// "address": "123 Maple St.", -// } -// -// Omit Empty Values -// -// When decoding from a struct to any other value, you may use the -// ",omitempty" suffix on your tag to omit that value if it equates to -// the zero value. The zero value of all types is specified in the Go -// specification. -// -// For example, the zero type of a numeric type is zero ("0"). If the struct -// field value is zero and a numeric type, the field is empty, and it won't -// be encoded into the destination type. -// -// type Source { -// Age int `mapstructure:",omitempty"` -// } -// -// Unexported fields -// -// Since unexported (private) struct fields cannot be set outside the package -// where they are defined, the decoder will simply skip them. -// -// For this output type definition: -// -// type Exported struct { -// private string // this unexported field will be skipped -// Public string -// } -// -// Using this map as input: -// -// map[string]interface{}{ -// "private": "I will be ignored", -// "Public": "I made it through!", -// } -// -// The following struct will be decoded: -// -// type Exported struct { -// private: "" // field is left with an empty string (zero value) -// Public: "I made it through!" -// } -// -// Other Configuration -// -// mapstructure is highly configurable. See the DecoderConfig struct -// for other features and options that are supported. -package mapstructure - -import ( - "encoding/json" - "errors" - "fmt" - "reflect" - "sort" - "strconv" - "strings" -) - -// DecodeHookFunc is the callback function that can be used for -// data transformations. See "DecodeHook" in the DecoderConfig -// struct. -// -// The type must be one of DecodeHookFuncType, DecodeHookFuncKind, or -// DecodeHookFuncValue. -// Values are a superset of Types (Values can return types), and Types are a -// superset of Kinds (Types can return Kinds) and are generally a richer thing -// to use, but Kinds are simpler if you only need those. -// -// The reason DecodeHookFunc is multi-typed is for backwards compatibility: -// we started with Kinds and then realized Types were the better solution, -// but have a promise to not break backwards compat so we now support -// both. -type DecodeHookFunc interface{} - -// DecodeHookFuncType is a DecodeHookFunc which has complete information about -// the source and target types. -type DecodeHookFuncType func(reflect.Type, reflect.Type, interface{}) (interface{}, error) - -// DecodeHookFuncKind is a DecodeHookFunc which knows only the Kinds of the -// source and target types. -type DecodeHookFuncKind func(reflect.Kind, reflect.Kind, interface{}) (interface{}, error) - -// DecodeHookFuncValue is a DecodeHookFunc which has complete access to both the source and target -// values. -type DecodeHookFuncValue func(from reflect.Value, to reflect.Value) (interface{}, error) - -// DecoderConfig is the configuration that is used to create a new decoder -// and allows customization of various aspects of decoding. -type DecoderConfig struct { - // DecodeHook, if set, will be called before any decoding and any - // type conversion (if WeaklyTypedInput is on). This lets you modify - // the values before they're set down onto the resulting struct. The - // DecodeHook is called for every map and value in the input. This means - // that if a struct has embedded fields with squash tags the decode hook - // is called only once with all of the input data, not once for each - // embedded struct. - // - // If an error is returned, the entire decode will fail with that error. - DecodeHook DecodeHookFunc - - // If ErrorUnused is true, then it is an error for there to exist - // keys in the original map that were unused in the decoding process - // (extra keys). - ErrorUnused bool - - // ZeroFields, if set to true, will zero fields before writing them. - // For example, a map will be emptied before decoded values are put in - // it. If this is false, a map will be merged. - ZeroFields bool - - // If WeaklyTypedInput is true, the decoder will make the following - // "weak" conversions: - // - // - bools to string (true = "1", false = "0") - // - numbers to string (base 10) - // - bools to int/uint (true = 1, false = 0) - // - strings to int/uint (base implied by prefix) - // - int to bool (true if value != 0) - // - string to bool (accepts: 1, t, T, TRUE, true, True, 0, f, F, - // FALSE, false, False. Anything else is an error) - // - empty array = empty map and vice versa - // - negative numbers to overflowed uint values (base 10) - // - slice of maps to a merged map - // - single values are converted to slices if required. Each - // element is weakly decoded. For example: "4" can become []int{4} - // if the target type is an int slice. - // - WeaklyTypedInput bool - - // Squash will squash embedded structs. A squash tag may also be - // added to an individual struct field using a tag. For example: - // - // type Parent struct { - // Child `mapstructure:",squash"` - // } - Squash bool - - // Metadata is the struct that will contain extra metadata about - // the decoding. If this is nil, then no metadata will be tracked. - Metadata *Metadata - - // Result is a pointer to the struct that will contain the decoded - // value. - Result interface{} - - // The tag name that mapstructure reads for field names. This - // defaults to "mapstructure" - TagName string - - // MatchName is the function used to match the map key to the struct - // field name or tag. Defaults to `strings.EqualFold`. This can be used - // to implement case-sensitive tag values, support snake casing, etc. - MatchName func(mapKey, fieldName string) bool -} - -// A Decoder takes a raw interface value and turns it into structured -// data, keeping track of rich error information along the way in case -// anything goes wrong. Unlike the basic top-level Decode method, you can -// more finely control how the Decoder behaves using the DecoderConfig -// structure. The top-level Decode method is just a convenience that sets -// up the most basic Decoder. -type Decoder struct { - config *DecoderConfig -} - -// Metadata contains information about decoding a structure that -// is tedious or difficult to get otherwise. -type Metadata struct { - // Keys are the keys of the structure which were successfully decoded - Keys []string - - // Unused is a slice of keys that were found in the raw value but - // weren't decoded since there was no matching field in the result interface - Unused []string -} - -// Decode takes an input structure and uses reflection to translate it to -// the output structure. output must be a pointer to a map or struct. -func Decode(input interface{}, output interface{}) error { - config := &DecoderConfig{ - Metadata: nil, - Result: output, - } - - decoder, err := NewDecoder(config) - if err != nil { - return err - } - - return decoder.Decode(input) -} - -// WeakDecode is the same as Decode but is shorthand to enable -// WeaklyTypedInput. See DecoderConfig for more info. -func WeakDecode(input, output interface{}) error { - config := &DecoderConfig{ - Metadata: nil, - Result: output, - WeaklyTypedInput: true, - } - - decoder, err := NewDecoder(config) - if err != nil { - return err - } - - return decoder.Decode(input) -} - -// DecodeMetadata is the same as Decode, but is shorthand to -// enable metadata collection. See DecoderConfig for more info. -func DecodeMetadata(input interface{}, output interface{}, metadata *Metadata) error { - config := &DecoderConfig{ - Metadata: metadata, - Result: output, - } - - decoder, err := NewDecoder(config) - if err != nil { - return err - } - - return decoder.Decode(input) -} - -// WeakDecodeMetadata is the same as Decode, but is shorthand to -// enable both WeaklyTypedInput and metadata collection. See -// DecoderConfig for more info. -func WeakDecodeMetadata(input interface{}, output interface{}, metadata *Metadata) error { - config := &DecoderConfig{ - Metadata: metadata, - Result: output, - WeaklyTypedInput: true, - } - - decoder, err := NewDecoder(config) - if err != nil { - return err - } - - return decoder.Decode(input) -} - -// NewDecoder returns a new decoder for the given configuration. Once -// a decoder has been returned, the same configuration must not be used -// again. -func NewDecoder(config *DecoderConfig) (*Decoder, error) { - val := reflect.ValueOf(config.Result) - if val.Kind() != reflect.Ptr { - return nil, errors.New("result must be a pointer") - } - - val = val.Elem() - if !val.CanAddr() { - return nil, errors.New("result must be addressable (a pointer)") - } - - if config.Metadata != nil { - if config.Metadata.Keys == nil { - config.Metadata.Keys = make([]string, 0) - } - - if config.Metadata.Unused == nil { - config.Metadata.Unused = make([]string, 0) - } - } - - if config.TagName == "" { - config.TagName = "mapstructure" - } - - if config.MatchName == nil { - config.MatchName = strings.EqualFold - } - - result := &Decoder{ - config: config, - } - - return result, nil -} - -// Decode decodes the given raw interface to the target pointer specified -// by the configuration. -func (d *Decoder) Decode(input interface{}) error { - return d.decode("", input, reflect.ValueOf(d.config.Result).Elem()) -} - -// Decodes an unknown data type into a specific reflection value. -func (d *Decoder) decode(name string, input interface{}, outVal reflect.Value) error { - var inputVal reflect.Value - if input != nil { - inputVal = reflect.ValueOf(input) - - // We need to check here if input is a typed nil. Typed nils won't - // match the "input == nil" below so we check that here. - if inputVal.Kind() == reflect.Ptr && inputVal.IsNil() { - input = nil - } - } - - if input == nil { - // If the data is nil, then we don't set anything, unless ZeroFields is set - // to true. - if d.config.ZeroFields { - outVal.Set(reflect.Zero(outVal.Type())) - - if d.config.Metadata != nil && name != "" { - d.config.Metadata.Keys = append(d.config.Metadata.Keys, name) - } - } - return nil - } - - if !inputVal.IsValid() { - // If the input value is invalid, then we just set the value - // to be the zero value. - outVal.Set(reflect.Zero(outVal.Type())) - if d.config.Metadata != nil && name != "" { - d.config.Metadata.Keys = append(d.config.Metadata.Keys, name) - } - return nil - } - - if d.config.DecodeHook != nil { - // We have a DecodeHook, so let's pre-process the input. - var err error - input, err = DecodeHookExec(d.config.DecodeHook, inputVal, outVal) - if err != nil { - return fmt.Errorf("error decoding '%s': %s", name, err) - } - } - - var err error - outputKind := getKind(outVal) - addMetaKey := true - switch outputKind { - case reflect.Bool: - err = d.decodeBool(name, input, outVal) - case reflect.Interface: - err = d.decodeBasic(name, input, outVal) - case reflect.String: - err = d.decodeString(name, input, outVal) - case reflect.Int: - err = d.decodeInt(name, input, outVal) - case reflect.Uint: - err = d.decodeUint(name, input, outVal) - case reflect.Float32: - err = d.decodeFloat(name, input, outVal) - case reflect.Struct: - err = d.decodeStruct(name, input, outVal) - case reflect.Map: - err = d.decodeMap(name, input, outVal) - case reflect.Ptr: - addMetaKey, err = d.decodePtr(name, input, outVal) - case reflect.Slice: - err = d.decodeSlice(name, input, outVal) - case reflect.Array: - err = d.decodeArray(name, input, outVal) - case reflect.Func: - err = d.decodeFunc(name, input, outVal) - default: - // If we reached this point then we weren't able to decode it - return fmt.Errorf("%s: unsupported type: %s", name, outputKind) - } - - // If we reached here, then we successfully decoded SOMETHING, so - // mark the key as used if we're tracking metainput. - if addMetaKey && d.config.Metadata != nil && name != "" { - d.config.Metadata.Keys = append(d.config.Metadata.Keys, name) - } - - return err -} - -// This decodes a basic type (bool, int, string, etc.) and sets the -// value to "data" of that type. -func (d *Decoder) decodeBasic(name string, data interface{}, val reflect.Value) error { - if val.IsValid() && val.Elem().IsValid() { - elem := val.Elem() - - // If we can't address this element, then its not writable. Instead, - // we make a copy of the value (which is a pointer and therefore - // writable), decode into that, and replace the whole value. - copied := false - if !elem.CanAddr() { - copied = true - - // Make *T - copy := reflect.New(elem.Type()) - - // *T = elem - copy.Elem().Set(elem) - - // Set elem so we decode into it - elem = copy - } - - // Decode. If we have an error then return. We also return right - // away if we're not a copy because that means we decoded directly. - if err := d.decode(name, data, elem); err != nil || !copied { - return err - } - - // If we're a copy, we need to set te final result - val.Set(elem.Elem()) - return nil - } - - dataVal := reflect.ValueOf(data) - - // If the input data is a pointer, and the assigned type is the dereference - // of that exact pointer, then indirect it so that we can assign it. - // Example: *string to string - if dataVal.Kind() == reflect.Ptr && dataVal.Type().Elem() == val.Type() { - dataVal = reflect.Indirect(dataVal) - } - - if !dataVal.IsValid() { - dataVal = reflect.Zero(val.Type()) - } - - dataValType := dataVal.Type() - if !dataValType.AssignableTo(val.Type()) { - return fmt.Errorf( - "'%s' expected type '%s', got '%s'", - name, val.Type(), dataValType) - } - - val.Set(dataVal) - return nil -} - -func (d *Decoder) decodeString(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataKind := getKind(dataVal) - - converted := true - switch { - case dataKind == reflect.String: - val.SetString(dataVal.String()) - case dataKind == reflect.Bool && d.config.WeaklyTypedInput: - if dataVal.Bool() { - val.SetString("1") - } else { - val.SetString("0") - } - case dataKind == reflect.Int && d.config.WeaklyTypedInput: - val.SetString(strconv.FormatInt(dataVal.Int(), 10)) - case dataKind == reflect.Uint && d.config.WeaklyTypedInput: - val.SetString(strconv.FormatUint(dataVal.Uint(), 10)) - case dataKind == reflect.Float32 && d.config.WeaklyTypedInput: - val.SetString(strconv.FormatFloat(dataVal.Float(), 'f', -1, 64)) - case dataKind == reflect.Slice && d.config.WeaklyTypedInput, - dataKind == reflect.Array && d.config.WeaklyTypedInput: - dataType := dataVal.Type() - elemKind := dataType.Elem().Kind() - switch elemKind { - case reflect.Uint8: - var uints []uint8 - if dataKind == reflect.Array { - uints = make([]uint8, dataVal.Len(), dataVal.Len()) - for i := range uints { - uints[i] = dataVal.Index(i).Interface().(uint8) - } - } else { - uints = dataVal.Interface().([]uint8) - } - val.SetString(string(uints)) - default: - converted = false - } - default: - converted = false - } - - if !converted { - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - - return nil -} - -func (d *Decoder) decodeInt(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataKind := getKind(dataVal) - dataType := dataVal.Type() - - switch { - case dataKind == reflect.Int: - val.SetInt(dataVal.Int()) - case dataKind == reflect.Uint: - val.SetInt(int64(dataVal.Uint())) - case dataKind == reflect.Float32: - val.SetInt(int64(dataVal.Float())) - case dataKind == reflect.Bool && d.config.WeaklyTypedInput: - if dataVal.Bool() { - val.SetInt(1) - } else { - val.SetInt(0) - } - case dataKind == reflect.String && d.config.WeaklyTypedInput: - str := dataVal.String() - if str == "" { - str = "0" - } - - i, err := strconv.ParseInt(str, 0, val.Type().Bits()) - if err == nil { - val.SetInt(i) - } else { - return fmt.Errorf("cannot parse '%s' as int: %s", name, err) - } - case dataType.PkgPath() == "encoding/json" && dataType.Name() == "Number": - jn := data.(json.Number) - i, err := jn.Int64() - if err != nil { - return fmt.Errorf( - "error decoding json.Number into %s: %s", name, err) - } - val.SetInt(i) - default: - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - - return nil -} - -func (d *Decoder) decodeUint(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataKind := getKind(dataVal) - dataType := dataVal.Type() - - switch { - case dataKind == reflect.Int: - i := dataVal.Int() - if i < 0 && !d.config.WeaklyTypedInput { - return fmt.Errorf("cannot parse '%s', %d overflows uint", - name, i) - } - val.SetUint(uint64(i)) - case dataKind == reflect.Uint: - val.SetUint(dataVal.Uint()) - case dataKind == reflect.Float32: - f := dataVal.Float() - if f < 0 && !d.config.WeaklyTypedInput { - return fmt.Errorf("cannot parse '%s', %f overflows uint", - name, f) - } - val.SetUint(uint64(f)) - case dataKind == reflect.Bool && d.config.WeaklyTypedInput: - if dataVal.Bool() { - val.SetUint(1) - } else { - val.SetUint(0) - } - case dataKind == reflect.String && d.config.WeaklyTypedInput: - str := dataVal.String() - if str == "" { - str = "0" - } - - i, err := strconv.ParseUint(str, 0, val.Type().Bits()) - if err == nil { - val.SetUint(i) - } else { - return fmt.Errorf("cannot parse '%s' as uint: %s", name, err) - } - case dataType.PkgPath() == "encoding/json" && dataType.Name() == "Number": - jn := data.(json.Number) - i, err := strconv.ParseUint(string(jn), 0, 64) - if err != nil { - return fmt.Errorf( - "error decoding json.Number into %s: %s", name, err) - } - val.SetUint(i) - default: - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - - return nil -} - -func (d *Decoder) decodeBool(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataKind := getKind(dataVal) - - switch { - case dataKind == reflect.Bool: - val.SetBool(dataVal.Bool()) - case dataKind == reflect.Int && d.config.WeaklyTypedInput: - val.SetBool(dataVal.Int() != 0) - case dataKind == reflect.Uint && d.config.WeaklyTypedInput: - val.SetBool(dataVal.Uint() != 0) - case dataKind == reflect.Float32 && d.config.WeaklyTypedInput: - val.SetBool(dataVal.Float() != 0) - case dataKind == reflect.String && d.config.WeaklyTypedInput: - b, err := strconv.ParseBool(dataVal.String()) - if err == nil { - val.SetBool(b) - } else if dataVal.String() == "" { - val.SetBool(false) - } else { - return fmt.Errorf("cannot parse '%s' as bool: %s", name, err) - } - default: - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - - return nil -} - -func (d *Decoder) decodeFloat(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataKind := getKind(dataVal) - dataType := dataVal.Type() - - switch { - case dataKind == reflect.Int: - val.SetFloat(float64(dataVal.Int())) - case dataKind == reflect.Uint: - val.SetFloat(float64(dataVal.Uint())) - case dataKind == reflect.Float32: - val.SetFloat(dataVal.Float()) - case dataKind == reflect.Bool && d.config.WeaklyTypedInput: - if dataVal.Bool() { - val.SetFloat(1) - } else { - val.SetFloat(0) - } - case dataKind == reflect.String && d.config.WeaklyTypedInput: - str := dataVal.String() - if str == "" { - str = "0" - } - - f, err := strconv.ParseFloat(str, val.Type().Bits()) - if err == nil { - val.SetFloat(f) - } else { - return fmt.Errorf("cannot parse '%s' as float: %s", name, err) - } - case dataType.PkgPath() == "encoding/json" && dataType.Name() == "Number": - jn := data.(json.Number) - i, err := jn.Float64() - if err != nil { - return fmt.Errorf( - "error decoding json.Number into %s: %s", name, err) - } - val.SetFloat(i) - default: - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - - return nil -} - -func (d *Decoder) decodeMap(name string, data interface{}, val reflect.Value) error { - valType := val.Type() - valKeyType := valType.Key() - valElemType := valType.Elem() - - // By default we overwrite keys in the current map - valMap := val - - // If the map is nil or we're purposely zeroing fields, make a new map - if valMap.IsNil() || d.config.ZeroFields { - // Make a new map to hold our result - mapType := reflect.MapOf(valKeyType, valElemType) - valMap = reflect.MakeMap(mapType) - } - - // Check input type and based on the input type jump to the proper func - dataVal := reflect.Indirect(reflect.ValueOf(data)) - switch dataVal.Kind() { - case reflect.Map: - return d.decodeMapFromMap(name, dataVal, val, valMap) - - case reflect.Struct: - return d.decodeMapFromStruct(name, dataVal, val, valMap) - - case reflect.Array, reflect.Slice: - if d.config.WeaklyTypedInput { - return d.decodeMapFromSlice(name, dataVal, val, valMap) - } - - fallthrough - - default: - return fmt.Errorf("'%s' expected a map, got '%s'", name, dataVal.Kind()) - } -} - -func (d *Decoder) decodeMapFromSlice(name string, dataVal reflect.Value, val reflect.Value, valMap reflect.Value) error { - // Special case for BC reasons (covered by tests) - if dataVal.Len() == 0 { - val.Set(valMap) - return nil - } - - for i := 0; i < dataVal.Len(); i++ { - err := d.decode( - name+"["+strconv.Itoa(i)+"]", - dataVal.Index(i).Interface(), val) - if err != nil { - return err - } - } - - return nil -} - -func (d *Decoder) decodeMapFromMap(name string, dataVal reflect.Value, val reflect.Value, valMap reflect.Value) error { - valType := val.Type() - valKeyType := valType.Key() - valElemType := valType.Elem() - - // Accumulate errors - errors := make([]string, 0) - - // If the input data is empty, then we just match what the input data is. - if dataVal.Len() == 0 { - if dataVal.IsNil() { - if !val.IsNil() { - val.Set(dataVal) - } - } else { - // Set to empty allocated value - val.Set(valMap) - } - - return nil - } - - for _, k := range dataVal.MapKeys() { - fieldName := name + "[" + k.String() + "]" - - // First decode the key into the proper type - currentKey := reflect.Indirect(reflect.New(valKeyType)) - if err := d.decode(fieldName, k.Interface(), currentKey); err != nil { - errors = appendErrors(errors, err) - continue - } - - // Next decode the data into the proper type - v := dataVal.MapIndex(k).Interface() - currentVal := reflect.Indirect(reflect.New(valElemType)) - if err := d.decode(fieldName, v, currentVal); err != nil { - errors = appendErrors(errors, err) - continue - } - - valMap.SetMapIndex(currentKey, currentVal) - } - - // Set the built up map to the value - val.Set(valMap) - - // If we had errors, return those - if len(errors) > 0 { - return &Error{errors} - } - - return nil -} - -func (d *Decoder) decodeMapFromStruct(name string, dataVal reflect.Value, val reflect.Value, valMap reflect.Value) error { - typ := dataVal.Type() - for i := 0; i < typ.NumField(); i++ { - // Get the StructField first since this is a cheap operation. If the - // field is unexported, then ignore it. - f := typ.Field(i) - if f.PkgPath != "" { - continue - } - - // Next get the actual value of this field and verify it is assignable - // to the map value. - v := dataVal.Field(i) - if !v.Type().AssignableTo(valMap.Type().Elem()) { - return fmt.Errorf("cannot assign type '%s' to map value field of type '%s'", v.Type(), valMap.Type().Elem()) - } - - tagValue := f.Tag.Get(d.config.TagName) - keyName := f.Name - - // If Squash is set in the config, we squash the field down. - squash := d.config.Squash && v.Kind() == reflect.Struct && f.Anonymous - - // Determine the name of the key in the map - if index := strings.Index(tagValue, ","); index != -1 { - if tagValue[:index] == "-" { - continue - } - // If "omitempty" is specified in the tag, it ignores empty values. - if strings.Index(tagValue[index+1:], "omitempty") != -1 && isEmptyValue(v) { - continue - } - - // If "squash" is specified in the tag, we squash the field down. - squash = !squash && strings.Index(tagValue[index+1:], "squash") != -1 - if squash { - // When squashing, the embedded type can be a pointer to a struct. - if v.Kind() == reflect.Ptr && v.Elem().Kind() == reflect.Struct { - v = v.Elem() - } - - // The final type must be a struct - if v.Kind() != reflect.Struct { - return fmt.Errorf("cannot squash non-struct type '%s'", v.Type()) - } - } - keyName = tagValue[:index] - } else if len(tagValue) > 0 { - if tagValue == "-" { - continue - } - keyName = tagValue - } - - switch v.Kind() { - // this is an embedded struct, so handle it differently - case reflect.Struct: - x := reflect.New(v.Type()) - x.Elem().Set(v) - - vType := valMap.Type() - vKeyType := vType.Key() - vElemType := vType.Elem() - mType := reflect.MapOf(vKeyType, vElemType) - vMap := reflect.MakeMap(mType) - - // Creating a pointer to a map so that other methods can completely - // overwrite the map if need be (looking at you decodeMapFromMap). The - // indirection allows the underlying map to be settable (CanSet() == true) - // where as reflect.MakeMap returns an unsettable map. - addrVal := reflect.New(vMap.Type()) - reflect.Indirect(addrVal).Set(vMap) - - err := d.decode(keyName, x.Interface(), reflect.Indirect(addrVal)) - if err != nil { - return err - } - - // the underlying map may have been completely overwritten so pull - // it indirectly out of the enclosing value. - vMap = reflect.Indirect(addrVal) - - if squash { - for _, k := range vMap.MapKeys() { - valMap.SetMapIndex(k, vMap.MapIndex(k)) - } - } else { - valMap.SetMapIndex(reflect.ValueOf(keyName), vMap) - } - - default: - valMap.SetMapIndex(reflect.ValueOf(keyName), v) - } - } - - if val.CanAddr() { - val.Set(valMap) - } - - return nil -} - -func (d *Decoder) decodePtr(name string, data interface{}, val reflect.Value) (bool, error) { - // If the input data is nil, then we want to just set the output - // pointer to be nil as well. - isNil := data == nil - if !isNil { - switch v := reflect.Indirect(reflect.ValueOf(data)); v.Kind() { - case reflect.Chan, - reflect.Func, - reflect.Interface, - reflect.Map, - reflect.Ptr, - reflect.Slice: - isNil = v.IsNil() - } - } - if isNil { - if !val.IsNil() && val.CanSet() { - nilValue := reflect.New(val.Type()).Elem() - val.Set(nilValue) - } - - return true, nil - } - - // Create an element of the concrete (non pointer) type and decode - // into that. Then set the value of the pointer to this type. - valType := val.Type() - valElemType := valType.Elem() - if val.CanSet() { - realVal := val - if realVal.IsNil() || d.config.ZeroFields { - realVal = reflect.New(valElemType) - } - - if err := d.decode(name, data, reflect.Indirect(realVal)); err != nil { - return false, err - } - - val.Set(realVal) - } else { - if err := d.decode(name, data, reflect.Indirect(val)); err != nil { - return false, err - } - } - return false, nil -} - -func (d *Decoder) decodeFunc(name string, data interface{}, val reflect.Value) error { - // Create an element of the concrete (non pointer) type and decode - // into that. Then set the value of the pointer to this type. - dataVal := reflect.Indirect(reflect.ValueOf(data)) - if val.Type() != dataVal.Type() { - return fmt.Errorf( - "'%s' expected type '%s', got unconvertible type '%s', value: '%v'", - name, val.Type(), dataVal.Type(), data) - } - val.Set(dataVal) - return nil -} - -func (d *Decoder) decodeSlice(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataValKind := dataVal.Kind() - valType := val.Type() - valElemType := valType.Elem() - sliceType := reflect.SliceOf(valElemType) - - // If we have a non array/slice type then we first attempt to convert. - if dataValKind != reflect.Array && dataValKind != reflect.Slice { - if d.config.WeaklyTypedInput { - switch { - // Slice and array we use the normal logic - case dataValKind == reflect.Slice, dataValKind == reflect.Array: - break - - // Empty maps turn into empty slices - case dataValKind == reflect.Map: - if dataVal.Len() == 0 { - val.Set(reflect.MakeSlice(sliceType, 0, 0)) - return nil - } - // Create slice of maps of other sizes - return d.decodeSlice(name, []interface{}{data}, val) - - case dataValKind == reflect.String && valElemType.Kind() == reflect.Uint8: - return d.decodeSlice(name, []byte(dataVal.String()), val) - - // All other types we try to convert to the slice type - // and "lift" it into it. i.e. a string becomes a string slice. - default: - // Just re-try this function with data as a slice. - return d.decodeSlice(name, []interface{}{data}, val) - } - } - - return fmt.Errorf( - "'%s': source data must be an array or slice, got %s", name, dataValKind) - } - - // If the input value is nil, then don't allocate since empty != nil - if dataVal.IsNil() { - return nil - } - - valSlice := val - if valSlice.IsNil() || d.config.ZeroFields { - // Make a new slice to hold our result, same size as the original data. - valSlice = reflect.MakeSlice(sliceType, dataVal.Len(), dataVal.Len()) - } - - // Accumulate any errors - errors := make([]string, 0) - - for i := 0; i < dataVal.Len(); i++ { - currentData := dataVal.Index(i).Interface() - for valSlice.Len() <= i { - valSlice = reflect.Append(valSlice, reflect.Zero(valElemType)) - } - currentField := valSlice.Index(i) - - fieldName := name + "[" + strconv.Itoa(i) + "]" - if err := d.decode(fieldName, currentData, currentField); err != nil { - errors = appendErrors(errors, err) - } - } - - // Finally, set the value to the slice we built up - val.Set(valSlice) - - // If there were errors, we return those - if len(errors) > 0 { - return &Error{errors} - } - - return nil -} - -func (d *Decoder) decodeArray(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - dataValKind := dataVal.Kind() - valType := val.Type() - valElemType := valType.Elem() - arrayType := reflect.ArrayOf(valType.Len(), valElemType) - - valArray := val - - if valArray.Interface() == reflect.Zero(valArray.Type()).Interface() || d.config.ZeroFields { - // Check input type - if dataValKind != reflect.Array && dataValKind != reflect.Slice { - if d.config.WeaklyTypedInput { - switch { - // Empty maps turn into empty arrays - case dataValKind == reflect.Map: - if dataVal.Len() == 0 { - val.Set(reflect.Zero(arrayType)) - return nil - } - - // All other types we try to convert to the array type - // and "lift" it into it. i.e. a string becomes a string array. - default: - // Just re-try this function with data as a slice. - return d.decodeArray(name, []interface{}{data}, val) - } - } - - return fmt.Errorf( - "'%s': source data must be an array or slice, got %s", name, dataValKind) - - } - if dataVal.Len() > arrayType.Len() { - return fmt.Errorf( - "'%s': expected source data to have length less or equal to %d, got %d", name, arrayType.Len(), dataVal.Len()) - - } - - // Make a new array to hold our result, same size as the original data. - valArray = reflect.New(arrayType).Elem() - } - - // Accumulate any errors - errors := make([]string, 0) - - for i := 0; i < dataVal.Len(); i++ { - currentData := dataVal.Index(i).Interface() - currentField := valArray.Index(i) - - fieldName := name + "[" + strconv.Itoa(i) + "]" - if err := d.decode(fieldName, currentData, currentField); err != nil { - errors = appendErrors(errors, err) - } - } - - // Finally, set the value to the array we built up - val.Set(valArray) - - // If there were errors, we return those - if len(errors) > 0 { - return &Error{errors} - } - - return nil -} - -func (d *Decoder) decodeStruct(name string, data interface{}, val reflect.Value) error { - dataVal := reflect.Indirect(reflect.ValueOf(data)) - - // If the type of the value to write to and the data match directly, - // then we just set it directly instead of recursing into the structure. - if dataVal.Type() == val.Type() { - val.Set(dataVal) - return nil - } - - dataValKind := dataVal.Kind() - switch dataValKind { - case reflect.Map: - return d.decodeStructFromMap(name, dataVal, val) - - case reflect.Struct: - // Not the most efficient way to do this but we can optimize later if - // we want to. To convert from struct to struct we go to map first - // as an intermediary. - - // Make a new map to hold our result - mapType := reflect.TypeOf((map[string]interface{})(nil)) - mval := reflect.MakeMap(mapType) - - // Creating a pointer to a map so that other methods can completely - // overwrite the map if need be (looking at you decodeMapFromMap). The - // indirection allows the underlying map to be settable (CanSet() == true) - // where as reflect.MakeMap returns an unsettable map. - addrVal := reflect.New(mval.Type()) - - reflect.Indirect(addrVal).Set(mval) - if err := d.decodeMapFromStruct(name, dataVal, reflect.Indirect(addrVal), mval); err != nil { - return err - } - - result := d.decodeStructFromMap(name, reflect.Indirect(addrVal), val) - return result - - default: - return fmt.Errorf("'%s' expected a map, got '%s'", name, dataVal.Kind()) - } -} - -func (d *Decoder) decodeStructFromMap(name string, dataVal, val reflect.Value) error { - dataValType := dataVal.Type() - if kind := dataValType.Key().Kind(); kind != reflect.String && kind != reflect.Interface { - return fmt.Errorf( - "'%s' needs a map with string keys, has '%s' keys", - name, dataValType.Key().Kind()) - } - - dataValKeys := make(map[reflect.Value]struct{}) - dataValKeysUnused := make(map[interface{}]struct{}) - for _, dataValKey := range dataVal.MapKeys() { - dataValKeys[dataValKey] = struct{}{} - dataValKeysUnused[dataValKey.Interface()] = struct{}{} - } - - errors := make([]string, 0) - - // This slice will keep track of all the structs we'll be decoding. - // There can be more than one struct if there are embedded structs - // that are squashed. - structs := make([]reflect.Value, 1, 5) - structs[0] = val - - // Compile the list of all the fields that we're going to be decoding - // from all the structs. - type field struct { - field reflect.StructField - val reflect.Value - } - - // remainField is set to a valid field set with the "remain" tag if - // we are keeping track of remaining values. - var remainField *field - - fields := []field{} - for len(structs) > 0 { - structVal := structs[0] - structs = structs[1:] - - structType := structVal.Type() - - for i := 0; i < structType.NumField(); i++ { - fieldType := structType.Field(i) - fieldVal := structVal.Field(i) - if fieldVal.Kind() == reflect.Ptr && fieldVal.Elem().Kind() == reflect.Struct { - // Handle embedded struct pointers as embedded structs. - fieldVal = fieldVal.Elem() - } - - // If "squash" is specified in the tag, we squash the field down. - squash := d.config.Squash && fieldVal.Kind() == reflect.Struct && fieldType.Anonymous - remain := false - - // We always parse the tags cause we're looking for other tags too - tagParts := strings.Split(fieldType.Tag.Get(d.config.TagName), ",") - for _, tag := range tagParts[1:] { - if tag == "squash" { - squash = true - break - } - - if tag == "remain" { - remain = true - break - } - } - - if squash { - if fieldVal.Kind() != reflect.Struct { - errors = appendErrors(errors, - fmt.Errorf("%s: unsupported type for squash: %s", fieldType.Name, fieldVal.Kind())) - } else { - structs = append(structs, fieldVal) - } - continue - } - - // Build our field - if remain { - remainField = &field{fieldType, fieldVal} - } else { - // Normal struct field, store it away - fields = append(fields, field{fieldType, fieldVal}) - } - } - } - - // for fieldType, field := range fields { - for _, f := range fields { - field, fieldValue := f.field, f.val - fieldName := field.Name - - tagValue := field.Tag.Get(d.config.TagName) - tagValue = strings.SplitN(tagValue, ",", 2)[0] - if tagValue != "" { - fieldName = tagValue - } - - rawMapKey := reflect.ValueOf(fieldName) - rawMapVal := dataVal.MapIndex(rawMapKey) - if !rawMapVal.IsValid() { - // Do a slower search by iterating over each key and - // doing case-insensitive search. - for dataValKey := range dataValKeys { - mK, ok := dataValKey.Interface().(string) - if !ok { - // Not a string key - continue - } - - if d.config.MatchName(mK, fieldName) { - rawMapKey = dataValKey - rawMapVal = dataVal.MapIndex(dataValKey) - break - } - } - - if !rawMapVal.IsValid() { - // There was no matching key in the map for the value in - // the struct. Just ignore. - continue - } - } - - if !fieldValue.IsValid() { - // This should never happen - panic("field is not valid") - } - - // If we can't set the field, then it is unexported or something, - // and we just continue onwards. - if !fieldValue.CanSet() { - continue - } - - // Delete the key we're using from the unused map so we stop tracking - delete(dataValKeysUnused, rawMapKey.Interface()) - - // If the name is empty string, then we're at the root, and we - // don't dot-join the fields. - if name != "" { - fieldName = name + "." + fieldName - } - - if err := d.decode(fieldName, rawMapVal.Interface(), fieldValue); err != nil { - errors = appendErrors(errors, err) - } - } - - // If we have a "remain"-tagged field and we have unused keys then - // we put the unused keys directly into the remain field. - if remainField != nil && len(dataValKeysUnused) > 0 { - // Build a map of only the unused values - remain := map[interface{}]interface{}{} - for key := range dataValKeysUnused { - remain[key] = dataVal.MapIndex(reflect.ValueOf(key)).Interface() - } - - // Decode it as-if we were just decoding this map onto our map. - if err := d.decodeMap(name, remain, remainField.val); err != nil { - errors = appendErrors(errors, err) - } - - // Set the map to nil so we have none so that the next check will - // not error (ErrorUnused) - dataValKeysUnused = nil - } - - if d.config.ErrorUnused && len(dataValKeysUnused) > 0 { - keys := make([]string, 0, len(dataValKeysUnused)) - for rawKey := range dataValKeysUnused { - keys = append(keys, rawKey.(string)) - } - sort.Strings(keys) - - err := fmt.Errorf("'%s' has invalid keys: %s", name, strings.Join(keys, ", ")) - errors = appendErrors(errors, err) - } - - if len(errors) > 0 { - return &Error{errors} - } - - // Add the unused keys to the list of unused keys if we're tracking metadata - if d.config.Metadata != nil { - for rawKey := range dataValKeysUnused { - key := rawKey.(string) - if name != "" { - key = name + "." + key - } - - d.config.Metadata.Unused = append(d.config.Metadata.Unused, key) - } - } - - return nil -} - -func isEmptyValue(v reflect.Value) bool { - switch getKind(v) { - case reflect.Array, reflect.Map, reflect.Slice, reflect.String: - return v.Len() == 0 - case reflect.Bool: - return !v.Bool() - case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: - return v.Int() == 0 - case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr: - return v.Uint() == 0 - case reflect.Float32, reflect.Float64: - return v.Float() == 0 - case reflect.Interface, reflect.Ptr: - return v.IsNil() - } - return false -} - -func getKind(val reflect.Value) reflect.Kind { - kind := val.Kind() - - switch { - case kind >= reflect.Int && kind <= reflect.Int64: - return reflect.Int - case kind >= reflect.Uint && kind <= reflect.Uint64: - return reflect.Uint - case kind >= reflect.Float32 && kind <= reflect.Float64: - return reflect.Float32 - default: - return kind - } -} diff --git a/vendor/modules.txt b/vendor/modules.txt index 1f878d2ef..dac0c8fca 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -46,7 +46,7 @@ github.com/blang/semver github.com/buger/goterm # github.com/cespare/xxhash/v2 v2.1.2 github.com/cespare/xxhash/v2 -# github.com/checkpoint-restore/checkpointctl v0.0.0-20211204171957-54b4ebfdb681 +# github.com/checkpoint-restore/checkpointctl v0.0.0-20220321135231-33f4a66335f0 ## explicit github.com/checkpoint-restore/checkpointctl/lib # github.com/checkpoint-restore/go-criu/v5 v5.3.0 @@ -109,7 +109,7 @@ github.com/containers/buildah/pkg/rusage github.com/containers/buildah/pkg/sshagent github.com/containers/buildah/pkg/util github.com/containers/buildah/util -# github.com/containers/common v0.47.5-0.20220413182852-c23a4e11f91b +# github.com/containers/common v0.47.5-0.20220421111103-112a47964ddb ## explicit github.com/containers/common/libimage github.com/containers/common/libimage/manifests @@ -153,7 +153,7 @@ github.com/containers/common/version # github.com/containers/conmon v2.0.20+incompatible ## explicit github.com/containers/conmon/runner/config -# github.com/containers/image/v5 v5.21.1-0.20220405081457-d1b64686e1d0 +# github.com/containers/image/v5 v5.21.1-0.20220421124950-8527e238867c ## explicit github.com/containers/image/v5/copy github.com/containers/image/v5/directory @@ -233,7 +233,7 @@ github.com/containers/psgo/internal/dev github.com/containers/psgo/internal/host github.com/containers/psgo/internal/proc github.com/containers/psgo/internal/process -# github.com/containers/storage v1.39.1-0.20220414183333-eea4e0f5f1f9 +# github.com/containers/storage v1.39.1-0.20220421071128-4899f8265d63 ## explicit github.com/containers/storage github.com/containers/storage/drivers @@ -386,7 +386,7 @@ github.com/docker/libnetwork/types github.com/dtylman/scp # github.com/felixge/httpsnoop v1.0.1 github.com/felixge/httpsnoop -# github.com/fsnotify/fsnotify v1.5.1 +# github.com/fsnotify/fsnotify v1.5.2 ## explicit github.com/fsnotify/fsnotify # github.com/fsouza/go-dockerclient v1.7.10 @@ -475,8 +475,6 @@ github.com/matttproud/golang_protobuf_extensions/pbutil github.com/miekg/pkcs11 # github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible github.com/mistifyio/go-zfs -# github.com/mitchellh/mapstructure v1.4.3 -github.com/mitchellh/mapstructure # github.com/moby/sys/mount v0.2.0 github.com/moby/sys/mount # github.com/moby/sys/mountinfo v0.6.1 @@ -645,7 +643,7 @@ github.com/stefanberger/go-pkcs11uri ## explicit github.com/stretchr/testify/assert github.com/stretchr/testify/require -# github.com/sylabs/sif/v2 v2.4.2 +# github.com/sylabs/sif/v2 v2.6.0 github.com/sylabs/sif/v2/pkg/sif # github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635 ## explicit |