diff options
57 files changed, 1378 insertions, 283 deletions
@@ -5,7 +5,7 @@ Podman (the POD MANager) is a tool for managing containers and images, volumes mounted into those containers, and pods made from groups of containers. Podman is based on libpod, a library for container lifecycle management that is also contained in this repository. The libpod library provides APIs for managing containers, pods, container images, and volumes. -* [Latest Version: 3.0.0](https://github.com/containers/podman/releases/latest) +* [Latest Version: 3.1.0](https://github.com/containers/podman/releases/latest) * Latest Remote client for Windows * Latest Remote client for MacOs * Latest Static Remote client for Linux diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 17d90aeca..85daba722 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,5 +1,136 @@ # Release Notes +## 3.1.0 +### Features +- A set of new commands has been added to manage secrets! The `podman secret create`, `podman secret inspect`, `podman secret ls` and `podman secret rm` commands have been added to handle secrets, along with the `--secret` option to `podman run` and `podman create` to add secrets to containers. The initial driver for secrets does not support encryption - this will be added in a future release. +- A new command to prune networks, `podman network prune`, has been added ([#8673](https://github.com/containers/podman/issues/8673)). +- The `-v` option to `podman run` and `podman create` now supports a new volume option, `:U`, to chown the volume's source directory on the host to match the UID and GID of the container and prevent permissions issues ([#7778](https://github.com/containers/podman/issues/7778)). +- Three new commands, `podman network exists`, `podman volume exists`, and `podman manifest exists`, have been added to check for the existence of networks, volumes, and manifest lists. +- The `podman cp` command can now copy files into directories mounted as `tmpfs` in a running container. +- The `podman volume prune` command will now list volumes that will be pruned when prompting the user whether to continue and perform the prune ([#8913](https://github.com/containers/podman/issues/8913)). +- The Podman remote client's `podman build` command now supports the `--disable-compression`, `--excludes`, and `--jobs` options. +- The Podman remote client's `podman push` command now supports the `--format` option. +- The Podman remote client's `podman rm` command now supports the `--all` and `--ignore` options. +- The Podman remote client's `podman search` command now supports the `--no-trunc` and `--list-tags` options. +- The `podman play kube` command can now read in Kubernetes YAML from `STDIN` when `-` is specified as file name (`podman play kube -`), allowing input to be piped into the command for scripting ([#8996](https://github.com/containers/podman/issues/8996)). +- The `podman generate systemd` command now supports a `--no-header` option, which disables creation of the header comment automatically added by Podman to generated unit files. +- The `podman generate kube` command can now generate `PersistentVolumeClaim` YAML for Podman named volumes ([#5788](https://github.com/containers/podman/issues/5788)). +- The `podman generate kube` command can now generate YAML files containing multiple resources (pods or deployments) ([#9129](https://github.com/containers/podman/issues/9129)). + +### Changes +- The Podman remote client's `podman build` command no longer allows the `-v` flag to be used. Volumes are not yet supported with remote Podman when the client and service are on different machines. +- The `podman kill` and `podman stop` commands now print the name given by the user for each container, instead of the full ID. +- When the `--security-opt unmask=ALL` or `--security-opt unmask=/sys/fs/cgroup` options to `podman create` or `podman run` are given, Podman will mount cgroups into the container as read-write, instead of read-only ([#8441](https://github.com/containers/podman/issues/8441)). +- The `podman rmi` command has been changed to better handle cases where an image is incomplete or corrupted, which can be caused by interrupted image pulls. +- The `podman rename` command has been improved to be more atomic, eliminating many race conditions that could potentially render a renamed container unusable. +- Detection of which OCI runtimes run using virtual machines and thus require custom SELinux labelling has been improved ([#9582](https://github.com/containers/podman/issues/9582)). +- The hidden `--trace` option to `podman` has been turned into a no-op. It was used in very early versions for performance tracing, but has not been supported for some time. +- The `podman generate systemd` command now generates `RequiresMountsFor` lines to ensure necessary storage directories are mounted before systemd starts Podman. +- Podman will now emit a warning when `--tty` and `--interactive` are both passed, but `STDIN` is not a TTY. This will be made into an error in the next major Podman release some time next year. + +### Bugfixes +- Fixed a bug where rootless Podman containers joined to CNI networks could not receive traffic from forwarded ports ([#9065](https://github.com/containers/podman/issues/9065)). +- Fixed a bug where `podman network create` with the `--macvlan` flag did not honor the `--gateway`, `--subnet`, and `--opt` options ([#9167](https://github.com/containers/podman/issues/9167)). +- Fixed a bug where the `podman generate kube` command generated invalid YAML for privileged containers ([#8897](https://github.com/containers/podman/issues/8897)). +- Fixed a bug where the `podman generate kube` command could not be used with containers that were not running. +- Fixed a bug where the `podman generate systemd` command could duplicate some parameters to Podman in generated unit files ([#9776](https://github.com/containers/podman/issues/9776)). +- Fixed a bug where Podman did not add annotations specified in `containers.conf` to containers. +- Foxed a bug where Podman did not respect the `no_hosts` default in `containers.conf` when creating containers. +- Fixed a bug where the `--tail=0`, `--since`, and `--follow` options to the `podman logs` command did not function properly when using the `journald` log backend. +- Fixed a bug where specifying more than one container to `podman logs` when the `journald` log backend was in use did not function correctly. +- Fixed a bug where the `podman run` and `podman create` commands would panic if a memory limit was set, but the swap limit was set to unlimited ([#9429](https://github.com/containers/podman/issues/9429)). +- Fixed a bug where the `--network` option to `podman run`, `podman create`, and `podman pod create` would error if the user attempted to specify CNI networks by ID, instead of name ([#9451](https://github.com/containers/podman/issues/9451)). +- Fixed a bug where Podman's cgroup handling for cgroups v1 systems did not properly handle cases where a cgroup existed on some, but not all, controllers, resulting in errors from the `podman stats` command ([#9252](https://github.com/containers/podman/issues/9252)). +- Fixed a bug where the `podman cp` did not properly handle cases where `/dev/stdout` was specified as the destination (it was treated identically to `-`) ([#9362](https://github.com/containers/podman/issues/9362)). +- Fixed a bug where the `podman cp` command would create files with incorrect ownership ([#9526](https://github.com/containers/podman/issues/9626)). +- Fixed a bug where the `podman cp` command did not properly handle cases where the destination directory did not exist. +- Fixed a bug where the `podman cp` command did not properly evaluate symlinks when copying out of containers. +- Fixed a bug where the `podman rm -fa` command would error when attempting to remove containers created with `--rm` ([#9479](https://github.com/containers/podman/issues/9479)). +- Fixed a bug where the ordering of capabilities was nondeterministic in the `CapDrop` field of the output of `podman inspect` on a container ([#9490](https://github.com/containers/podman/issues/9490)). +- Fixed a bug where the `podman network connect` command could be used with containers that were not initially connected to a CNI bridge network (e.g. containers created with `--net=host`) ([#9496](https://github.com/containers/podman/issues/9496)). +- Fixed a bug where DNS search domains required by the `dnsname` CNI plugin were not being added to container's `resolv.conf` under some circumstances. +- Fixed a bug where the `--ignorefile` option to `podman build` was nonfunctional ([#9570](https://github.com/containers/podman/issues/9570)). +- Fixed a bug where the `--timestamp` option to `podman build` was nonfunctional ([#9569](https://github.com/containers/podman/issues/9569)). +- Fixed a bug where the `--iidfile` option to `podman build` could cause Podman to panic if an error occurred during the build. +- Fixed a bug where the `--dns-search` option to `podman build` was nonfunctional ([#9574](https://github.com/containers/podman/issues/9574)). +- Fixed a bug where the `--pull-never` option to `podman build` was nonfunctional ([#9573](https://github.com/containers/podman/issues/9573)). +- Fixed a bug where the `--build-arg` option to `podman build` would, when given a key but not a value, error (instead of attempting to look up the key as an environment variable) ([#9571](https://github.com/containers/podman/issues/9571)). +- Fixed a bug where the `--isolation` option to `podman build` in the remote Podman client was nonfunctional. +- Fixed a bug where the `podman network disconnect` command could cause errors when the container that had a network removed was stopped and its network was cleaned up ([#9602](https://github.com/containers/podman/issues/9602)). +- Fixed a bug where the `podman network rm` command did not properly check what networks a container was present in, resulting in unexpected behavior if `podman network connect` or `podman network disconnect` had been used with the network ([#9632](https://github.com/containers/podman/issues/9632)). +- Fixed a bug where some errors with stopping a container could cause Podman to panic, and the container to be stuck in an unusable `stopping` state ([#9615](https://github.com/containers/podman/issues/9615)). +- Fixed a bug where the `podman load` command could return 0 even in cases where an error occurred ([#9672](https://github.com/containers/podman/issues/9672)). +- Fixed a bug where specifying storage options to Podman using the `--storage-opt` option would override all storage options. Instead, storage options are now overridden only when the `--storage-driver` option is used to override the current graph driver ([#9657](https://github.com/containers/podman/issues/9657)). +- Fixed a bug where containers created with `--privileged` could request more capabilities than were available to Podman. +- Fixed a bug where `podman commit` did not use the `TMPDIR` environment variable to place temporary files created during the commit ([#9825](https://github.com/containers/podman/issues/9825)). +- Fixed a bug where remote Podman could error when attempting to resize short-lived containers ([#9831](https://github.com/containers/podman/issues/9831)). +- Fixed a bug where Podman was unusable on kernels built without `CONFIG_USER_NS`. +- Fixed a bug where the ownership of volumes created by `podman volume create` and then mounted into a container could be incorrect ([#9608](https://github.com/containers/podman/issues/9608)). +- Fixed a bug where Podman volumes using a volume plugin could not pass certain options, and could not be used as non-root users. +- Fixed a bug where the `--tz` option to `podman create` and `podman run` did not properly validate its input. + +### API +- Fixed a bug where the `X-Registry-Auth` header did not accept `null` as a valid value. +- A new compat endpoint, `/auth`, has been added. This endpoint validates credentials against a registry ([#9564](https://github.com/containers/podman/issues/9564)). +- Fixed a bug where the compat Build endpoint for Images specified labels using the wrong type (array vs map). Both formats will be accepted now. +- Fixed a bug where the compat Build endpoint for Images did not report that it successfully tagged the built image in its response. +- Fixed a bug where the compat Create endpoint for Images did not provide progress information on pulling the image in its response. +- Fixed a bug where the compat Push endpoint for Images did not properly handle the destination (used a query parameter, instead of a path parameter). +- Fixed a bug where the compat Push endpoint for Images did not send the progress of the push and the digest of the pushed image in the response body. +- Fixed a bug where the compat List endpoint for Networks returned null, instead of an empty array (`[]`), when no networks were present ([#9293](https://github.com/containers/podman/issues/9293)). +- Fixed a bug where the compat List endpoint for Networks returned nulls, instead of empty maps, for networks that do not have Labels and/or Options. +- The Libpod Inspect endpoint for networks (`/libpod/network/$ID/json`) now has an alias at `/libpod/network/$ID` ([#9691](https://github.com/containers/podman/issues/9691)). +- Fixed a bug where the libpod Inspect endpoint for Networks returned a 1-size array of results, instead of a single result ([#9690](https://github.com/containers/podman/issues/9690)). +- The Compat List endpoint for Networks now supports the legacy format for filters in parallel with the current filter format ([#9526](https://github.com/containers/podman/issues/9526)). +- Fixed a bug where the compat Create endpoint for Containers did not properly handle tmpfs filesystems specified with options ([#9511](https://github.com/containers/podman/issues/9511)). +- Fixed a bug where the compat Create endpoint for Containers did not create bind-mount source directories ([#9510](https://github.com/containers/podman/issues/9510)). +- Fixed a bug where the compat Create endpoint for Containers did not properly handle the `NanoCpus` option ([#9523](https://github.com/containers/podman/issues/9523)). +- Fixed a bug where the Libpod create endpoint for Containers has a misnamed field in its JSON. +- Fixed a bug where the compat List endpoint for Containers did not populate information on forwarded ports ([#9553](https://github.com/containers/podman/issues/9553)) +- Fixed a bug where the compat List endpoint for Containers did not populate information on container CNI networks ([#9529](https://github.com/containers/podman/issues/9529)). +- Fixed a bug where the compat and libpod Stop endpoints for Containers would ignore a timeout of 0. +- Fixed a bug where the compat and libpod Resize endpoints for Containers did not set the correct terminal sizes (dimensions were reversed) ([#9756](https://github.com/containers/podman/issues/9756)). +- Fixed a bug where the compat Remove endpoint for Containers would not return 404 when attempting to remove a container that does not exist ([#9675](https://github.com/containers/podman/issues/9675)). +- Fixed a bug where the compat Prune endpoint for Volumes would still prune even if an invalid filter was specified. +- Numerous bugs related to filters have been addressed. + +### Misc +- Updated Buildah to v1.20.0 +- Updated the containers/storage library to v1.28.1 +- Updated the containers/image library to v5.10.5 +- Updated the containers/common library to v0.35.4 + +## 3.0.1 +### Changes +- Several frequently-occurring `WARN` level log messages have been downgraded to `INFO` or `DEBUG` to not clutter terminal output. + +### Bugfixes +- Fixed a bug where the `Created` field of `podman ps --format=json` was formatted as a string instead of an Unix timestamp (integer) ([#9315](https://github.com/containers/podman/issues/9315)). +- Fixed a bug where failing lookups of individual layers during the `podman images` command would cause the whole command to fail without printing output. +- Fixed a bug where `--cgroups=split` did not function properly on cgroups v1 systems. +- Fixed a bug where mounting a volume over an directory in the container that existed, but was empty, could fail ([#9393](https://github.com/containers/podman/issues/9393)). +- Fixed a bug where mounting a volume over a directory in the container that existed could copy the entirety of the container's rootfs, instead of just the directory mounted over, into the volume ([#9415](https://github.com/containers/podman/pull/9415)). +- Fixed a bug where Podman would treat the `--entrypoint=[""]` option to `podman run` and `podman create` as a literal empty string in the entrypoint, when instead it should have been ignored ([#9377](https://github.com/containers/podman/issues/9377)). +- Fixed a bug where Podman would set the `HOME` environment variable to `""` when the container ran as a user without an assigned home directory ([#9378](https://github.com/containers/podman/issues/9378)). +- Fixed a bug where specifying a pod infra image that had no tags (by using its ID) would cause `podman pod create` to panic ([#9374](https://github.com/containers/podman/issues/9374)). +- Fixed a bug where the `--runtime` option was not properly handled by the `podman build` command ([#9365](https://github.com/containers/podman/issues/9365)). +- Fixed a bug where Podman would incorrectly print an error message related to the remote API when the remote API was not in use and starting Podman failed. +- Fixed a bug where Podman would change ownership of a container's working directory, even if it already existed ([#9387](https://github.com/containers/podman/issues/9387)). +- Fixed a bug where the `podman generate systemd --new` command would incorrectly escape `%t` when generating the path for the PID file ([#9373](https://github.com/containers/podman/issues/9373)). +- Fixed a bug where Podman could, when run inside a Podman container with the host's containers/storage directory mounted into the container, erroneously detect a reboot and reset container state if the temporary directory was not also mounted in ([#9191](https://github.com/containers/podman/issues/9191)). +- Fixed a bug where some options of the `podman build` command (including but not limited to `--jobs`) were nonfunctional ([#9247](https://github.com/containers/podman/issues/9247)). + +### API +- Fixed a breaking change to the Libpod Wait API for Containers where the Conditions parameter changed type in Podman v3.0 ([#9351](https://github.com/containers/podman/issues/9351)). +- Fixed a bug where the Compat Create endpoint for Containers did not properly handle forwarded ports that did not specify a host port. +- Fixed a bug where the Libpod Wait endpoint for Containers could write duplicate headers after an error occurred. +- Fixed a bug where the Compat Create endpoint for Images would not pull images that already had a matching tag present locally, even if a more recent version was available at the registry ([#9232](https://github.com/containers/podman/issues/9232)). +- The Compat Create endpoint for Images has had its compatibility with Docker improved, allowing its use with the `docker-java` library. + +### Misc +- Updated Buildah to v1.19.4 +- Updated the containers/storage library to v1.24.6 + ## 3.0.0 ### Features - Podman now features initial support for Docker Compose. diff --git a/cmd/podman/common/create.go b/cmd/podman/common/create.go index d1170710b..dda709ecd 100644 --- a/cmd/podman/common/create.go +++ b/cmd/podman/common/create.go @@ -765,11 +765,15 @@ func DefineCreateFlags(cmd *cobra.Command, cf *ContainerCLIOpts) { ) _ = cmd.RegisterFlagCompletionFunc(mountFlagName, AutocompleteMountFlag) + volumeDesciption := "Bind mount a volume into the container" + if registry.IsRemote() { + volumeDesciption = "Bind mount a volume into the container. Volume src will be on the server machine, not the client" + } volumeFlagName := "volume" createFlags.StringArrayVarP( &cf.Volume, volumeFlagName, "v", volumes(), - "Bind mount a volume into the container", + volumeDesciption, ) _ = cmd.RegisterFlagCompletionFunc(volumeFlagName, AutocompleteVolumeFlag) @@ -804,4 +808,10 @@ func DefineCreateFlags(cmd *cobra.Command, cf *ContainerCLIOpts) { "Configure cgroup v2 (key=value)", ) _ = cmd.RegisterFlagCompletionFunc(cgroupConfFlagName, completion.AutocompleteNone) + + _ = createFlags.MarkHidden("signature-policy") + if registry.IsRemote() { + _ = createFlags.MarkHidden("env-host") + _ = createFlags.MarkHidden("http-proxy") + } } diff --git a/cmd/podman/containers/create.go b/cmd/podman/containers/create.go index af9278ce1..2da9aaf5e 100644 --- a/cmd/podman/containers/create.go +++ b/cmd/podman/containers/create.go @@ -63,11 +63,6 @@ func createFlags(cmd *cobra.Command) { common.DefineNetFlags(cmd) flags.SetNormalizeFunc(utils.AliasFlags) - - _ = flags.MarkHidden("signature-policy") - if registry.IsRemote() { - _ = flags.MarkHidden("http-proxy") - } } func init() { diff --git a/cmd/podman/containers/run.go b/cmd/podman/containers/run.go index db7180ca9..1a9fa2f0f 100644 --- a/cmd/podman/containers/run.go +++ b/cmd/podman/containers/run.go @@ -76,13 +76,11 @@ func runFlags(cmd *cobra.Command) { detachKeysFlagName := "detach-keys" flags.StringVar(&runOpts.DetachKeys, detachKeysFlagName, containerConfig.DetachKeys(), "Override the key sequence for detaching a container. Format is a single character `[a-Z]` or a comma separated sequence of `ctrl-<value>`, where `<value>` is one of: `a-cf`, `@`, `^`, `[`, `\\`, `]`, `^` or `_`") _ = cmd.RegisterFlagCompletionFunc(detachKeysFlagName, common.AutocompleteDetachKeys) - - _ = flags.MarkHidden("signature-policy") if registry.IsRemote() { - _ = flags.MarkHidden("http-proxy") _ = flags.MarkHidden("preserve-fds") } } + func init() { registry.Commands = append(registry.Commands, registry.CliCommand{ Mode: []entities.EngineMode{entities.ABIMode, entities.TunnelMode}, diff --git a/cmd/podman/images/build.go b/cmd/podman/images/build.go index da6d556b1..da7f5d862 100644 --- a/cmd/podman/images/build.go +++ b/cmd/podman/images/build.go @@ -8,7 +8,7 @@ import ( "strings" "time" - "github.com/containers/buildah/define" + buildahDefine "github.com/containers/buildah/define" buildahCLI "github.com/containers/buildah/pkg/cli" "github.com/containers/buildah/pkg/parse" "github.com/containers/common/pkg/completion" @@ -158,11 +158,11 @@ func buildFlags(cmd *cobra.Command) { flags.SetNormalizeFunc(buildahCLI.AliasFlags) if registry.IsRemote() { flag = flags.Lookup("isolation") - buildOpts.Isolation = define.OCI - if err := flag.Value.Set(define.OCI); err != nil { - logrus.Errorf("unable to set --isolation to %v: %v", define.OCI, err) + buildOpts.Isolation = buildahDefine.OCI + if err := flag.Value.Set(buildahDefine.OCI); err != nil { + logrus.Errorf("unable to set --isolation to %v: %v", buildahDefine.OCI, err) } - flag.DefValue = define.OCI + flag.DefValue = buildahDefine.OCI _ = flags.MarkHidden("disable-content-trust") _ = flags.MarkHidden("cache-from") _ = flags.MarkHidden("sign-by") @@ -195,7 +195,7 @@ func build(cmd *cobra.Command, args []string) error { var contextDir string if len(args) > 0 { // The context directory could be a URL. Try to handle that. - tempDir, subDir, err := define.TempDirForURL("", "buildah", args[0]) + tempDir, subDir, err := buildahDefine.TempDirForURL("", "buildah", args[0]) if err != nil { return errors.Wrapf(err, "error prepping temporary context directory") } @@ -318,16 +318,16 @@ func buildFlagsWrapperToOptions(c *cobra.Command, contextDir string, flags *buil return nil, errors.Errorf("can only set one of 'pull' or 'pull-always' or 'pull-never'") } - pullPolicy := define.PullIfMissing + pullPolicy := buildahDefine.PullIfMissing if c.Flags().Changed("pull") && flags.Pull { - pullPolicy = define.PullAlways + pullPolicy = buildahDefine.PullAlways } if flags.PullAlways { - pullPolicy = define.PullAlways + pullPolicy = buildahDefine.PullAlways } if flags.PullNever { - pullPolicy = define.PullNever + pullPolicy = buildahDefine.PullNever } args := make(map[string]string) @@ -402,9 +402,9 @@ func buildFlagsWrapperToOptions(c *cobra.Command, contextDir string, flags *buil flags.Layers = false } - compression := define.Gzip + compression := buildahDefine.Gzip if flags.DisableCompression { - compression = define.Uncompressed + compression = buildahDefine.Uncompressed } isolation, err := parse.IsolationOption(flags.Isolation) @@ -426,10 +426,10 @@ func buildFlagsWrapperToOptions(c *cobra.Command, contextDir string, flags *buil format := "" flags.Format = strings.ToLower(flags.Format) switch { - case strings.HasPrefix(flags.Format, define.OCI): - format = define.OCIv1ImageManifest - case strings.HasPrefix(flags.Format, define.DOCKER): - format = define.Dockerv2ImageManifest + case strings.HasPrefix(flags.Format, buildahDefine.OCI): + format = buildahDefine.OCIv1ImageManifest + case strings.HasPrefix(flags.Format, buildahDefine.DOCKER): + format = buildahDefine.Dockerv2ImageManifest default: return nil, errors.Errorf("unrecognized image type %q", flags.Format) } @@ -457,7 +457,7 @@ func buildFlagsWrapperToOptions(c *cobra.Command, contextDir string, flags *buil return nil, errors.Wrapf(err, "unable to obtain decrypt config") } - opts := define.BuildOptions{ + opts := buildahDefine.BuildOptions{ AddCapabilities: flags.CapAdd, AdditionalTags: tags, Annotations: flags.Annotation, diff --git a/cmd/podman/machine/init.go b/cmd/podman/machine/init.go index 05474fd89..61261e008 100644 --- a/cmd/podman/machine/init.go +++ b/cmd/podman/machine/init.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/machine/machine.go b/cmd/podman/machine/machine.go index ed284ee10..9a2377d12 100644 --- a/cmd/podman/machine/machine.go +++ b/cmd/podman/machine/machine.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/machine/machine_unsupported.go b/cmd/podman/machine/machine_unsupported.go index cb1636419..3bb44b51f 100644 --- a/cmd/podman/machine/machine_unsupported.go +++ b/cmd/podman/machine/machine_unsupported.go @@ -1,4 +1,4 @@ -// +build !amd64 arm64,linux amd64,windows +// +build !amd64 amd64,windows package machine diff --git a/cmd/podman/machine/rm.go b/cmd/podman/machine/rm.go index cd2cc84f2..002a793a3 100644 --- a/cmd/podman/machine/rm.go +++ b/cmd/podman/machine/rm.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/machine/ssh.go b/cmd/podman/machine/ssh.go index 879122a14..586c4267d 100644 --- a/cmd/podman/machine/ssh.go +++ b/cmd/podman/machine/ssh.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/machine/start.go b/cmd/podman/machine/start.go index 80fd77102..40800160e 100644 --- a/cmd/podman/machine/start.go +++ b/cmd/podman/machine/start.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/machine/stop.go b/cmd/podman/machine/stop.go index 4fcb065a3..7d655f0ba 100644 --- a/cmd/podman/machine/stop.go +++ b/cmd/podman/machine/stop.go @@ -1,4 +1,4 @@ -// +build amd64,linux amd64,darwin arm64,darwin +// +build amd64,linux arm64,linux amd64,darwin arm64,darwin package machine diff --git a/cmd/podman/system/service_abi.go b/cmd/podman/system/service_abi.go index 0e96e1fbb..9e8a9f9b4 100644 --- a/cmd/podman/system/service_abi.go +++ b/cmd/podman/system/service_abi.go @@ -24,16 +24,28 @@ func restService(opts entities.ServiceOptions, flags *pflag.FlagSet, cfg *entiti ) if opts.URI != "" { - fields := strings.Split(opts.URI, ":") - if len(fields) == 1 { - return errors.Errorf("%s is an invalid socket destination", opts.URI) + if os.Getenv("LISTEN_FDS") != "" { + // If it is activated by systemd, use the first LISTEN_FD (3) + // instead of opening the socket file. + f := os.NewFile(uintptr(3), "podman.sock") + l, err := net.FileListener(f) + if err != nil { + return err + } + listener = &l + } else { + fields := strings.Split(opts.URI, ":") + if len(fields) == 1 { + return errors.Errorf("%s is an invalid socket destination", opts.URI) + } + network := fields[0] + address := strings.Join(fields[1:], ":") + l, err := net.Listen(network, address) + if err != nil { + return errors.Wrapf(err, "unable to create socket") + } + listener = &l } - address := strings.Join(fields[1:], ":") - l, err := net.Listen(fields[0], address) - if err != nil { - return errors.Wrapf(err, "unable to create socket") - } - listener = &l } // Close stdin, so shortnames will not prompt diff --git a/contrib/spec/podman.spec.in b/contrib/spec/podman.spec.in index 5e820719f..78ff664f7 100644 --- a/contrib/spec/podman.spec.in +++ b/contrib/spec/podman.spec.in @@ -42,7 +42,7 @@ Epoch: 99 %else Epoch: 0 %endif -Version: 3.1.0 +Version: 3.2.0 Release: #COMMITDATE#.git%{shortcommit0}%{?dist} Summary: Manage Pods, Containers and Container Images License: ASL 2.0 diff --git a/docs/source/markdown/podman-create.1.md b/docs/source/markdown/podman-create.1.md index 9ae4ab207..f56319cf3 100644 --- a/docs/source/markdown/podman-create.1.md +++ b/docs/source/markdown/podman-create.1.md @@ -1029,7 +1029,7 @@ Create a bind mount. If you specify, ` -v /HOST-DIR:/CONTAINER-DIR`, Podman bind mounts `/HOST-DIR` in the host to `/CONTAINER-DIR` in the Podman container. Similarly, `-v SOURCE-VOLUME:/CONTAINER-DIR` will mount the volume in the host to the container. If no such named volume exists, Podman will -create one. The `OPTIONS` are a comma delimited list and can be: <sup>[[1]](#Footnote1)</sup> +create one. The `OPTIONS` are a comma delimited list and can be: <sup>[[1]](#Footnote1)</sup> (Note when using the remote client, the volumes will be mounted from the remote server, not necessarly the client machine.) The _options_ is a comma delimited list and can be: diff --git a/docs/source/markdown/podman-run.1.md b/docs/source/markdown/podman-run.1.md index 6d9d5ba28..fcb5a13ec 100644 --- a/docs/source/markdown/podman-run.1.md +++ b/docs/source/markdown/podman-run.1.md @@ -744,7 +744,7 @@ If a container is run within a pod, and the pod has an infra-container, the infr #### **\-\-preserve-fds**=*N* Pass down to the process N additional file descriptors (in addition to 0, 1, 2). -The total FDs will be 3+N. +The total FDs will be 3+N. (This option is not available with the remote Podman client) #### **\-\-privileged**=**true**|**false** @@ -1104,7 +1104,7 @@ Create a bind mount. If you specify _/HOST-DIR_:_/CONTAINER-DIR_, Podman bind mounts _host-dir_ in the host to _CONTAINER-DIR_ in the Podman container. Similarly, _SOURCE-VOLUME_:_/CONTAINER-DIR_ will mount the volume in the host to the container. If no such named volume exists, Podman will -create one. +create one. (Note when using the remote client, the volumes will be mounted from the remote server, not necessarly the client machine.) The _options_ is a comma delimited list and can be: <sup>[[1]](#Footnote1)</sup> diff --git a/hack/xref-helpmsgs-manpages b/hack/xref-helpmsgs-manpages index 42dd3634f..25e972fbc 100755 --- a/hack/xref-helpmsgs-manpages +++ b/hack/xref-helpmsgs-manpages @@ -331,6 +331,15 @@ sub podman_man { @most_recent_flags = (); # As of PR #8292, all options are <h4> and anchored if ($line =~ s/^\#{4}\s+//) { + # 2021-03: PR #9856: some (bleep) markdown processor converts + # double dashes to a single em-dash. We need to escape every + # instance with backslashes. This is anti-intuitive, and + # developers will naturally write --foo; try to catch that + # and warn with a helpful message. + if ($line =~ /--([a-z]+)/) { + warn "$ME: $subpath:$.: You probably need to backslash-escape '--$1' as '\\-\\-$1'\n"; + } + # If option has long and short form, long must come first. # This is a while-loop because there may be multiple long # option names, e.g. --net/--network diff --git a/libpod/events/logfile.go b/libpod/events/logfile.go index 7a32ea311..0f00525e8 100644 --- a/libpod/events/logfile.go +++ b/libpod/events/logfile.go @@ -7,7 +7,7 @@ import ( "time" "github.com/containers/podman/v3/pkg/util" - "github.com/containers/storage" + "github.com/containers/storage/pkg/lockfile" "github.com/pkg/errors" ) @@ -20,7 +20,7 @@ type EventLogFile struct { // Writes to the log file func (e EventLogFile) Write(ee Event) error { // We need to lock events file - lock, err := storage.GetLockfile(e.options.LogFilePath + ".lock") + lock, err := lockfile.GetLockfile(e.options.LogFilePath + ".lock") if err != nil { return err } diff --git a/libpod/networking_linux.go b/libpod/networking_linux.go index d6968a6b5..8bf532f66 100644 --- a/libpod/networking_linux.go +++ b/libpod/networking_linux.go @@ -916,7 +916,8 @@ func (r *Runtime) reloadContainerNetwork(ctr *Container) ([]*cnitypes.Result, er // teardownCNI will error if the iptables rules do not exists and this is the case after // a firewall reload. The purpose of network reload is to recreate the rules if they do // not exists so we should not log this specific error as error. This would confuse users otherwise. - b, rerr := regexp.MatchString("Couldn't load target `CNI-[a-f0-9]{24}':No such file or directory", err.Error()) + // iptables-legacy and iptables-nft will create different errors make sure to match both. + b, rerr := regexp.MatchString("Couldn't load target `CNI-[a-f0-9]{24}':No such file or directory|Chain 'CNI-[a-f0-9]{24}' does not exist", err.Error()) if rerr == nil && !b { logrus.Error(err) } else { diff --git a/libpod/runtime_img.go b/libpod/runtime_img.go index 13ac42e7d..b427125db 100644 --- a/libpod/runtime_img.go +++ b/libpod/runtime_img.go @@ -9,6 +9,7 @@ import ( "net/url" "os" + buildahDefine "github.com/containers/buildah/define" "github.com/containers/buildah/imagebuildah" "github.com/containers/image/v5/directory" "github.com/containers/image/v5/docker/reference" @@ -165,7 +166,7 @@ func (r *Runtime) newImageBuildCompleteEvent(idOrName string) { } // Build adds the runtime to the imagebuildah call -func (r *Runtime) Build(ctx context.Context, options imagebuildah.BuildOptions, dockerfiles ...string) (string, reference.Canonical, error) { +func (r *Runtime) Build(ctx context.Context, options buildahDefine.BuildOptions, dockerfiles ...string) (string, reference.Canonical, error) { if options.Runtime == "" { // Make sure that build containers use the same runtime as Podman (see #9365). conf := util.DefaultContainerConfig() diff --git a/pkg/api/handlers/compat/images_build.go b/pkg/api/handlers/compat/images_build.go index 36785a362..fd310711f 100644 --- a/pkg/api/handlers/compat/images_build.go +++ b/pkg/api/handlers/compat/images_build.go @@ -13,8 +13,7 @@ import ( "time" "github.com/containers/buildah" - "github.com/containers/buildah/define" - "github.com/containers/buildah/imagebuildah" + buildahDefine "github.com/containers/buildah/define" "github.com/containers/buildah/util" "github.com/containers/image/v5/types" "github.com/containers/podman/v3/libpod" @@ -277,13 +276,13 @@ func BuildImage(w http.ResponseWriter, r *http.Request) { jobs = query.Jobs } - pullPolicy := define.PullIfMissing + pullPolicy := buildahDefine.PullIfMissing if utils.IsLibpodRequest(r) { - pullPolicy = define.PolicyMap[query.PullPolicy] + pullPolicy = buildahDefine.PolicyMap[query.PullPolicy] } else { if _, found := r.URL.Query()["pull"]; found { if query.Pull { - pullPolicy = define.PullAlways + pullPolicy = buildahDefine.PullAlways } } } @@ -315,7 +314,7 @@ func BuildImage(w http.ResponseWriter, r *http.Request) { utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "Decode()")) return } - buildOptions := imagebuildah.BuildOptions{ + buildOptions := buildahDefine.BuildOptions{ AddCapabilities: addCaps, AdditionalTags: additionalTags, Annotations: annotations, diff --git a/pkg/api/handlers/compat/version.go b/pkg/api/handlers/compat/version.go index fae147440..f1cd77a9a 100644 --- a/pkg/api/handlers/compat/version.go +++ b/pkg/api/handlers/compat/version.go @@ -10,8 +10,8 @@ import ( "github.com/containers/podman/v3/libpod/define" "github.com/containers/podman/v3/pkg/api/handlers/utils" "github.com/containers/podman/v3/pkg/domain/entities" + "github.com/containers/podman/v3/pkg/domain/entities/types" "github.com/containers/podman/v3/version" - docker "github.com/docker/docker/api/types" "github.com/pkg/errors" ) @@ -32,7 +32,7 @@ func VersionHandler(w http.ResponseWriter, r *http.Request) { return } - components := []docker.ComponentVersion{{ + components := []types.ComponentVersion{{ Name: "Podman Engine", Version: versionInfo.Version, Details: map[string]string{ @@ -52,7 +52,7 @@ func VersionHandler(w http.ResponseWriter, r *http.Request) { minVersion := version.APIVersion[version.Compat][version.MinimalAPI] utils.WriteResponse(w, http.StatusOK, entities.ComponentVersion{ - Version: docker.Version{ + Version: types.Version{ Platform: struct { Name string }{ diff --git a/pkg/api/handlers/types.go b/pkg/api/handlers/types.go index f5eaf6f6d..736203171 100644 --- a/pkg/api/handlers/types.go +++ b/pkg/api/handlers/types.go @@ -28,10 +28,6 @@ type ContainerConfig struct { dockerContainer.Config } -type LibpodImagesLoadReport struct { - ID string `json:"id"` -} - type LibpodImagesPullReport struct { entities.ImagePullReport } diff --git a/pkg/api/handlers/types/types.go b/pkg/api/handlers/types/types.go new file mode 100644 index 000000000..71165364f --- /dev/null +++ b/pkg/api/handlers/types/types.go @@ -0,0 +1,23 @@ +package types + +import ( + "github.com/containers/podman/v3/pkg/domain/entities" +) + +// LibpodImagesRemoveReport is the return type for image removal via the rest +// api. +type LibpodImagesRemoveReport struct { + entities.ImageRemoveReport + // Image removal requires is to return data and an error. + Errors []string +} + +// HistoryResponse provides details on image layers +type HistoryResponse struct { + ID string `json:"Id"` + Created int64 + CreatedBy string + Tags []string + Size int64 + Comment string +} diff --git a/pkg/api/server/swagger.go b/pkg/api/server/swagger.go index 12fd083bb..d282edf23 100644 --- a/pkg/api/server/swagger.go +++ b/pkg/api/server/swagger.go @@ -205,7 +205,7 @@ type swagHealthCheckRunResponse struct { type swagVersion struct { // in:body Body struct { - entities.SystemVersionReport + entities.ComponentVersion } } diff --git a/pkg/bindings/connection.go b/pkg/bindings/connection.go index 21a8e7a8b..fd93c5ac7 100644 --- a/pkg/bindings/connection.go +++ b/pkg/bindings/connection.go @@ -22,14 +22,6 @@ import ( "golang.org/x/crypto/ssh/agent" ) -var ( - BasePath = &url.URL{ - Scheme: "http", - Host: "d", - Path: "/v" + version.APIVersion[version.Libpod][version.CurrentAPI].String() + "/libpod", - } -) - type APIResponse struct { *http.Response Request *http.Request @@ -318,16 +310,24 @@ func (c *Connection) DoRequest(httpBody io.Reader, httpMethod, endpoint string, err error response *http.Response ) - safePathValues := make([]interface{}, len(pathValues)) - // Make sure path values are http url safe + + params := make([]interface{}, len(pathValues)+3) + + // Including the semver suffices breaks older services... so do not include them + v := version.APIVersion[version.Libpod][version.CurrentAPI] + params[0] = v.Major + params[1] = v.Minor + params[2] = v.Patch for i, pv := range pathValues { - safePathValues[i] = url.PathEscape(pv) + // url.URL lacks the semantics for escaping embedded path parameters... so we manually + // escape each one and assume the caller included the correct formatting in "endpoint" + params[i+3] = url.PathEscape(pv) } - // Lets eventually use URL for this which might lead to safer - // usage - safeEndpoint := fmt.Sprintf(endpoint, safePathValues...) - e := BasePath.String() + safeEndpoint - req, err := http.NewRequest(httpMethod, e, httpBody) + + uri := fmt.Sprintf("http://d/v%d.%d.%d/libpod"+endpoint, params...) + logrus.Debugf("DoRequest Method: %s URI: %v", httpMethod, uri) + + req, err := http.NewRequest(httpMethod, uri, httpBody) if err != nil { return nil, err } diff --git a/pkg/bindings/images/build.go b/pkg/bindings/images/build.go index 17095b84b..c79d79136 100644 --- a/pkg/bindings/images/build.go +++ b/pkg/bindings/images/build.go @@ -174,7 +174,6 @@ func Build(ctx context.Context, containerFiles []string, options entities.BuildO if len(platform) > 0 { params.Set("platform", platform) } - params.Set("pullpolicy", options.PullPolicy.String()) if options.Quiet { diff --git a/pkg/bindings/images/images.go b/pkg/bindings/images/images.go index 1be2bdfdd..8680d6baa 100644 --- a/pkg/bindings/images/images.go +++ b/pkg/bindings/images/images.go @@ -8,7 +8,7 @@ import ( "net/url" "strconv" - "github.com/containers/podman/v3/pkg/api/handlers" + "github.com/containers/podman/v3/pkg/api/handlers/types" "github.com/containers/podman/v3/pkg/auth" "github.com/containers/podman/v3/pkg/bindings" "github.com/containers/podman/v3/pkg/domain/entities" @@ -96,12 +96,12 @@ func Tree(ctx context.Context, nameOrID string, options *TreeOptions) (*entities } // History returns the parent layers of an image. -func History(ctx context.Context, nameOrID string, options *HistoryOptions) ([]*handlers.HistoryResponse, error) { +func History(ctx context.Context, nameOrID string, options *HistoryOptions) ([]*types.HistoryResponse, error) { if options == nil { options = new(HistoryOptions) } _ = options - var history []*handlers.HistoryResponse + var history []*types.HistoryResponse conn, err := bindings.GetClient(ctx) if err != nil { return nil, err diff --git a/pkg/bindings/images/rm.go b/pkg/bindings/images/rm.go index beecce7bf..e45e583f4 100644 --- a/pkg/bindings/images/rm.go +++ b/pkg/bindings/images/rm.go @@ -4,7 +4,7 @@ import ( "context" "net/http" - "github.com/containers/podman/v3/pkg/api/handlers" + "github.com/containers/podman/v3/pkg/api/handlers/types" "github.com/containers/podman/v3/pkg/bindings" "github.com/containers/podman/v3/pkg/domain/entities" "github.com/containers/podman/v3/pkg/errorhandling" @@ -19,7 +19,7 @@ func Remove(ctx context.Context, images []string, options *RemoveOptions) (*enti // FIXME - bindings tests are missing for this endpoint. Once the CI is // re-enabled for bindings, we need to add them. At the time of writing, // the tests don't compile. - var report handlers.LibpodImagesRemoveReport + var report types.LibpodImagesRemoveReport conn, err := bindings.GetClient(ctx) if err != nil { return nil, []error{err} diff --git a/pkg/bindings/images/types.go b/pkg/bindings/images/types.go index 7bf70c82b..1f3e46729 100644 --- a/pkg/bindings/images/types.go +++ b/pkg/bindings/images/types.go @@ -1,7 +1,7 @@ package images import ( - "github.com/containers/buildah/imagebuildah" + buildahDefine "github.com/containers/buildah/define" ) //go:generate go run ../generator/generator.go RemoveOptions @@ -162,7 +162,7 @@ type PullOptions struct { //BuildOptions are optional options for building images type BuildOptions struct { - imagebuildah.BuildOptions + buildahDefine.BuildOptions } //go:generate go run ../generator/generator.go ExistsOptions diff --git a/pkg/domain/entities/images.go b/pkg/domain/entities/images.go index 7999d8209..3cc46ed0a 100644 --- a/pkg/domain/entities/images.go +++ b/pkg/domain/entities/images.go @@ -8,33 +8,32 @@ import ( "github.com/containers/image/v5/types" "github.com/containers/podman/v3/pkg/inspect" "github.com/containers/podman/v3/pkg/trust" - docker "github.com/docker/docker/api/types" "github.com/docker/docker/api/types/container" "github.com/opencontainers/go-digest" v1 "github.com/opencontainers/image-spec/specs-go/v1" ) type Image struct { - ID string `json:"Id"` - RepoTags []string `json:",omitempty"` - RepoDigests []string `json:",omitempty"` - Parent string `json:",omitempty"` - Comment string `json:",omitempty"` - Created string `json:",omitempty"` - Container string `json:",omitempty"` - ContainerConfig *container.Config `json:",omitempty"` - DockerVersion string `json:",omitempty"` - Author string `json:",omitempty"` - Config *container.Config `json:",omitempty"` - Architecture string `json:",omitempty"` - Variant string `json:",omitempty"` - Os string `json:",omitempty"` - OsVersion string `json:",omitempty"` - Size int64 `json:",omitempty"` - VirtualSize int64 `json:",omitempty"` - GraphDriver docker.GraphDriverData `json:",omitempty"` - RootFS docker.RootFS `json:",omitempty"` - Metadata docker.ImageMetadata `json:",omitempty"` + ID string `json:"Id"` + RepoTags []string `json:",omitempty"` + RepoDigests []string `json:",omitempty"` + Parent string `json:",omitempty"` + Comment string `json:",omitempty"` + Created string `json:",omitempty"` + Container string `json:",omitempty"` + ContainerConfig *container.Config `json:",omitempty"` + DockerVersion string `json:",omitempty"` + Author string `json:",omitempty"` + Config *container.Config `json:",omitempty"` + Architecture string `json:",omitempty"` + Variant string `json:",omitempty"` + Os string `json:",omitempty"` + OsVersion string `json:",omitempty"` + Size int64 `json:",omitempty"` + VirtualSize int64 `json:",omitempty"` + GraphDriver string `json:",omitempty"` + RootFS string `json:",omitempty"` + Metadata string `json:",omitempty"` // Podman extensions Digest digest.Digest `json:",omitempty"` diff --git a/pkg/domain/entities/system.go b/pkg/domain/entities/system.go index 4b8383613..1a671d59e 100644 --- a/pkg/domain/entities/system.go +++ b/pkg/domain/entities/system.go @@ -5,7 +5,7 @@ import ( "github.com/containers/podman/v3/libpod/define" "github.com/containers/podman/v3/pkg/domain/entities/reports" - "github.com/docker/docker/api/types" + "github.com/containers/podman/v3/pkg/domain/entities/types" "github.com/spf13/cobra" ) diff --git a/pkg/domain/entities/types.go b/pkg/domain/entities/types.go index 9efc64c93..02e374111 100644 --- a/pkg/domain/entities/types.go +++ b/pkg/domain/entities/types.go @@ -3,7 +3,7 @@ package entities import ( "net" - "github.com/containers/buildah/imagebuildah" + buildahDefine "github.com/containers/buildah/define" "github.com/containers/podman/v3/libpod/events" "github.com/containers/podman/v3/pkg/specgen" "github.com/containers/storage/pkg/archive" @@ -91,7 +91,7 @@ type ContainerCreateResponse struct { // BuildOptions describe the options for building container images. type BuildOptions struct { - imagebuildah.BuildOptions + buildahDefine.BuildOptions } // BuildReport is the image-build report. diff --git a/pkg/domain/entities/types/auth.go b/pkg/domain/entities/types/auth.go new file mode 100644 index 000000000..ddf15bb18 --- /dev/null +++ b/pkg/domain/entities/types/auth.go @@ -0,0 +1,22 @@ +package types // import "github.com/docker/docker/api/types" + +// AuthConfig contains authorization information for connecting to a Registry +type AuthConfig struct { + Username string `json:"username,omitempty"` + Password string `json:"password,omitempty"` + Auth string `json:"auth,omitempty"` + + // Email is an optional value associated with the username. + // This field is deprecated and will be removed in a later + // version of docker. + Email string `json:"email,omitempty"` + + ServerAddress string `json:"serveraddress,omitempty"` + + // IdentityToken is used to authenticate the user and get + // an access token for the registry. + IdentityToken string `json:"identitytoken,omitempty"` + + // RegistryToken is a bearer token to be sent to a registry + RegistryToken string `json:"registrytoken,omitempty"` +} diff --git a/pkg/domain/entities/types/types.go b/pkg/domain/entities/types/types.go new file mode 100644 index 000000000..77834c0cb --- /dev/null +++ b/pkg/domain/entities/types/types.go @@ -0,0 +1,28 @@ +package types // import "github.com/docker/docker/api/types" + +// ComponentVersion describes the version information for a specific component. +type ComponentVersion struct { + Name string + Version string + Details map[string]string `json:",omitempty"` +} + +// Version contains response of Engine API: +// GET "/version" +type Version struct { + Platform struct{ Name string } `json:",omitempty"` + Components []ComponentVersion `json:",omitempty"` + + // The following fields are deprecated, they relate to the Engine component and are kept for backwards compatibility + + Version string + APIVersion string `json:"ApiVersion"` + MinAPIVersion string `json:"MinAPIVersion,omitempty"` + GitCommit string + GoVersion string + Os string + Arch string + KernelVersion string `json:",omitempty"` + Experimental bool `json:",omitempty"` + BuildTime string `json:",omitempty"` +} diff --git a/pkg/domain/entities/volumes.go b/pkg/domain/entities/volumes.go index beb2a75ac..55a6a1b14 100644 --- a/pkg/domain/entities/volumes.go +++ b/pkg/domain/entities/volumes.go @@ -4,10 +4,72 @@ import ( "net/url" "github.com/containers/podman/v3/libpod/define" - docker_api_types "github.com/docker/docker/api/types" - docker_api_types_volume "github.com/docker/docker/api/types/volume" ) +// Volume volume +// swagger:model Volume +type volume struct { + + // Date/Time the volume was created. + CreatedAt string `json:"CreatedAt,omitempty"` + + // Name of the volume driver used by the volume. + // Required: true + Driver string `json:"Driver"` + + // User-defined key/value metadata. + // Required: true + Labels map[string]string `json:"Labels"` + + // Mount path of the volume on the host. + // Required: true + Mountpoint string `json:"Mountpoint"` + + // Name of the volume. + // Required: true + Name string `json:"Name"` + + // The driver specific options used when creating the volume. + // + // Required: true + Options map[string]string `json:"Options"` + + // The level at which the volume exists. Either `global` for cluster-wide, + // or `local` for machine level. + // + // Required: true + Scope string `json:"Scope"` + + // Low-level details about the volume, provided by the volume driver. + // Details are returned as a map with key/value pairs: + // `{"key":"value","key2":"value2"}`. + // + // The `Status` field is optional, and is omitted if the volume driver + // does not support this feature. + // + Status map[string]interface{} `json:"Status,omitempty"` + + // usage data + UsageData *VolumeUsageData `json:"UsageData,omitempty"` +} + +type VolumeUsageData struct { + + // The number of containers referencing this volume. This field + // is set to `-1` if the reference-count is not available. + // + // Required: true + RefCount int64 `json:"RefCount"` + + // Amount of disk space used by the volume (in bytes). This information + // is only available for volumes created with the `"local"` volume + // driver. For volumes created with other volume drivers, this field + // is set to `-1` ("not available") + // + // Required: true + Size int64 `json:"Size"` +} + // swagger:model VolumeCreate type VolumeCreateOptions struct { // New volume's name. Can be left blank @@ -113,14 +175,14 @@ type SwagVolumeListResponse struct { */ // swagger:model DockerVolumeCreate -type DockerVolumeCreate docker_api_types_volume.VolumeCreateBody +type DockerVolumeCreate VolumeCreateBody // This response definition is used for both the create and inspect endpoints // swagger:response DockerVolumeInfoResponse type SwagDockerVolumeInfoResponse struct { // in:body Body struct { - docker_api_types.Volume + volume } } @@ -129,6 +191,30 @@ type SwagDockerVolumeInfoResponse struct { type SwagDockerVolumePruneResponse struct { // in:body Body struct { - docker_api_types.VolumesPruneReport + // docker_api_types.VolumesPruneReport } } + +// VolumeCreateBody Volume configuration +// swagger:model VolumeCreateBody +type VolumeCreateBody struct { + + // Name of the volume driver to use. + // Required: true + Driver string `json:"Driver"` + + // A mapping of driver options and values. These options are + // passed directly to the driver and are driver specific. + // + // Required: true + DriverOpts map[string]string `json:"DriverOpts"` + + // User-defined key/value metadata. + // Required: true + Labels map[string]string `json:"Labels"` + + // The new volume's name. If not specified, Docker generates a name. + // + // Required: true + Name string `json:"Name"` +} diff --git a/pkg/domain/infra/abi/play.go b/pkg/domain/infra/abi/play.go index 7d87fc83a..3b5c141d7 100644 --- a/pkg/domain/infra/abi/play.go +++ b/pkg/domain/infra/abi/play.go @@ -9,6 +9,7 @@ import ( "os" "strings" + "github.com/containers/common/pkg/secrets" "github.com/containers/image/v5/types" "github.com/containers/podman/v3/libpod" "github.com/containers/podman/v3/libpod/define" @@ -135,6 +136,12 @@ func (ic *ContainerEngine) playKubePod(ctx context.Context, podName string, podY report entities.PlayKubeReport ) + // Create the secret manager before hand + secretsManager, err := secrets.NewManager(ic.Libpod.GetSecretsStorageDir()) + if err != nil { + return nil, err + } + // check for name collision between pod and container if podName == "" { return nil, errors.Errorf("pod does not have a name") @@ -261,16 +268,17 @@ func (ic *ContainerEngine) playKubePod(ctx context.Context, podName string, podY } specgenOpts := kube.CtrSpecGenOptions{ - Container: container, - Image: newImage, - Volumes: volumes, - PodID: pod.ID(), - PodName: podName, - PodInfraID: podInfraID, - ConfigMaps: configMaps, - SeccompPaths: seccompPaths, - RestartPolicy: ctrRestartPolicy, - NetNSIsHost: p.NetNS.IsHost(), + Container: container, + Image: newImage, + Volumes: volumes, + PodID: pod.ID(), + PodName: podName, + PodInfraID: podInfraID, + ConfigMaps: configMaps, + SeccompPaths: seccompPaths, + RestartPolicy: ctrRestartPolicy, + NetNSIsHost: p.NetNS.IsHost(), + SecretsManager: secretsManager, } specGen, err := kube.ToSpecGen(ctx, &specgenOpts) if err != nil { diff --git a/pkg/domain/infra/abi/system.go b/pkg/domain/infra/abi/system.go index 9f7c8919b..a3e753384 100644 --- a/pkg/domain/infra/abi/system.go +++ b/pkg/domain/infra/abi/system.go @@ -67,7 +67,7 @@ func (ic *ContainerEngine) SetupRootless(_ context.Context, cmd *cobra.Command) if os.Geteuid() == 0 { ownsCgroup, err := cgroups.UserOwnsCurrentSystemdCgroup() if err != nil { - logrus.Warnf("Failed to detect the owner for the current cgroup: %v", err) + logrus.Infof("Failed to detect the owner for the current cgroup: %v", err) } if !ownsCgroup { conf, err := ic.Config(context.Background()) diff --git a/pkg/domain/infra/runtime_libpod.go b/pkg/domain/infra/runtime_libpod.go index 8b6581c7b..b0d9dc797 100644 --- a/pkg/domain/infra/runtime_libpod.go +++ b/pkg/domain/infra/runtime_libpod.go @@ -15,8 +15,8 @@ import ( "github.com/containers/podman/v3/pkg/domain/entities" "github.com/containers/podman/v3/pkg/namespaces" "github.com/containers/podman/v3/pkg/rootless" - "github.com/containers/storage" "github.com/containers/storage/pkg/idtools" + "github.com/containers/storage/types" "github.com/pkg/errors" "github.com/sirupsen/logrus" flag "github.com/spf13/pflag" @@ -100,7 +100,7 @@ func GetRuntimeNoStore(ctx context.Context, fs *flag.FlagSet, cfg *entities.Podm func getRuntime(ctx context.Context, fs *flag.FlagSet, opts *engineOpts) (*libpod.Runtime, error) { options := []libpod.RuntimeOption{} - storageOpts := storage.StoreOptions{} + storageOpts := types.StoreOptions{} cfg := opts.config storageSet := false @@ -237,8 +237,8 @@ func getRuntime(ctx context.Context, fs *flag.FlagSet, opts *engineOpts) (*libpo } // ParseIDMapping takes idmappings and subuid and subgid maps and returns a storage mapping -func ParseIDMapping(mode namespaces.UsernsMode, uidMapSlice, gidMapSlice []string, subUIDMap, subGIDMap string) (*storage.IDMappingOptions, error) { - options := storage.IDMappingOptions{ +func ParseIDMapping(mode namespaces.UsernsMode, uidMapSlice, gidMapSlice []string, subUIDMap, subGIDMap string) (*types.IDMappingOptions, error) { + options := types.IDMappingOptions{ HostUIDMapping: true, HostGIDMapping: true, } diff --git a/pkg/machine/fcos.go b/pkg/machine/fcos.go index 0c6a2485e..32f943c87 100644 --- a/pkg/machine/fcos.go +++ b/pkg/machine/fcos.go @@ -2,17 +2,13 @@ package machine import ( "crypto/sha256" - "io" "io/ioutil" url2 "net/url" - "os" "path/filepath" "runtime" "strings" - "github.com/containers/storage/pkg/archive" digest "github.com/opencontainers/go-digest" - "github.com/sirupsen/logrus" ) // These should eventually be moved into machine/qemu as @@ -75,41 +71,7 @@ func (f FcosDownload) DownloadImage() error { return err } } - uncompressedFileWriter, err := os.OpenFile(f.getLocalUncompressedName(), os.O_CREATE|os.O_RDWR, 0600) - if err != nil { - return err - } - sourceFile, err := ioutil.ReadFile(f.LocalPath) - if err != nil { - return err - } - compressionType := archive.DetectCompression(sourceFile) - f.CompressionType = compressionType.Extension() - - switch f.CompressionType { - case "tar.xz": - return decompressXZ(f.LocalPath, uncompressedFileWriter) - default: - // File seems to be uncompressed, make a copy - if err := copyFile(f.LocalPath, uncompressedFileWriter); err != nil { - return err - } - } - return nil -} - -func copyFile(src string, dest *os.File) error { - source, err := os.Open(src) - if err != nil { - return err - } - defer func() { - if err := source.Close(); err != nil { - logrus.Error(err) - } - }() - _, err = io.Copy(dest, source) - return err + return Decompress(f.LocalPath, f.getLocalUncompressedName()) } func (f FcosDownload) Get() *Download { diff --git a/pkg/machine/fcos_arm64.go b/pkg/machine/fcos_arm64.go index ab50ca874..f5cd5a505 100644 --- a/pkg/machine/fcos_arm64.go +++ b/pkg/machine/fcos_arm64.go @@ -2,9 +2,9 @@ package machine import ( "encoding/json" - "fmt" "io/ioutil" "net/http" + url2 "net/url" "github.com/sirupsen/logrus" ) @@ -14,9 +14,7 @@ const aarchBaseURL = "https://fedorapeople.org/groups/fcos-images/builds/latest/ // Total hack until automation is possible. // We need a proper json file at least to automate func getFCOSDownload() (*fcosDownloadInfo, error) { - meta := Build{} - fmt.Println(aarchBaseURL + "meta.json") resp, err := http.Get(aarchBaseURL + "meta.json") if err != nil { return nil, err @@ -33,8 +31,18 @@ func getFCOSDownload() (*fcosDownloadInfo, error) { if err := json.Unmarshal(body, &meta); err != nil { return nil, err } + pathURL, err := url2.Parse(meta.BuildArtifacts.Qemu.Path) + if err != nil { + return nil, err + } + + baseURL, err := url2.Parse(aarchBaseURL) + if err != nil { + return nil, err + } + pullURL := baseURL.ResolveReference(pathURL) return &fcosDownloadInfo{ - Location: "https://fedorapeople.org/groups/fcos-images/builds/latest/aarch64/fedora-coreos-33.20210310.dev.0-qemu.aarch64.qcow2", + Location: pullURL.String(), Release: "", Sha256Sum: meta.BuildArtifacts.Qemu.Sha256, }, nil diff --git a/pkg/machine/pull.go b/pkg/machine/pull.go index 39dde15b8..41abe6993 100644 --- a/pkg/machine/pull.go +++ b/pkg/machine/pull.go @@ -3,17 +3,94 @@ package machine import ( "fmt" "io" + "io/ioutil" "net/http" + url2 "net/url" "os" "os/exec" + "path/filepath" "strings" "time" + "github.com/containers/image/v5/pkg/compression" + "github.com/docker/docker/pkg/archive" "github.com/sirupsen/logrus" "github.com/vbauerster/mpb/v6" "github.com/vbauerster/mpb/v6/decor" ) +// GenericDownload is used when a user provides a URL +// or path for an image +type GenericDownload struct { + Download +} + +// NewGenericDownloader is used when the disk image is provided by the user +func NewGenericDownloader(vmType, vmName, pullPath string) (DistributionDownload, error) { + var ( + imageName string + ) + dataDir, err := GetDataDir(vmType) + if err != nil { + return nil, err + } + dl := Download{} + // Is pullpath a file or url? + getURL, err := url2.Parse(pullPath) + if err != nil { + return nil, err + } + if len(getURL.Scheme) > 0 { + urlSplit := strings.Split(pullPath, "/") + imageName = urlSplit[len(urlSplit)-1] + dl.LocalUncompressedFile = filepath.Join(dataDir, imageName) + dl.URL = getURL + dl.LocalPath = filepath.Join(dataDir, imageName) + } else { + // Dealing with FilePath + imageName = filepath.Base(pullPath) + dl.LocalUncompressedFile = filepath.Join(dataDir, imageName) + dl.LocalPath = pullPath + } + dl.VMName = vmName + dl.ImageName = imageName + // The download needs to be pulled into the datadir + + gd := GenericDownload{Download: dl} + gd.LocalUncompressedFile = gd.getLocalUncompressedName() + return gd, nil +} + +func (g GenericDownload) getLocalUncompressedName() string { + var ( + extension string + ) + switch { + case strings.HasSuffix(g.LocalPath, ".bz2"): + extension = ".bz2" + case strings.HasSuffix(g.LocalPath, ".gz"): + extension = ".gz" + case strings.HasSuffix(g.LocalPath, ".xz"): + extension = ".xz" + } + uncompressedFilename := filepath.Join(filepath.Dir(g.LocalUncompressedFile), g.VMName+"_"+g.ImageName) + return strings.TrimSuffix(uncompressedFilename, extension) +} + +func (g GenericDownload) DownloadImage() error { + // If we have a URL for this "downloader", we now pull it + if g.URL != nil { + if err := DownloadVMImage(g.URL, g.LocalPath); err != nil { + return err + } + } + return Decompress(g.LocalPath, g.getLocalUncompressedName()) +} + +func (g GenericDownload) Get() *Download { + return &g.Download +} + // DownloadVMImage downloads a VM image from url to given path // with download status func DownloadVMImage(downloadURL fmt.Stringer, localImagePath string) error { @@ -38,7 +115,7 @@ func DownloadVMImage(downloadURL fmt.Stringer, localImagePath string) error { }() if resp.StatusCode != http.StatusOK { - return fmt.Errorf("error downloading VM image: %s", resp.Status) + return fmt.Errorf("error downloading VM image %s: %s", downloadURL, resp.Status) } size := resp.ContentLength urlSplit := strings.Split(downloadURL.String(), "/") @@ -75,6 +152,22 @@ func DownloadVMImage(downloadURL fmt.Stringer, localImagePath string) error { return nil } +func Decompress(localPath, uncompressedPath string) error { + uncompressedFileWriter, err := os.OpenFile(uncompressedPath, os.O_CREATE|os.O_RDWR, 0600) + if err != nil { + return err + } + sourceFile, err := ioutil.ReadFile(localPath) + if err != nil { + return err + } + + if compressionType := archive.DetectCompression(sourceFile); compressionType.Extension() == "tar.xz" { + return decompressXZ(localPath, uncompressedFileWriter) + } + return decompressEverythingElse(localPath, uncompressedFileWriter) +} + // Will error out if file without .xz already exists // Maybe extracting then renameing is a good idea here.. // depends on xz: not pre-installed on mac, so it becomes a brew dependecy @@ -95,3 +188,23 @@ func decompressXZ(src string, output io.Writer) error { }() return cmd.Run() } + +func decompressEverythingElse(src string, output io.Writer) error { + fmt.Println("Extracting compressed file") + f, err := os.Open(src) + if err != nil { + return err + } + uncompressStream, _, err := compression.AutoDecompress(f) + if err != nil { + return err + } + defer func() { + if err := uncompressStream.Close(); err != nil { + logrus.Error(err) + } + }() + + _, err = io.Copy(output, uncompressStream) + return err +} diff --git a/pkg/machine/qemu/machine.go b/pkg/machine/qemu/machine.go index fe155750f..fdb528a86 100644 --- a/pkg/machine/qemu/machine.go +++ b/pkg/machine/qemu/machine.go @@ -12,9 +12,8 @@ import ( "strconv" "time" - "github.com/containers/podman/v3/utils" - "github.com/containers/podman/v3/pkg/machine" + "github.com/containers/podman/v3/utils" "github.com/containers/storage/pkg/homedir" "github.com/digitalocean/go-qemu/qmp" "github.com/pkg/errors" @@ -83,7 +82,7 @@ func NewMachine(opts machine.InitOptions) (machine.VM, error) { return nil, err } vm.QMPMonitor = monitor - cmd = append(cmd, []string{"-qmp", monitor.Network + ":/" + monitor.Address + ",server,nowait"}...) + cmd = append(cmd, []string{"-qmp", monitor.Network + ":/" + monitor.Address + ",server=on,wait=off"}...) // Add network cmd = append(cmd, "-nic", "user,model=virtio,hostfwd=tcp::"+strconv.Itoa(vm.Port)+"-:22") @@ -96,7 +95,7 @@ func NewMachine(opts machine.InitOptions) (machine.VM, error) { // Add serial port for readiness cmd = append(cmd, []string{ "-device", "virtio-serial", - "-chardev", "socket,path=" + virtualSocketPath + ",server,nowait,id=" + vm.Name + "_ready", + "-chardev", "socket,path=" + virtualSocketPath + ",server=on,wait=off,id=" + vm.Name + "_ready", "-device", "virtserialport,chardev=" + vm.Name + "_ready" + ",name=org.fedoraproject.port.0"}...) vm.CmdLine = cmd return vm, nil @@ -135,15 +134,29 @@ func (v *MachineVM) Init(opts machine.InitOptions) error { jsonFile := filepath.Join(vmConfigDir, v.Name) + ".json" v.IdentityPath = filepath.Join(sshDir, v.Name) - dd, err := machine.NewFcosDownloader(vmtype, v.Name) - if err != nil { - return err + // The user has provided an alternate image which can be a file path + // or URL. + if len(opts.ImagePath) > 0 { + g, err := machine.NewGenericDownloader(vmtype, v.Name, opts.ImagePath) + if err != nil { + return err + } + v.ImagePath = g.Get().LocalUncompressedFile + if err := g.DownloadImage(); err != nil { + return err + } + } else { + // Get the image as usual + dd, err := machine.NewFcosDownloader(vmtype, v.Name) + if err != nil { + return err + } + v.ImagePath = dd.Get().LocalUncompressedFile + if err := dd.DownloadImage(); err != nil { + return err + } } - v.ImagePath = dd.Get().LocalUncompressedFile - if err := dd.DownloadImage(); err != nil { - return err - } // Add arch specific options including image location v.CmdLine = append(v.CmdLine, v.addArchOptions()...) @@ -171,10 +184,20 @@ func (v *MachineVM) Init(opts machine.InitOptions) error { return err } + originalDiskSize, err := getDiskSize(v.ImagePath) + if err != nil { + return err + } // Resize the disk image to input disk size - resize := exec.Command("qemu-img", []string{"resize", v.ImagePath, strconv.Itoa(int(opts.DiskSize)) + "G"}...) - if err := resize.Run(); err != nil { - return errors.Errorf("error resizing image: %q", err) + // only if the virtualdisk size is less than + // the given disk size + if opts.DiskSize<<(10*3) > originalDiskSize { + resize := exec.Command("qemu-img", []string{"resize", v.ImagePath, strconv.Itoa(int(opts.DiskSize)) + "G"}...) + resize.Stdout = os.Stdout + resize.Stderr = os.Stderr + if err := resize.Run(); err != nil { + return errors.Errorf("error resizing image: %q", err) + } } // Write the ignition file ign := machine.DynamicIgnition{ @@ -372,3 +395,34 @@ func (v *MachineVM) SSH(name string, opts machine.SSHOptions) error { return cmd.Run() } + +// executes qemu-image info to get the virtual disk size +// of the diskimage +func getDiskSize(path string) (uint64, error) { + diskInfo := exec.Command("qemu-img", "info", "--output", "json", path) + stdout, err := diskInfo.StdoutPipe() + if err != nil { + return 0, err + } + if err := diskInfo.Start(); err != nil { + return 0, err + } + tmpInfo := struct { + VirtualSize uint64 `json:"virtual-size"` + Filename string `json:"filename"` + ClusterSize int64 `json:"cluster-size"` + Format string `json:"format"` + FormatSpecific struct { + Type string `json:"type"` + Data map[string]string `json:"data"` + } + DirtyFlag bool `json:"dirty-flag"` + }{} + if err := json.NewDecoder(stdout).Decode(&tmpInfo); err != nil { + return 0, err + } + if err := diskInfo.Wait(); err != nil { + return 0, err + } + return tmpInfo.VirtualSize, nil +} diff --git a/pkg/machine/qemu/options_linux_arm64.go b/pkg/machine/qemu/options_linux_arm64.go new file mode 100644 index 000000000..948117653 --- /dev/null +++ b/pkg/machine/qemu/options_linux_arm64.go @@ -0,0 +1,41 @@ +package qemu + +import ( + "os" + "path/filepath" +) + +var ( + QemuCommand = "qemu-system-aarch64" +) + +func (v *MachineVM) addArchOptions() []string { + opts := []string{ + "-accel", "kvm", + "-cpu", "host", + "-M", "virt,gic-version=max", + "-bios", getQemuUefiFile("QEMU_EFI.fd"), + } + return opts +} + +func (v *MachineVM) prepare() error { + return nil +} + +func (v *MachineVM) archRemovalFiles() []string { + return []string{} +} + +func getQemuUefiFile(name string) string { + dirs := []string{ + "/usr/share/qemu-efi-aarch64", + "/usr/share/edk2/aarch64", + } + for _, dir := range dirs { + if _, err := os.Stat(dir); err == nil { + return filepath.Join(dir, name) + } + } + return name +} diff --git a/pkg/namespaces/namespaces.go b/pkg/namespaces/namespaces.go index c35f68e02..a7736aee0 100644 --- a/pkg/namespaces/namespaces.go +++ b/pkg/namespaces/namespaces.go @@ -5,7 +5,7 @@ import ( "strconv" "strings" - "github.com/containers/storage" + "github.com/containers/storage/types" ) const ( @@ -109,12 +109,12 @@ func (n UsernsMode) IsDefaultValue() bool { // GetAutoOptions returns a AutoUserNsOptions with the settings to setup automatically // a user namespace. -func (n UsernsMode) GetAutoOptions() (*storage.AutoUserNsOptions, error) { +func (n UsernsMode) GetAutoOptions() (*types.AutoUserNsOptions, error) { parts := strings.SplitN(string(n), ":", 2) if parts[0] != "auto" { return nil, fmt.Errorf("wrong user namespace mode") } - options := storage.AutoUserNsOptions{} + options := types.AutoUserNsOptions{} if len(parts) == 1 { return &options, nil } @@ -131,13 +131,13 @@ func (n UsernsMode) GetAutoOptions() (*storage.AutoUserNsOptions, error) { } options.Size = uint32(s) case "uidmapping": - mapping, err := storage.ParseIDMapping([]string{v[1]}, nil, "", "") + mapping, err := types.ParseIDMapping([]string{v[1]}, nil, "", "") if err != nil { return nil, err } options.AdditionalUIDMappings = append(options.AdditionalUIDMappings, mapping.UIDMap...) case "gidmapping": - mapping, err := storage.ParseIDMapping(nil, []string{v[1]}, "", "") + mapping, err := types.ParseIDMapping(nil, []string{v[1]}, "", "") if err != nil { return nil, err } diff --git a/pkg/rootless/rootless.go b/pkg/rootless/rootless.go index b5538efc3..0b9d719a9 100644 --- a/pkg/rootless/rootless.go +++ b/pkg/rootless/rootless.go @@ -5,7 +5,7 @@ import ( "sort" "sync" - "github.com/containers/storage" + "github.com/containers/storage/pkg/lockfile" "github.com/opencontainers/runc/libcontainer/user" spec "github.com/opencontainers/runtime-spec/specs-go" "github.com/pkg/errors" @@ -25,7 +25,7 @@ func TryJoinPauseProcess(pausePidPath string) (bool, int, error) { } // It could not join the pause process, let's lock the file before trying to delete it. - pidFileLock, err := storage.GetLockfile(pausePidPath) + pidFileLock, err := lockfile.GetLockfile(pausePidPath) if err != nil { // The file was deleted by another process. if os.IsNotExist(err) { diff --git a/pkg/rootless/rootless_linux.c b/pkg/rootless/rootless_linux.c index d93e4c10c..7a2bf0377 100644 --- a/pkg/rootless/rootless_linux.c +++ b/pkg/rootless/rootless_linux.c @@ -233,9 +233,8 @@ int is_fd_inherited(int fd) { if (open_files_set == NULL || fd > open_files_max_fd || fd < 0) - { return 0; - } + return FD_ISSET(fd % FD_SETSIZE, &(open_files_set[fd / FD_SETSIZE])) ? 1 : 0; } @@ -633,9 +632,10 @@ reexec_userns_join (int pid_to_join, char *pause_pid_file_path) close (user_ns); close (mnt_ns); - for (f = 3; f < open_files_max_fd; f++) - if (open_files_set == NULL || FD_ISSET (f % FD_SETSIZE, &(open_files_set[f / FD_SETSIZE]))) + for (f = 3; f <= open_files_max_fd; f++) + if (is_fd_inherited (f)) close (f); + return pid; } @@ -813,13 +813,14 @@ reexec_in_user_namespace (int ready, char *pause_pid_file_path, char *file_to_re if (do_socket_activation) { long num_fds; + num_fds = strtol (listen_fds, NULL, 10); if (num_fds != LONG_MIN && num_fds != LONG_MAX) { int f; for (f = 3; f < num_fds + 3; f++) - if (open_files_set == NULL || FD_ISSET (f % FD_SETSIZE, &(open_files_set[f / FD_SETSIZE]))) + if (is_fd_inherited (f)) close (f); } unsetenv ("LISTEN_PID"); diff --git a/pkg/specgen/generate/container_create.go b/pkg/specgen/generate/container_create.go index 03697b353..1d724ffb0 100644 --- a/pkg/specgen/generate/container_create.go +++ b/pkg/specgen/generate/container_create.go @@ -11,7 +11,7 @@ import ( "github.com/containers/podman/v3/libpod/image" "github.com/containers/podman/v3/pkg/specgen" "github.com/containers/podman/v3/pkg/util" - "github.com/containers/storage" + "github.com/containers/storage/types" "github.com/opencontainers/selinux/go-selinux/label" "github.com/pkg/errors" "github.com/sirupsen/logrus" @@ -367,7 +367,7 @@ func createContainerOptions(ctx context.Context, rt *libpod.Runtime, s *specgen. return options, nil } -func CreateExitCommandArgs(storageConfig storage.StoreOptions, config *config.Config, syslog, rm, exec bool) ([]string, error) { +func CreateExitCommandArgs(storageConfig types.StoreOptions, config *config.Config, syslog, rm, exec bool) ([]string, error) { // We need a cleanup process for containers in the current model. // But we can't assume that the caller is Podman - it could be another // user of the API. diff --git a/pkg/specgen/generate/kube/kube.go b/pkg/specgen/generate/kube/kube.go index d61c8bd19..31ed3fd7c 100644 --- a/pkg/specgen/generate/kube/kube.go +++ b/pkg/specgen/generate/kube/kube.go @@ -2,11 +2,13 @@ package kube import ( "context" + "encoding/json" "fmt" "net" "strings" "github.com/containers/common/pkg/parse" + "github.com/containers/common/pkg/secrets" "github.com/containers/podman/v3/libpod/image" ann "github.com/containers/podman/v3/pkg/annotations" "github.com/containers/podman/v3/pkg/specgen" @@ -94,6 +96,8 @@ type CtrSpecGenOptions struct { RestartPolicy string // NetNSIsHost tells the container to use the host netns NetNSIsHost bool + // SecretManager to access the secrets + SecretsManager *secrets.SecretsManager } func ToSpecGen(ctx context.Context, opts *CtrSpecGenOptions) (*specgen.SpecGenerator, error) { @@ -210,12 +214,18 @@ func ToSpecGen(ctx context.Context, opts *CtrSpecGenOptions) (*specgen.SpecGener } for _, env := range opts.Container.Env { - value := envVarValue(env, opts.ConfigMaps) + value, err := envVarValue(env, opts) + if err != nil { + return nil, err + } envs[env.Name] = value } for _, envFrom := range opts.Container.EnvFrom { - cmEnvs := envVarsFromConfigMap(envFrom, opts.ConfigMaps) + cmEnvs, err := envVarsFrom(envFrom, opts) + if err != nil { + return nil, err + } for k, v := range cmEnvs { envs[k] = v @@ -325,40 +335,96 @@ func quantityToInt64(quantity *resource.Quantity) (int64, error) { return 0, errors.Errorf("Quantity cannot be represented as int64: %v", quantity) } -// envVarsFromConfigMap returns all key-value pairs as env vars from a configMap that matches the envFrom setting of a container -func envVarsFromConfigMap(envFrom v1.EnvFromSource, configMaps []v1.ConfigMap) map[string]string { +// read a k8s secret in JSON format from the secret manager +func k8sSecretFromSecretManager(name string, secretsManager *secrets.SecretsManager) (map[string][]byte, error) { + _, jsonSecret, err := secretsManager.LookupSecretData(name) + if err != nil { + return nil, err + } + + var secrets map[string][]byte + if err := json.Unmarshal(jsonSecret, &secrets); err != nil { + return nil, errors.Errorf("Secret %v is not valid JSON: %v", name, err) + } + return secrets, nil +} + +// envVarsFrom returns all key-value pairs as env vars from a configMap or secret that matches the envFrom setting of a container +func envVarsFrom(envFrom v1.EnvFromSource, opts *CtrSpecGenOptions) (map[string]string, error) { envs := map[string]string{} if envFrom.ConfigMapRef != nil { - cmName := envFrom.ConfigMapRef.Name + cmRef := envFrom.ConfigMapRef + err := errors.Errorf("Configmap %v not found", cmRef.Name) - for _, c := range configMaps { - if cmName == c.Name { + for _, c := range opts.ConfigMaps { + if cmRef.Name == c.Name { envs = c.Data + err = nil break } } + + if err != nil && (cmRef.Optional == nil || !*cmRef.Optional) { + return nil, err + } } - return envs + if envFrom.SecretRef != nil { + secRef := envFrom.SecretRef + secret, err := k8sSecretFromSecretManager(secRef.Name, opts.SecretsManager) + if err == nil { + for k, v := range secret { + envs[k] = string(v) + } + } else if secRef.Optional == nil || !*secRef.Optional { + return nil, err + } + } + + return envs, nil } // envVarValue returns the environment variable value configured within the container's env setting. -// It gets the value from a configMap if specified, otherwise returns env.Value -func envVarValue(env v1.EnvVar, configMaps []v1.ConfigMap) string { - for _, c := range configMaps { - if env.ValueFrom != nil { - if env.ValueFrom.ConfigMapKeyRef != nil { - if env.ValueFrom.ConfigMapKeyRef.Name == c.Name { - if value, ok := c.Data[env.ValueFrom.ConfigMapKeyRef.Key]; ok { - return value +// It gets the value from a configMap or secret if specified, otherwise returns env.Value +func envVarValue(env v1.EnvVar, opts *CtrSpecGenOptions) (string, error) { + if env.ValueFrom != nil { + if env.ValueFrom.ConfigMapKeyRef != nil { + cmKeyRef := env.ValueFrom.ConfigMapKeyRef + err := errors.Errorf("Cannot set env %v: configmap %v not found", env.Name, cmKeyRef.Name) + + for _, c := range opts.ConfigMaps { + if cmKeyRef.Name == c.Name { + if value, ok := c.Data[cmKeyRef.Key]; ok { + return value, nil } + err = errors.Errorf("Cannot set env %v: key %s not found in configmap %v", env.Name, cmKeyRef.Key, cmKeyRef.Name) + break + } + } + if cmKeyRef.Optional == nil || !*cmKeyRef.Optional { + return "", err + } + return "", nil + } + + if env.ValueFrom.SecretKeyRef != nil { + secKeyRef := env.ValueFrom.SecretKeyRef + secret, err := k8sSecretFromSecretManager(secKeyRef.Name, opts.SecretsManager) + if err == nil { + if val, ok := secret[secKeyRef.Key]; ok { + return string(val), nil } + err = errors.Errorf("Secret %v has not %v key", secKeyRef.Name, secKeyRef.Key) + } + if secKeyRef.Optional == nil || !*secKeyRef.Optional { + return "", errors.Errorf("Cannot set env %v: %v", env.Name, err) } + return "", nil } } - return env.Value + return env.Value, nil } // getPodPorts converts a slice of kube container descriptions to an diff --git a/pkg/specgen/generate/kube/play_test.go b/pkg/specgen/generate/kube/play_test.go index 148540e9f..f714826f0 100644 --- a/pkg/specgen/generate/kube/play_test.go +++ b/pkg/specgen/generate/kube/play_test.go @@ -1,19 +1,49 @@ package kube import ( + "encoding/json" + "io/ioutil" + "os" "testing" + "github.com/containers/common/pkg/secrets" "github.com/stretchr/testify/assert" v1 "k8s.io/api/core/v1" v12 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -func TestEnvVarsFromConfigMap(t *testing.T) { +func createSecrets(t *testing.T, d string) *secrets.SecretsManager { + secretsManager, err := secrets.NewManager(d) + assert.NoError(t, err) + + driver := "file" + driverOpts := map[string]string{ + "path": d, + } + + for _, s := range k8sSecrets { + data, err := json.Marshal(s.Data) + assert.NoError(t, err) + + _, err = secretsManager.Store(s.ObjectMeta.Name, data, driver, driverOpts) + assert.NoError(t, err) + } + + return secretsManager +} + +func TestEnvVarsFrom(t *testing.T) { + d, err := ioutil.TempDir("", "secrets") + assert.NoError(t, err) + defer os.RemoveAll(d) + secretsManager := createSecrets(t, d) + tests := []struct { - name string - envFrom v1.EnvFromSource - configMapList []v1.ConfigMap - expected map[string]string + name string + envFrom v1.EnvFromSource + options CtrSpecGenOptions + succeed bool + expected map[string]string }{ { "ConfigMapExists", @@ -24,7 +54,10 @@ func TestEnvVarsFromConfigMap(t *testing.T) { }, }, }, - configMapList, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + true, map[string]string{ "myvar": "foo", }, @@ -38,7 +71,26 @@ func TestEnvVarsFromConfigMap(t *testing.T) { }, }, }, - configMapList, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + false, + nil, + }, + { + "OptionalConfigMapDoesNotExist", + v1.EnvFromSource{ + ConfigMapRef: &v1.ConfigMapEnvSource{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + Optional: &optional, + }, + }, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + true, map[string]string{}, }, { @@ -50,7 +102,74 @@ func TestEnvVarsFromConfigMap(t *testing.T) { }, }, }, - []v1.ConfigMap{}, + CtrSpecGenOptions{ + ConfigMaps: []v1.ConfigMap{}, + }, + false, + nil, + }, + { + "OptionalEmptyConfigMapList", + v1.EnvFromSource{ + ConfigMapRef: &v1.ConfigMapEnvSource{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Optional: &optional, + }, + }, + CtrSpecGenOptions{ + ConfigMaps: []v1.ConfigMap{}, + }, + true, + map[string]string{}, + }, + { + "SecretExists", + v1.EnvFromSource{ + SecretRef: &v1.SecretEnvSource{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + true, + map[string]string{ + "myvar": "foo", + }, + }, + { + "SecretDoesNotExist", + v1.EnvFromSource{ + SecretRef: &v1.SecretEnvSource{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + false, + nil, + }, + { + "OptionalSecretDoesNotExist", + v1.EnvFromSource{ + SecretRef: &v1.SecretEnvSource{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + Optional: &optional, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + true, map[string]string{}, }, } @@ -58,18 +177,25 @@ func TestEnvVarsFromConfigMap(t *testing.T) { for _, test := range tests { test := test t.Run(test.name, func(t *testing.T) { - result := envVarsFromConfigMap(test.envFrom, test.configMapList) + result, err := envVarsFrom(test.envFrom, &test.options) + assert.Equal(t, err == nil, test.succeed) assert.Equal(t, test.expected, result) }) } } func TestEnvVarValue(t *testing.T) { + d, err := ioutil.TempDir("", "secrets") + assert.NoError(t, err) + defer os.RemoveAll(d) + secretsManager := createSecrets(t, d) + tests := []struct { - name string - envVar v1.EnvVar - configMapList []v1.ConfigMap - expected string + name string + envVar v1.EnvVar + options CtrSpecGenOptions + succeed bool + expected string }{ { "ConfigMapExists", @@ -84,7 +210,10 @@ func TestEnvVarValue(t *testing.T) { }, }, }, - configMapList, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + true, "foo", }, { @@ -100,7 +229,30 @@ func TestEnvVarValue(t *testing.T) { }, }, }, - configMapList, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + false, + "", + }, + { + "OptionalContainerKeyDoesNotExistInConfigMap", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + ConfigMapKeyRef: &v1.ConfigMapKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Key: "doesnotexist", + Optional: &optional, + }, + }, + }, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + true, "", }, { @@ -116,7 +268,30 @@ func TestEnvVarValue(t *testing.T) { }, }, }, - configMapList, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + false, + "", + }, + { + "OptionalConfigMapDoesNotExist", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + ConfigMapKeyRef: &v1.ConfigMapKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + Key: "myvar", + Optional: &optional, + }, + }, + }, + CtrSpecGenOptions{ + ConfigMaps: configMapList, + }, + true, "", }, { @@ -132,7 +307,127 @@ func TestEnvVarValue(t *testing.T) { }, }, }, - []v1.ConfigMap{}, + CtrSpecGenOptions{ + ConfigMaps: []v1.ConfigMap{}, + }, + false, + "", + }, + { + "OptionalEmptyConfigMapList", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + ConfigMapKeyRef: &v1.ConfigMapKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Key: "myvar", + Optional: &optional, + }, + }, + }, + CtrSpecGenOptions{ + ConfigMaps: []v1.ConfigMap{}, + }, + true, + "", + }, + { + "SecretExists", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + SecretKeyRef: &v1.SecretKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Key: "myvar", + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + true, + "foo", + }, + { + "ContainerKeyDoesNotExistInSecret", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + SecretKeyRef: &v1.SecretKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Key: "doesnotexist", + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + false, + "", + }, + { + "OptionalContainerKeyDoesNotExistInSecret", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + SecretKeyRef: &v1.SecretKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "foo", + }, + Key: "doesnotexist", + Optional: &optional, + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + true, + "", + }, + { + "SecretDoesNotExist", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + SecretKeyRef: &v1.SecretKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + Key: "myvar", + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + false, + "", + }, + { + "OptionalSecretDoesNotExist", + v1.EnvVar{ + Name: "FOO", + ValueFrom: &v1.EnvVarSource{ + SecretKeyRef: &v1.SecretKeySelector{ + LocalObjectReference: v1.LocalObjectReference{ + Name: "doesnotexist", + }, + Key: "myvar", + Optional: &optional, + }, + }, + }, + CtrSpecGenOptions{ + SecretsManager: secretsManager, + }, + true, "", }, } @@ -140,7 +435,8 @@ func TestEnvVarValue(t *testing.T) { for _, test := range tests { test := test t.Run(test.name, func(t *testing.T) { - result := envVarValue(test.envVar, test.configMapList) + result, err := envVarValue(test.envVar, &test.options) + assert.Equal(t, err == nil, test.succeed) assert.Equal(t, test.expected, result) }) } @@ -170,3 +466,30 @@ var configMapList = []v1.ConfigMap{ }, }, } + +var optional = true + +var k8sSecrets = []v1.Secret{ + { + TypeMeta: v12.TypeMeta{ + Kind: "Secret", + }, + ObjectMeta: v12.ObjectMeta{ + Name: "bar", + }, + Data: map[string][]byte{ + "myvar": []byte("bar"), + }, + }, + { + TypeMeta: v12.TypeMeta{ + Kind: "Secret", + }, + ObjectMeta: v12.ObjectMeta{ + Name: "foo", + }, + Data: map[string][]byte{ + "myvar": []byte("foo"), + }, + }, +} diff --git a/pkg/specgen/specgen.go b/pkg/specgen/specgen.go index 732579bf0..c10dc5ef5 100644 --- a/pkg/specgen/specgen.go +++ b/pkg/specgen/specgen.go @@ -5,7 +5,7 @@ import ( "syscall" "github.com/containers/image/v5/manifest" - "github.com/containers/storage" + "github.com/containers/storage/types" spec "github.com/opencontainers/runtime-spec/specs-go" "github.com/pkg/errors" ) @@ -302,7 +302,7 @@ type ContainerSecurityConfig struct { // IDMappings are UID and GID mappings that will be used by user // namespaces. // Required if UserNS is private. - IDMappings *storage.IDMappingOptions `json:"idmappings,omitempty"` + IDMappings *types.IDMappingOptions `json:"idmappings,omitempty"` // ReadOnlyFilesystem indicates that everything will be mounted // as read-only ReadOnlyFilesystem bool `json:"read_only_filesystem,omitempty"` diff --git a/pkg/util/utils.go b/pkg/util/utils.go index a4c8f3a64..bbaf72981 100644 --- a/pkg/util/utils.go +++ b/pkg/util/utils.go @@ -20,8 +20,8 @@ import ( "github.com/containers/podman/v3/pkg/namespaces" "github.com/containers/podman/v3/pkg/rootless" "github.com/containers/podman/v3/pkg/signal" - "github.com/containers/storage" "github.com/containers/storage/pkg/idtools" + stypes "github.com/containers/storage/types" v1 "github.com/opencontainers/image-spec/specs-go/v1" "github.com/opencontainers/runtime-spec/specs-go" "github.com/pkg/errors" @@ -344,8 +344,8 @@ func ParseSignal(rawSignal string) (syscall.Signal, error) { } // GetKeepIDMapping returns the mappings and the user to use when keep-id is used -func GetKeepIDMapping() (*storage.IDMappingOptions, int, int, error) { - options := storage.IDMappingOptions{ +func GetKeepIDMapping() (*stypes.IDMappingOptions, int, int, error) { + options := stypes.IDMappingOptions{ HostUIDMapping: true, HostGIDMapping: true, } @@ -395,8 +395,8 @@ func GetKeepIDMapping() (*storage.IDMappingOptions, int, int, error) { } // ParseIDMapping takes idmappings and subuid and subgid maps and returns a storage mapping -func ParseIDMapping(mode namespaces.UsernsMode, uidMapSlice, gidMapSlice []string, subUIDMap, subGIDMap string) (*storage.IDMappingOptions, error) { - options := storage.IDMappingOptions{ +func ParseIDMapping(mode namespaces.UsernsMode, uidMapSlice, gidMapSlice []string, subUIDMap, subGIDMap string) (*stypes.IDMappingOptions, error) { + options := stypes.IDMappingOptions{ HostUIDMapping: true, HostGIDMapping: true, } @@ -479,7 +479,7 @@ type tomlConfig struct { } `toml:"storage"` } -func getTomlStorage(storeOptions *storage.StoreOptions) *tomlConfig { +func getTomlStorage(storeOptions *stypes.StoreOptions) *tomlConfig { config := new(tomlConfig) config.Storage.Driver = storeOptions.GraphDriverName @@ -496,7 +496,7 @@ func getTomlStorage(storeOptions *storage.StoreOptions) *tomlConfig { } // WriteStorageConfigFile writes the configuration to a file -func WriteStorageConfigFile(storageOpts *storage.StoreOptions, storageConf string) error { +func WriteStorageConfigFile(storageOpts *stypes.StoreOptions, storageConf string) error { if err := os.MkdirAll(filepath.Dir(storageConf), 0755); err != nil { return err } diff --git a/test/apiv2/01-basic.at b/test/apiv2/01-basic.at index 788007069..64aafa013 100644 --- a/test/apiv2/01-basic.at +++ b/test/apiv2/01-basic.at @@ -16,13 +16,13 @@ t GET libpod/_ping 200 OK t HEAD libpod/_ping 200 for i in /version version; do - t GET $i 200 \ - .Components[0].Name="Podman Engine" \ - .Components[0].Details.APIVersion=3.1.0-dev \ - .Components[0].Details.MinAPIVersion=3.0.0 \ - .Components[0].Details.Os=linux \ - .ApiVersion=1.40 \ - .MinAPIVersion=1.24 \ + t GET $i 200 \ + .Components[0].Name="Podman Engine" \ + .Components[0].Details.APIVersion~3[0-9.-]\\+ \ + .Components[0].Details.MinAPIVersion=3.1.0 \ + .Components[0].Details.Os=linux \ + .ApiVersion=1.40 \ + .MinAPIVersion=1.24 \ .Os=linux done diff --git a/test/buildah-bud/buildah-tests.diff b/test/buildah-bud/buildah-tests.diff index 1d14f2a6a..ad35e5926 100644 --- a/test/buildah-bud/buildah-tests.diff +++ b/test/buildah-bud/buildah-tests.diff @@ -1,16 +1,16 @@ -From 6f8e097d8b46190df60e830adce1185532a939d0 Mon Sep 17 00:00:00 2001 +From a49a2e48421c6f3bb1a56ae372de1f3d1a45d1f1 Mon Sep 17 00:00:00 2001 From: Ed Santiago <santiago@redhat.com> Date: Tue, 9 Feb 2021 17:28:05 -0700 Subject: [PATCH] tweaks for running buildah tests under podman Signed-off-by: Ed Santiago <santiago@redhat.com> --- - tests/bud.bats | 27 +++++++++++++++++---------- + tests/bud.bats | 26 ++++++++++++++++---------- tests/helpers.bash | 28 ++++++++++++++++++++++++---- - 2 files changed, 41 insertions(+), 14 deletions(-) + 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/tests/bud.bats b/tests/bud.bats -index cf55d9a4..e55b05b5 100644 +index cf55d9a4..60cb6f96 100644 --- a/tests/bud.bats +++ b/tests/bud.bats @@ -4,7 +4,7 @@ load helpers @@ -103,15 +103,7 @@ index cf55d9a4..e55b05b5 100644 } @test "bud with additional directory of devices" { -@@ -2115,6 +2120,7 @@ _EOF - } - - @test "bud pull never" { -+ skip "FIXME: podman issue #9573" - target=pull - run_buildah 125 bud --signature-policy ${TESTSDIR}/policy.json -t ${target} --pull-never ${TESTSDIR}/bud/pull - expect_output --substring "pull policy is \"never\" but \"" -@@ -2134,6 +2140,7 @@ _EOF +@@ -2134,6 +2139,7 @@ _EOF } @test "bud with Containerfile should fail with nonexistent authfile" { @@ -119,7 +111,7 @@ index cf55d9a4..e55b05b5 100644 target=alpine-image run_buildah 125 bud --authfile /tmp/nonexistent --signature-policy ${TESTSDIR}/policy.json -t ${target} ${TESTSDIR}/bud/containerfile } -@@ -2261,6 +2268,7 @@ EOM +@@ -2261,6 +2267,7 @@ EOM } @test "bud with encrypted FROM image" { @@ -127,7 +119,7 @@ index cf55d9a4..e55b05b5 100644 _prefetch busybox mkdir ${TESTDIR}/tmp openssl genrsa -out ${TESTDIR}/tmp/mykey.pem 1024 -@@ -2333,8 +2341,6 @@ EOM +@@ -2333,8 +2340,6 @@ EOM _prefetch alpine run_buildah bud --timestamp=0 --quiet --pull=false --signature-policy ${TESTSDIR}/policy.json -t timestamp -f Dockerfile.1 ${TESTSDIR}/bud/cache-stages cid=$output @@ -136,7 +128,7 @@ index cf55d9a4..e55b05b5 100644 run_buildah inspect --format '{{ .OCIv1.Created }}' timestamp expect_output --substring "1970-01-01" run_buildah inspect --format '{{ .History }}' timestamp -@@ -2594,6 +2600,7 @@ _EOF +@@ -2594,6 +2599,7 @@ _EOF } @test "bud with --arch flag" { diff --git a/test/e2e/play_kube_test.go b/test/e2e/play_kube_test.go index a4c738f17..93c8426a7 100644 --- a/test/e2e/play_kube_test.go +++ b/test/e2e/play_kube_test.go @@ -141,7 +141,16 @@ spec: configMapKeyRef: name: {{ .RefName }} key: {{ .RefKey }} - {{ else }} + optional: {{ .Optional }} + {{ end }} + {{ if (eq .ValueFrom "secret") }} + valueFrom: + secretKeyRef: + name: {{ .RefName }} + key: {{ .RefKey }} + optional: {{ .Optional }} + {{ end }} + {{ if (eq .ValueFrom "") }} value: {{ .Value }} {{ end }} {{ end }} @@ -151,6 +160,12 @@ spec: {{ if (eq .From "configmap") }} - configMapRef: name: {{ .Name }} + optional: {{ .Optional }} + {{ end }} + {{ if (eq .From "secret") }} + - secretRef: + name: {{ .Name }} + optional: {{ .Optional }} {{ end }} {{ end }} {{ end }} @@ -340,6 +355,8 @@ var ( seccompPwdEPERM = []byte(`{"defaultAction":"SCMP_ACT_ALLOW","syscalls":[{"name":"getcwd","action":"SCMP_ACT_ERRNO"}]}`) // CPU Period in ms defaultCPUPeriod = 100 + // Default secret in JSON. Note that the values ("foo" and "bar") are base64 encoded. + defaultSecret = []byte(`{"FOO":"Zm9v","BAR":"YmFy"}`) ) func writeYaml(content string, fileName string) error { @@ -407,6 +424,16 @@ func generateMultiDocKubeYaml(kubeObjects []string, pathname string) error { return writeYaml(multiKube, pathname) } +func createSecret(podmanTest *PodmanTestIntegration, name string, value []byte) { + secretFilePath := filepath.Join(podmanTest.TempDir, "secret") + err := ioutil.WriteFile(secretFilePath, value, 0755) + Expect(err).To(BeNil()) + + secret := podmanTest.Podman([]string{"secret", "create", name, secretFilePath}) + secret.WaitWithDefaultTimeout() + Expect(secret.ExitCode()).To(Equal(0)) +} + // ConfigMap describes the options a kube yaml can be configured at configmap level type ConfigMap struct { Name string @@ -746,7 +773,7 @@ func withVolumeMount(mountPath string, readonly bool) ctrOption { } } -func withEnv(name, value, valueFrom, refName, refKey string) ctrOption { +func withEnv(name, value, valueFrom, refName, refKey string, optional bool) ctrOption { return func(c *Ctr) { e := Env{ Name: name, @@ -754,17 +781,19 @@ func withEnv(name, value, valueFrom, refName, refKey string) ctrOption { ValueFrom: valueFrom, RefName: refName, RefKey: refKey, + Optional: optional, } c.Env = append(c.Env, e) } } -func withEnvFrom(name, from string) ctrOption { +func withEnvFrom(name, from string, optional bool) ctrOption { return func(c *Ctr) { e := EnvFrom{ - Name: name, - From: from, + Name: name, + From: from, + Optional: optional, } c.EnvFrom = append(c.EnvFrom, e) @@ -822,11 +851,13 @@ type Env struct { ValueFrom string RefName string RefKey string + Optional bool } type EnvFrom struct { - Name string - From string + Name string + From string + Optional bool } func milliCPUToQuota(milliCPU string) int { @@ -1062,7 +1093,7 @@ var _ = Describe("Podman play kube", func() { err := generateKubeYaml("configmap", cm, cmYamlPathname) Expect(err).To(BeNil()) - pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "foo", "FOO")))) + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "foo", "FOO", false)))) err = generateKubeYaml("pod", pod, kubeYaml) Expect(err).To(BeNil()) @@ -1076,6 +1107,68 @@ var _ = Describe("Podman play kube", func() { Expect(inspect.OutputToString()).To(ContainSubstring(`FOO=foo`)) }) + It("podman play kube test required env value from configmap with missing key", func() { + SkipIfRemote("configmap list is not supported as a param") + cmYamlPathname := filepath.Join(podmanTest.TempDir, "foo-cm.yaml") + cm := getConfigMap(withConfigMapName("foo"), withConfigMapData("FOO", "foo")) + err := generateKubeYaml("configmap", cm, cmYamlPathname) + Expect(err).To(BeNil()) + + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "foo", "MISSING_KEY", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml, "--configmap", cmYamlPathname}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test required env value from missing configmap", func() { + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "missing_cm", "FOO", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test optional env value from configmap with missing key", func() { + SkipIfRemote("configmap list is not supported as a param") + cmYamlPathname := filepath.Join(podmanTest.TempDir, "foo-cm.yaml") + cm := getConfigMap(withConfigMapName("foo"), withConfigMapData("FOO", "foo")) + err := generateKubeYaml("configmap", cm, cmYamlPathname) + Expect(err).To(BeNil()) + + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "foo", "MISSING_KEY", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml, "--configmap", cmYamlPathname}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ range .Config.Env }}[{{ . }}]{{end}}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`[FOO=]`)) + }) + + It("podman play kube test optional env value from missing configmap", func() { + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "configmap", "missing_cm", "FOO", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ range .Config.Env }}[{{ . }}]{{end}}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`[FOO=]`)) + }) + It("podman play kube test get all key-value pairs from configmap as envs", func() { SkipIfRemote("configmap list is not supported as a param") cmYamlPathname := filepath.Join(podmanTest.TempDir, "foo-cm.yaml") @@ -1083,7 +1176,7 @@ var _ = Describe("Podman play kube", func() { err := generateKubeYaml("configmap", cm, cmYamlPathname) Expect(err).To(BeNil()) - pod := getPod(withCtr(getCtr(withEnvFrom("foo", "configmap")))) + pod := getPod(withCtr(getCtr(withEnvFrom("foo", "configmap", false)))) err = generateKubeYaml("pod", pod, kubeYaml) Expect(err).To(BeNil()) @@ -1098,6 +1191,131 @@ var _ = Describe("Podman play kube", func() { Expect(inspect.OutputToString()).To(ContainSubstring(`FOO2=foo2`)) }) + It("podman play kube test get all key-value pairs from required configmap as envs", func() { + pod := getPod(withCtr(getCtr(withEnvFrom("missing_cm", "configmap", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test get all key-value pairs from optional configmap as envs", func() { + pod := getPod(withCtr(getCtr(withEnvFrom("missing_cm", "configmap", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + }) + + It("podman play kube test env value from secret", func() { + createSecret(podmanTest, "foo", defaultSecret) + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "secret", "foo", "FOO", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ .Config.Env }}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`FOO=foo`)) + }) + + It("podman play kube test required env value from missing secret", func() { + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "secret", "foo", "FOO", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test required env value from secret with missing key", func() { + createSecret(podmanTest, "foo", defaultSecret) + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "secret", "foo", "MISSING", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test optional env value from missing secret", func() { + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "secret", "foo", "FOO", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ range .Config.Env }}[{{ . }}]{{end}}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`[FOO=]`)) + }) + + It("podman play kube test optional env value from secret with missing key", func() { + createSecret(podmanTest, "foo", defaultSecret) + pod := getPod(withCtr(getCtr(withEnv("FOO", "", "secret", "foo", "MISSING", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ range .Config.Env }}[{{ . }}]{{end}}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`[FOO=]`)) + }) + + It("podman play kube test get all key-value pairs from secret as envs", func() { + createSecret(podmanTest, "foo", defaultSecret) + pod := getPod(withCtr(getCtr(withEnvFrom("foo", "secret", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + + inspect := podmanTest.Podman([]string{"inspect", getCtrNameInPod(pod), "--format", "'{{ .Config.Env }}'"}) + inspect.WaitWithDefaultTimeout() + Expect(inspect.ExitCode()).To(Equal(0)) + Expect(inspect.OutputToString()).To(ContainSubstring(`FOO=foo`)) + Expect(inspect.OutputToString()).To(ContainSubstring(`BAR=bar`)) + }) + + It("podman play kube test get all key-value pairs from required secret as envs", func() { + pod := getPod(withCtr(getCtr(withEnvFrom("missing_secret", "secret", false)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Not(Equal(0))) + }) + + It("podman play kube test get all key-value pairs from optional secret as envs", func() { + pod := getPod(withCtr(getCtr(withEnvFrom("missing_secret", "secret", true)))) + err = generateKubeYaml("pod", pod, kubeYaml) + Expect(err).To(BeNil()) + + kube := podmanTest.Podman([]string{"play", "kube", kubeYaml}) + kube.WaitWithDefaultTimeout() + Expect(kube.ExitCode()).To(Equal(0)) + }) + It("podman play kube test hostname", func() { pod := getPod() err := generateKubeYaml("pod", pod, kubeYaml) diff --git a/version/version.go b/version/version.go index 6b93ed8ea..1cbd9e309 100644 --- a/version/version.go +++ b/version/version.go @@ -27,7 +27,7 @@ const ( // NOTE: remember to bump the version at the top // of the top-level README.md file when this is // bumped. -var Version = semver.MustParse("3.1.0-dev") +var Version = semver.MustParse("3.2.0-dev") // See https://docs.docker.com/engine/api/v1.40/ // libpod compat handlers are expected to honor docker API versions @@ -38,7 +38,7 @@ var Version = semver.MustParse("3.1.0-dev") var APIVersion = map[Tree]map[Level]semver.Version{ Libpod: { CurrentAPI: Version, - MinimalAPI: semver.MustParse("3.0.0"), + MinimalAPI: semver.MustParse("3.1.0"), }, Compat: { CurrentAPI: semver.MustParse("1.40.0"), |