diff options
90 files changed, 1430 insertions, 342 deletions
@@ -5,7 +5,7 @@ Podman (the POD MANager) is a tool for managing containers and images, volumes mounted into those containers, and pods made from groups of containers. Podman is based on libpod, a library for container lifecycle management that is also contained in this repository. The libpod library provides APIs for managing containers, pods, container images, and volumes. -* [Latest Version: 3.1.0](https://github.com/containers/podman/releases/latest) +* [Latest Version: 3.2.0](https://github.com/containers/podman/releases/latest) * Latest Remote client for Windows * Latest Remote client for MacOs * Latest Static Remote client for Linux diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 5ba5e251b..c6efff5dd 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,5 +1,130 @@ # Release Notes +## 3.2.0 +### Features +- Docker Compose is now supported with rootless Podman ([#9169](https://github.com/containers/podman/issues/9169)). +- The `podman network connect`, `podman network disconnect`, and `podman network reload` commands have been enabled for rootless Podman. +- An experimental new set of commands, `podman machine`, was added to assist in managing virtual machines containing a Podman server. These are intended for easing the use of Podman on OS X by handling the creation of a Linux VM for running Podman. +- The `podman generate kube` command can now be run on Podman named volumes (generating `PersistentVolumeClaim` YAML), in addition to pods and containers. +- The `podman play kube` command now supports two new options, `--ip` and `--mac`, to set static IPs and MAC addresses for created pods ([#8442](https://github.com/containers/podman/issues/8442) and [#9731](https://github.com/containers/podman/issues/9731)). +- The `podman play kube` command's support for `PersistentVolumeClaim` YAML has been greatly improved. +- The `podman generate kube` command now preserves the label used by `podman auto-update` to identify containers to update as a Kubernetes annotation, and the `podman play kube` command will convert this annotation back into a label. This allows `podman auto-update` to be used with containers created by `podman play kube`. +- The `podman play kube` command now supports Kubernetes `secretRef` YAML (using the secrets support from `podman secret`) for environment variables. +- Secrets can now be added to containers as environment variables using the `type=env` option to the `--secret` flag to `podman create` and `podman run`. +- The `podman start` command now supports the `--all` option, allowing all containers to be started simultaneously with a single command. The `--filter` option has also been added to filter which containers to start when `--all` is used. +- Filtering containers with the `--filter` option to `podman ps` and `podman start` now supports a new filter, `restart-policy`, to filter containers based on their restart policy. +- The `--group-add` option to rootless `podman run` and `podman create` now accepts a new value, `keep-groups`, which instructs Podman to retain the supplemental groups of the user running Podman in the created container. This is only supported with the `crun` OCI runtime. +- The `podman run` and `podman create` commands now support a new option, `--timeout`. This sets a maximum time the container is allowed to run, after which it is killed ([#6412](https://github.com/containers/podman/issues/6412)). +- The `podman run` and `podman create` commands now support a new option, `--pidfile`. This will create a file when the container is started containing the PID of the first process in the container. +- The `podman run` and `podman create` commands now support a new option, `--requires`. The `--requires` option adds dependency containers - containers that must be running before the current container. Commands like `podman start` will automatically start the requirements of a container before starting the container itself. +- Auto-updating containers can now be done with locally-built images, not just images hosted on a registry, by creating containers with the `io.containers.autoupdate` label set to `local`. +- Podman now supports the [Container Device Interface](https://github.com/container-orchestrated-devices/container-device-interface) (CDI) standard. +- Podman now adds an entry to `/etc/hosts`, `host.containers.internal`, pointing to the current gateway (which, for root containers, is usually a bridge interface on the host system) ([#5651](https://github.com/containers/podman/issues/5651)). +- The `podman ps`, `podman pod ps`, `podman network list`, `podman secret list`, and `podman volume list` commands now support a `--noheading` option, which will cause Podman to omit the heading line including column names. +- The `podman unshare` command now supports a new flag, `--rootless-cni`, to join the rootless network namespace. This allows commands to be run in the same network environment as rootless containers with CNI networking. +- The `--security-opt unmask=` option to `podman run` and `podman create` now supports glob operations to unmask a group of paths at once (e.g. `podman run --security-opt unmask=/proc/* ...` will unmask all paths in `/proc` in the container). +- The `podman network prune` command now supports a `--filter` option to filter which networks will be pruned. + +### Changes +- The change in Podman 3.1.2 where the `:z` and `:Z` mount options for volumes were ignored for privileged containers has been reverted after discussion in [#10209](https://github.com/containers/podman/issues/10209). +- Podman's rootless CNI functionality no longer requires a sidecar container! The removal of the requirement for the `rootless-cni-infra` container means that rootless CNI is now usable on all architectures, not just AMD64, and no longer requires pulling an image ([#8709](https://github.com/containers/podman/issues/8709)). +- The Image handling code used by Podman has seen a major rewrite to improve code sharing with our other projects, Buildah and CRI-O. This should result in fewer bugs and performance gains in the long term. Work on this is still ongoing. +- The `podman auto-update` command now prunes previous versions of images after updating if they are unused, to prevent disk exhaustion after repeated updates ([#10190](https://github.com/containers/podman/issues/10190)). +- The `podman play kube` now treats environment variables configured as references to a `ConfigMap` as mandatory unless the `optional` parameter was set; this better matches the behavior of Kubernetes. +- Podman now supports the `--context=default` flag from Docker as a no-op for compatibility purposes. +- When Podman is run as root, but without `CAP_SYS_ADMIN` being available, it will run in a user namespace using the same code as rootless Podman (instead of failing outright). +- The `podman info` command now includes the path of the Seccomp profile Podman is using, available cgroup controllers, and whether Podman is connected to a remote service or running containers locally. +- Containers created with the `--rm` option now automatically use the `volatile` storage flag when available for their root filesystems, causing them not to write changes to disk as often as they will be removed at completion anyways. This should result in improved performance. +- The `podman generate systemd --new` command will now include environment variables referenced by the container in generated unit files if the value would be looked up from the system environment. +- Podman now requires that Conmon v2.0.24 be available. + +### Bugfixes +- Fixed a bug where the remote Podman client's `podman build` command did not support the `--arch`, `--platform`, and `--os`, options. +- Fixed a bug where the remote Podman client's `podman build` command ignored the `--rm=false` option ([#9869](https://github.com/containers/podman/issues/9869)). +- Fixed a bug where the remote Podman client's `podman build --iidfile` command could include extra output (in addition to just the image ID) in the image ID file written ([#10233](https://github.com/containers/podman/issues/10233)). +- Fixed a bug where the remote Podman client's `podman build` command did not preserve hardlinks when moving files into the container via `COPY` instructions ([#9893](https://github.com/containers/podman/issues/9893)). +- Fixed a bug where the `podman generate systemd --new` command could generate extra `--iidfile` arguments if the container was already created with one. +- Fixed a bug where the `podman generate systemd --new` command would generate unit files that did not include `RequiresMountsFor` lines ([#10493](https://github.com/containers/podman/issues/10493)). +- Fixed a bug where the `podman generate kube` command produced incorrect YAML for containers which bind-mounted both `/` and `/root` from the host system into the container ([#9764](https://github.com/containers/podman/issues/9764)). +- Fixed a bug where pods created by `podman play kube` from YAML that specified `ShareProcessNamespace` would only share the PID namespace (and not also the UTS, Network, and IPC namespaces) ([#9128](https://github.com/containers/podman/issues/9128)). +- Fixed a bug where the `podman network reload` command could generate spurious error messages when `iptables-nft` was in use. +- Fixed a bug where rootless Podman could fail to attach to containers when the user running Podman had a large UID. +- Fixed a bug where the `podman ps` command could fail with a `no such container` error due to a race condition with container removal ([#10120](https://github.com/containers/podman/issues/10120)). +- Fixed a bug where containers using the `slirp4netns` network mode and setting a custom `slirp4netns` subnet while using the `rootlesskit` port forwarder would not be able to forward ports ([#9828](https://github.com/containers/podman/issues/9828)). +- Fixed a bug where the `--filter ancestor=` option to `podman ps` did not require an exact match of the image name/ID to include a container in its results. +- Fixed a bug where the `--filter until=` option to `podman image prune` would prune images created after the specified time (instead of before). +- Fixed a bug where setting a custom Seccomp profile via the `seccomp_profile` option in `containers.conf` had no effect, and the default profile was used instead. +- Fixed a bug where the `--cgroup-parent` option to `podman create` and `podman run` was ignored in rootless Podman on cgroups v2 systems with the `cgroupfs` cgroup manager ([#10173](https://github.com/containers/podman/issues/10173)). +- Fixed a bug where the `IMAGE` and `NAME` variables in `podman container runlabel` were not being correctly substituted ([#10192](https://github.com/containers/podman/issues/10192)). +- Fixed a bug where Podman could freeze when creating containers with a specific combination of volumes and working directory ([#10216](https://github.com/containers/podman/issues/10216)). +- Fixed a bug where rootless Podman containers restarted by restart policy (e.g. containers created with `--restart=always`) would lose networking after being restarted ([#8047](https://github.com/containers/podman/issues/8047)). +- Fixed a bug where the `podman cp` command could not copy files into containers created with the `--pid=host` flag ([#9985](https://github.com/containers/podman/issues/9985)). +- Fixed a bug where filters to the `podman events` command could not be specified twice (if a filter is specified more than once, it will match if any of the given values match - logical or) ([#10507](https://github.com/containers/podman/issues/10507)). +- Fixed a bug where Podman would include IPv6 nameservers in `resolv.conf` in containers without IPv6 connectivity ([#10158](https://github.com/containers/podman/issues/10158)). +- Fixed a bug where containers could not be created with static IP addresses when connecting to a network using the `macvlan` driver ([#10283](https://github.com/containers/podman/issues/10283)). + +### API +- Fixed a bug where the Compat Create endpoint for Containers did not allow advanced network options to be set ([#10110](https://github.com/containers/podman/issues/10110)). +- Fixed a bug where the Compat Create endpoint for Containers ignored static IP information provided in the `IPAMConfig` block ([#10245](https://github.com/containers/podman/issues/10245)). +- Fixed a bug where the Compat Inspect endpoint for Containers returned null (instead of an empty list) for Networks when the container was not joined to a CNI network ([#9837](https://github.com/containers/podman/issues/9837)). +- Fixed a bug where the Compat Wait endpoint for Containers could miss containers exiting if they were immediately restarted. +- Fixed a bug where the Compat Create endpoint for Volumes required that the user provide a name for the new volume ([#9803](https://github.com/containers/podman/issues/9803)). +- Fixed a bug where the Libpod Info handler would sometimes not return the correct path to the Podman API socket. +- Fixed a bug where the Compat Events handler used the wrong name for container exited events (`died` instead of `die`) ([#10168](https://github.com/containers/podman/issues/10168)). +- Fixed a bug where the Compat Push endpoint for Images could leak goroutines if the remote end closed the connection prematurely. + +### Misc +- Updated Buildah to v1.21.0 +- Updated the containers/common library to v0.38.5 +- Updated the containers/storage library to v1.31.3 + +## 3.1.2 +### Bugfixes +- The Compat Export endpoint for Images now supports exporting multiple images at the same time to a single archive. +- Fixed a bug where images with empty layers were stored incorrectly, causing them to be unable to be pushed or saved. +- Fixed a bug where the `podman rmi` command could fail to remove corrupt images from storage. +- Fixed a bug where the remote Podman client's `podman save` command did not support the `oci-dir` and `docker-dir` formats ([#9742](https://github.com/containers/podman/issues/9742)). +- Fixed a bug where volume mounts from `podman play kube` created with a trailing `/` in the container path were were not properly superceding named volumes from the image ([#9618](https://github.com/containers/podman/issues/9618)). +- Fixed a bug where Podman could fail to build on 32-bit architectures. + +### Misc +- Updated the containers/image library to v5.11.1 + +## 3.1.1 +### Changes +- Podman now recognizes `trace` as a valid argument to the `--log-level` command. Trace logging is now the most verbose level of logging available. +- The `:z` and `:Z` options for volume mounts are now ignored when the container is privileged or is run with SELinux isolation disabled (`--security-opt label=disable`). This matches better matches Docker's behavior in this case. + +### Bugfixes +- Fixed a bug where pruning images with the `podman image prune` or `podman system prune` commands could cause Podman to panic. +- Fixed a bug where the `podman save` command did not properly error when the `--compress` flag was used with incompatible format types. +- Fixed a bug where the `--security-opt` and `--ulimit` options to the remote Podman client's `podman build` command were nonfunctional. +- Fixed a bug where the `--log-rusage` option to the remote Podman client's `podman build` command was nonfunctional ([#9489](https://github.com/containers/podman/issues/9889)). +- Fixed a bug where the `podman build` command could, in some circumstances, use the wrong OCI runtime ([#9459](https://github.com/containers/podman/issues/9459)). +- Fixed a bug where the remote Podman client's `podman build` command could return 0 despite failing ([#10029](https://github.com/containers/podman/issues/10029)). +- Fixed a bug where the `podman container runlabel` command did not properly expand the `IMAGE` and `NAME` variables in the label ([#9405](https://github.com/containers/podman/issues/9405)). +- Fixed a bug where poststop OCI hooks would be executed twice on containers started with the `--rm` argument ([#9983](https://github.com/containers/podman/issues/9983)). +- Fixed a bug where rootless Podman could fail to launch containers on cgroups v2 systems when the `cgroupfs` cgroup manager was in use. +- Fixed a bug where the `podman stats` command could error when statistics tracked exceeded the maximum size of a 32-bit signed integer ([#9979](https://github.com/containers/podman/issues/9979)). +- Fixed a bug where rootless Podman containers run with `--userns=keepid` (without a `--user` flag in addition) would grant exec sessions run in them too many capabilities ([#9919](https://github.com/containers/podman/issues/9919)). +- Fixed a bug where the `--authfile` option to `podman build` did not validate that the path given existed ([#9572](https://github.com/containers/podman/issues/9572)). +- Fixed a bug where the `--storage-opt` option to Podman was appending to, instead of overriding (as is documented), the default storage options. +- Fixed a bug where the `podman system service` connection did not function properly when run in a socket-activated systemd unit file as a non-root user. +- Fixed a bug where the `--network` option to the `podman play kube` command of the remote Podman client was being ignored ([#9698](https://github.com/containers/podman/issues/9698)). +- Fixed a bug where the `--log-driver` option to the `podman play kube` command was nonfunctional ([#10015](https://github.com/containers/podman/issues/10015)). + +### API +- Fixed a bug where the Libpod Create endpoint for Manifests did not properly validate the image the manifest was being created with. +- Fixed a bug where the Libpod DF endpoint could, in error cases, append an extra null to the JSON response, causing decode errors. +- Fixed a bug where the Libpod and Compat Top endpoint for Containers would return process names that included extra whitespace. +- Fixed a bug where the Compat Prune endpoint for Containers accepted too many types of filter. + +### Misc +- Updated Buildah to v1.20.1 +- Updated the containers/storage library to v1.29.0 +- Updated the containers/image library to v5.11.0 +- Updated the containers/common library to v0.36.0 + ## 3.1.0 ### Features - A set of new commands has been added to manage secrets! The `podman secret create`, `podman secret inspect`, `podman secret ls` and `podman secret rm` commands have been added to handle secrets, along with the `--secret` option to `podman run` and `podman create` to add secrets to containers. The initial driver for secrets does not support encryption - this will be added in a future release. diff --git a/cmd/podman/common/completion.go b/cmd/podman/common/completion.go index de5b2995a..c93f2017c 100644 --- a/cmd/podman/common/completion.go +++ b/cmd/podman/common/completion.go @@ -1211,3 +1211,10 @@ func AutocompleteVolumeFilters(cmd *cobra.Command, args []string, toComplete str } return completeKeyValues(toComplete, kv) } + +// AutocompleteCheckpointCompressType - Autocomplete checkpoint compress type options. +// -> "gzip", "none", "zstd" +func AutocompleteCheckpointCompressType(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { + types := []string{"gzip", "none", "zstd"} + return types, cobra.ShellCompDirectiveNoFileComp +} diff --git a/cmd/podman/common/netflags.go b/cmd/podman/common/netflags.go index 4f634f355..78cfe2f13 100644 --- a/cmd/podman/common/netflags.go +++ b/cmd/podman/common/netflags.go @@ -170,7 +170,7 @@ func NetFlagsToNetOptions(cmd *cobra.Command, netnsFromConfig bool) (*entities.N return nil, err } if len(inputPorts) > 0 { - opts.PublishPorts, err = createPortBindings(inputPorts) + opts.PublishPorts, err = CreatePortBindings(inputPorts) if err != nil { return nil, err } diff --git a/cmd/podman/common/util.go b/cmd/podman/common/util.go index afee55914..6a0af4dff 100644 --- a/cmd/podman/common/util.go +++ b/cmd/podman/common/util.go @@ -89,8 +89,8 @@ func createExpose(expose []string) (map[uint16]string, error) { return toReturn, nil } -// createPortBindings iterates ports mappings into SpecGen format. -func createPortBindings(ports []string) ([]specgen.PortMapping, error) { +// CreatePortBindings iterates ports mappings into SpecGen format. +func CreatePortBindings(ports []string) ([]specgen.PortMapping, error) { // --publish is formatted as follows: // [[hostip:]hostport[-endPort]:]containerport[-endPort][/protocol] toReturn := make([]specgen.PortMapping, 0, len(ports)) diff --git a/cmd/podman/containers/checkpoint.go b/cmd/podman/containers/checkpoint.go index 47d60453b..4fa72d520 100644 --- a/cmd/podman/containers/checkpoint.go +++ b/cmd/podman/containers/checkpoint.go @@ -3,6 +3,7 @@ package containers import ( "context" "fmt" + "strings" "github.com/containers/common/pkg/completion" "github.com/containers/podman/v3/cmd/podman/common" @@ -11,6 +12,7 @@ import ( "github.com/containers/podman/v3/cmd/podman/validate" "github.com/containers/podman/v3/pkg/domain/entities" "github.com/containers/podman/v3/pkg/rootless" + "github.com/containers/storage/pkg/archive" "github.com/pkg/errors" "github.com/spf13/cobra" ) @@ -36,9 +38,7 @@ var ( } ) -var ( - checkpointOptions entities.CheckpointOptions -) +var checkpointOptions entities.CheckpointOptions func init() { registry.Commands = append(registry.Commands, registry.CliCommand{ @@ -60,11 +60,32 @@ func init() { flags.BoolVarP(&checkpointOptions.PreCheckPoint, "pre-checkpoint", "P", false, "Dump container's memory information only, leave the container running") flags.BoolVar(&checkpointOptions.WithPrevious, "with-previous", false, "Checkpoint container with pre-checkpoint images") + flags.StringP("compress", "c", "zstd", "Select compression algorithm (gzip, none, zstd) for checkpoint archive.") + _ = checkpointCommand.RegisterFlagCompletionFunc("compress", common.AutocompleteCheckpointCompressType) + validate.AddLatestFlag(checkpointCommand, &checkpointOptions.Latest) } func checkpoint(cmd *cobra.Command, args []string) error { var errs utils.OutputErrors + if cmd.Flags().Changed("compress") { + if checkpointOptions.Export == "" { + return errors.Errorf("--compress can only be used with --export") + } + compress, _ := cmd.Flags().GetString("compress") + switch strings.ToLower(compress) { + case "none": + checkpointOptions.Compression = archive.Uncompressed + case "gzip": + checkpointOptions.Compression = archive.Gzip + case "zstd": + checkpointOptions.Compression = archive.Zstd + default: + return errors.Errorf("Selected compression algorithm (%q) not supported. Please select one from: gzip, none, zstd", compress) + } + } else { + checkpointOptions.Compression = archive.Zstd + } if rootless.IsRootless() { return errors.New("checkpointing a container requires root") } diff --git a/cmd/podman/containers/restore.go b/cmd/podman/containers/restore.go index 3b1848abb..b908ea493 100644 --- a/cmd/podman/containers/restore.go +++ b/cmd/podman/containers/restore.go @@ -36,9 +36,7 @@ var ( } ) -var ( - restoreOptions entities.RestoreOptions -) +var restoreOptions entities.RestoreOptions func init() { registry.Commands = append(registry.Commands, registry.CliCommand{ @@ -66,10 +64,17 @@ func init() { flags.BoolVar(&restoreOptions.IgnoreStaticIP, "ignore-static-ip", false, "Ignore IP address set via --static-ip") flags.BoolVar(&restoreOptions.IgnoreStaticMAC, "ignore-static-mac", false, "Ignore MAC address set via --mac-address") flags.BoolVar(&restoreOptions.IgnoreVolumes, "ignore-volumes", false, "Do not export volumes associated with container") + + flags.StringSliceP( + "publish", "p", []string{}, + "Publish a container's port, or a range of ports, to the host (default [])", + ) + _ = restoreCommand.RegisterFlagCompletionFunc("publish", completion.AutocompleteNone) + validate.AddLatestFlag(restoreCommand, &restoreOptions.Latest) } -func restore(_ *cobra.Command, args []string) error { +func restore(cmd *cobra.Command, args []string) error { var errs utils.OutputErrors if rootless.IsRootless() { return errors.New("restoring a container requires root") @@ -90,6 +95,17 @@ func restore(_ *cobra.Command, args []string) error { return errors.Errorf("--tcp-established cannot be used with --name") } + inputPorts, err := cmd.Flags().GetStringSlice("publish") + if err != nil { + return err + } + if len(inputPorts) > 0 { + restoreOptions.PublishPorts, err = common.CreatePortBindings(inputPorts) + if err != nil { + return err + } + } + argLen := len(args) if restoreOptions.Import != "" { if restoreOptions.All || restoreOptions.Latest { diff --git a/cmd/podman/system/service.go b/cmd/podman/system/service.go index 63f2de51e..a30f43839 100644 --- a/cmd/podman/system/service.go +++ b/cmd/podman/system/service.go @@ -39,7 +39,8 @@ Enable a listening service for API access to Podman commands. } srvArgs = struct { - Timeout int64 + Timeout int64 + CorsHeaders string }{} ) @@ -54,6 +55,8 @@ func init() { timeFlagName := "time" flags.Int64VarP(&srvArgs.Timeout, timeFlagName, "t", 5, "Time until the service session expires in seconds. Use 0 to disable the timeout") _ = srvCmd.RegisterFlagCompletionFunc(timeFlagName, completion.AutocompleteNone) + flags.StringVarP(&srvArgs.CorsHeaders, "cors", "", "", "Set CORS Headers") + _ = srvCmd.RegisterFlagCompletionFunc("cors", completion.AutocompleteNone) flags.SetNormalizeFunc(aliasTimeoutFlag) } @@ -71,7 +74,6 @@ func service(cmd *cobra.Command, args []string) error { return err } logrus.Infof("using API endpoint: '%s'", apiURI) - // Clean up any old existing unix domain socket if len(apiURI) > 0 { uri, err := url.Parse(apiURI) @@ -90,8 +92,9 @@ func service(cmd *cobra.Command, args []string) error { } opts := entities.ServiceOptions{ - URI: apiURI, - Command: cmd, + URI: apiURI, + Command: cmd, + CorsHeaders: srvArgs.CorsHeaders, } opts.Timeout = time.Duration(srvArgs.Timeout) * time.Second diff --git a/cmd/podman/system/service_abi.go b/cmd/podman/system/service_abi.go index 364663323..d59a45564 100644 --- a/cmd/podman/system/service_abi.go +++ b/cmd/podman/system/service_abi.go @@ -72,7 +72,7 @@ func restService(opts entities.ServiceOptions, flags *pflag.FlagSet, cfg *entiti } infra.StartWatcher(rt) - server, err := api.NewServerWithSettings(rt, opts.Timeout, listener) + server, err := api.NewServerWithSettings(rt, listener, api.Options{Timeout: opts.Timeout, CorsHeaders: opts.CorsHeaders}) if err != nil { return err } diff --git a/docs/source/Commands.rst b/docs/source/Commands.rst index 766b6a02e..767b09c08 100644 --- a/docs/source/Commands.rst +++ b/docs/source/Commands.rst @@ -55,7 +55,7 @@ Commands :doc:`logs <markdown/podman-logs.1>` Fetch the logs of a container -:doc:`machine <markdown/podman-machine.1>` Manage podman's virtual machine +:doc:`machine <machine>` Manage podman's virtual machine :doc:`manifest <manifest>` Create and manipulate manifest lists and image indexes @@ -91,7 +91,7 @@ Commands :doc:`search <markdown/podman-search.1>` Search registry for image -:doc:`secret <markdown/podman-secret.1>` Manage podman secrets +:doc:`secret <secret>` Manage podman secrets :doc:`start <markdown/podman-start.1>` Start one or more containers diff --git a/docs/source/machine.rst b/docs/source/machine.rst index 16c042173..3962fca99 100644 --- a/docs/source/machine.rst +++ b/docs/source/machine.rst @@ -3,8 +3,13 @@ Machine :doc:`init <markdown/podman-machine-init.1>` Initialize a new virtual machine + :doc:`list <markdown/podman-machine-list.1>` List virtual machines + :doc:`rm <markdown/podman-machine-rm.1>` Remove a virtual machine + :doc:`ssh <markdown/podman-machine-ssh.1>` SSH into a virtual machine + :doc:`start <markdown/podman-machine-start.1>` Start a virtual machine + :doc:`stop <markdown/podman-machine-stop.1>` Stop a virtual machine diff --git a/docs/source/markdown/podman-container-checkpoint.1.md b/docs/source/markdown/podman-container-checkpoint.1.md index 46b6cb646..271a44c9d 100644 --- a/docs/source/markdown/podman-container-checkpoint.1.md +++ b/docs/source/markdown/podman-container-checkpoint.1.md @@ -10,31 +10,25 @@ podman\-container\-checkpoint - Checkpoints one or more running containers Checkpoints all the processes in one or more containers. You may use container IDs or names as input. ## OPTIONS -#### **--keep**, **-k** - -Keep all temporary log and statistics files created by CRIU during checkpointing. These files -are not deleted if checkpointing fails for further debugging. If checkpointing succeeds these -files are theoretically not needed, but if these files are needed Podman can keep the files -for further analysis. - #### **--all**, **-a** Checkpoint all running containers. -#### **--latest**, **-l** - -Instead of providing the container name or ID, checkpoint the last created container. (This option is not available with the remote Podman client) - -#### **--leave-running**, **-R** +#### **--compress**, **-c** -Leave the container running after checkpointing instead of stopping it. +Specify the compression algorithm used for the checkpoint archive created +with the **--export, -e** option. Possible algorithms are *gzip*, *none* +and *zstd*. If no compression algorithm is specified Podman will use +*zstd*. -#### **--tcp-established** +One possible reason to use *none* is to enable faster creation of checkpoint +archives. Not compressing the checkpoint archive can result in faster checkpoint +archive creation. -Checkpoint a container with established TCP connections. If the checkpoint -image contains established TCP connections, this options is required during -restore. Defaults to not checkpointing containers with established TCP -connections. +``` +# podman container checkpoint -l --compress=none --export=dump.tar +# podman container checkpoint -l --compress=gzip --export=dump.tar.gz +``` #### **--export**, **-e** @@ -56,11 +50,33 @@ This option must be used in combination with the **--export, -e** option. When this option is specified, the content of volumes associated with the container will not be included into the checkpoint tar.gz file. +#### **--keep**, **-k** + +Keep all temporary log and statistics files created by CRIU during checkpointing. These files +are not deleted if checkpointing fails for further debugging. If checkpointing succeeds these +files are theoretically not needed, but if these files are needed Podman can keep the files +for further analysis. + +#### **--latest**, **-l** + +Instead of providing the container name or ID, checkpoint the last created container. (This option is not available with the remote Podman client) + +#### **--leave-running**, **-R** + +Leave the container running after checkpointing instead of stopping it. + #### **--pre-checkpoint**, **-P** Dump the container's memory information only, leaving the container running. Later operations will supersede prior dumps. It only works on runc 1.0-rc3 or higher. +#### **--tcp-established** + +Checkpoint a container with established TCP connections. If the checkpoint +image contains established TCP connections, this options is required during +restore. Defaults to not checkpointing containers with established TCP +connections. + #### **--with-previous** Check out the container with previous criu image files in pre-dump. It only works diff --git a/docs/source/markdown/podman-container-restore.1.md b/docs/source/markdown/podman-container-restore.1.md index ef8722279..82bf76d1e 100644 --- a/docs/source/markdown/podman-container-restore.1.md +++ b/docs/source/markdown/podman-container-restore.1.md @@ -95,6 +95,19 @@ This option must be used in combination with the **--import, -i** option. When restoring containers from a checkpoint tar.gz file with this option, the content of associated volumes will not be restored. +#### **--publish**, **-p** + +Replaces the ports that the container publishes, as configured during the +initial container start, with a new set of port forwarding rules. + +``` +# podman run --rm -p 2345:80 -d webserver +# podman container checkpoint -l --export=dump.tar +# podman container restore -p 5432:8080 --import=dump.tar +``` + +For more details please see **podman run --publish**. + ## EXAMPLE podman container restore mywebserver @@ -104,7 +117,7 @@ podman container restore 860a4b23 podman container restore --import-previous pre-checkpoint.tar.gz --import checkpoint.tar.gz ## SEE ALSO -podman(1), podman-container-checkpoint(1) +podman(1), podman-container-checkpoint(1), podman-run(1) ## HISTORY September 2018, Originally compiled by Adrian Reber <areber@redhat.com> diff --git a/docs/source/markdown/podman-system-service.1.md b/docs/source/markdown/podman-system-service.1.md index 2c8be73c2..dfb026de1 100644 --- a/docs/source/markdown/podman-system-service.1.md +++ b/docs/source/markdown/podman-system-service.1.md @@ -30,6 +30,10 @@ Note: The default systemd unit files (system and user) change the log-level opti The time until the session expires in _seconds_. The default is 5 seconds. A value of `0` means no timeout, therefore the session will not expire. +#### **--cors** + +CORS headers to inject to the HTTP response. The default value is empty string which disables CORS headers. + #### **--help**, **-h** Print usage statement. diff --git a/docs/tutorials/podman-go-bindings.md b/docs/tutorials/podman-go-bindings.md index a952f2dc0..2bbf4e5de 100644 --- a/docs/tutorials/podman-go-bindings.md +++ b/docs/tutorials/podman-go-bindings.md @@ -174,7 +174,7 @@ This binding takes three arguments: ```Go // Pull Busybox image (Sample 1) fmt.Println("Pulling Busybox image...") - _, err = images.Pull(connText, "docker.io/busybox", entities.ImagePullOptions{}) + _, err = images.Pull(connText, "docker.io/busybox", &images.PullOptions{}) if err != nil { fmt.Println(err) os.Exit(1) @@ -183,7 +183,7 @@ This binding takes three arguments: // Pull Fedora image (Sample 2) rawImage := "registry.fedoraproject.org/fedora:latest" fmt.Println("Pulling Fedora image...") - _, err = images.Pull(connText, rawImage, entities.ImagePullOptions{}) + _, err = images.Pull(connText, rawImage, &images.PullOptions{}) if err != nil { fmt.Println(err) os.Exit(1) @@ -229,7 +229,7 @@ This binding takes three arguments: ```Go // List images - imageSummary, err := images.List(connText, nil, nil) + imageSummary, err := images.List(connText, &images.ListOptions{}) if err != nil { fmt.Println(err) os.Exit(1) @@ -287,7 +287,7 @@ containers.Wait() takes three arguments: // Container create s := specgen.NewSpecGenerator(rawImage, false) s.Terminal = true - r, err := containers.CreateWithSpec(connText, s) + r, err := containers.CreateWithSpec(connText, s, nil) if err != nil { fmt.Println(err) os.Exit(1) @@ -302,7 +302,7 @@ containers.Wait() takes three arguments: } running := define.ContainerStateRunning - _, err = containers.Wait(connText, r.ID, &running) + _, err = containers.Wait(connText, r.ID, &containers.WaitOptions{Condition: []define.ContainerStatus{running}}) if err != nil { fmt.Println(err) os.Exit(1) @@ -346,7 +346,7 @@ containers.List() takes seven arguments: ```Go // Container list var latestContainers = 1 - containerLatestList, err := containers.List(connText, nil, nil, &latestContainers, nil, nil, nil) + containerLatestList, err := containers.List(connText, &containers.ListOptions{Last: &latestContainers}) if err != nil { fmt.Println(err) os.Exit(1) diff --git a/docs/tutorials/remote_client.md b/docs/tutorials/remote_client.md index e39d804a6..889947397 100644 --- a/docs/tutorials/remote_client.md +++ b/docs/tutorials/remote_client.md @@ -108,5 +108,9 @@ podman-remote system connection --help You can use the Podman remote clients to manage your containers running on a Linux server. The communication between client and server relies heavily on SSH connections and the use of SSH keys are encouraged. Once you have Podman installed on your remote client, you should set up a connection using `podman-remote system connection add` which will then be used by subsequent Podman commands. +# Troubleshooting + +See the [Troubleshooting](../../troubleshooting.md) document if you run into issues. + ## History Adapted from the [Mac and Windows tutorial](https://github.com/containers/podman/blob/master/docs/tutorials/mac_win_client.md) @@ -25,7 +25,7 @@ require ( github.com/davecgh/go-spew v1.1.1 github.com/digitalocean/go-qemu v0.0.0-20210209191958-152a1535e49f github.com/docker/distribution v2.7.1+incompatible - github.com/docker/docker v20.10.6+incompatible + github.com/docker/docker v20.10.7+incompatible github.com/docker/go-connections v0.4.0 github.com/docker/go-plugins-helpers v0.0.0-20200102110956-c9a8a2d92ccc github.com/docker/go-units v0.4.0 @@ -42,14 +42,14 @@ require ( github.com/mattn/go-colorable v0.1.8 // indirect github.com/moby/term v0.0.0-20201216013528-df9cb8a40635 github.com/mrunalp/fileutils v0.5.0 - github.com/onsi/ginkgo v1.16.3 + github.com/onsi/ginkgo v1.16.4 github.com/onsi/gomega v1.13.0 github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/image-spec v1.0.2-0.20190823105129-775207bd45b6 github.com/opencontainers/runc v1.0.0-rc95 github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 github.com/opencontainers/runtime-tools v0.9.0 - github.com/opencontainers/selinux v1.8.1 + github.com/opencontainers/selinux v1.8.2 github.com/pkg/errors v0.9.1 github.com/pmezard/go-difflib v1.0.0 github.com/rootless-containers/rootlesskit v0.14.2 @@ -61,7 +61,7 @@ require ( github.com/uber/jaeger-client-go v2.29.1+incompatible github.com/vbauerster/mpb/v6 v6.0.4 github.com/vishvananda/netlink v1.1.1-0.20201029203352-d40f9887b852 - go.etcd.io/bbolt v1.3.5 + go.etcd.io/bbolt v1.3.6 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 golang.org/x/sync v0.0.0-20201207232520-09787c993a3a golang.org/x/sys v0.0.0-20210514084401-e8d321eab015 @@ -90,6 +90,8 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= +github.com/bits-and-blooms/bitset v1.2.0 h1:Kn4yilvwNtMACtf1eYDlG8H77R07mZSPbMjLyS07ChA= +github.com/bits-and-blooms/bitset v1.2.0/go.mod h1:gIdJ4wp64HaoK2YrL1Q5/N7Y16edYb8uY+O0FJTyyDA= github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= github.com/blang/semver v3.1.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ= @@ -295,8 +297,9 @@ github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BU github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/docker v1.4.2-0.20191219165747-a9416c67da9f/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker v20.10.3-0.20210216175712-646072ed6524+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/docker v20.10.6+incompatible h1:oXI3Vas8TI8Eu/EjH4srKHJBVqraSzJybhxY7Om9faQ= github.com/docker/docker v20.10.6+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/docker v20.10.7+incompatible h1:Z6O9Nhsjv+ayUEeI1IojKbYcsGdgYSNqxe1s2MYzUhQ= +github.com/docker/docker v20.10.7+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker-credential-helpers v0.6.3 h1:zI2p9+1NQYdnG6sMU26EX4aVGlqbInSQxQXLvzJ4RPQ= github.com/docker/docker-credential-helpers v0.6.3/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y= github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ= @@ -649,8 +652,8 @@ github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+ github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.15.0/go.mod h1:hF8qUzuuC8DJGygJH3726JnCZX4MYbRB8yFfISqnKUg= github.com/onsi/ginkgo v1.16.2/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E= -github.com/onsi/ginkgo v1.16.3 h1:3s86PZkI1ApJh6HFIzC1gXby/mIyZqfE5zxSvtoBSsM= -github.com/onsi/ginkgo v1.16.3/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E= +github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc= +github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/gomega v0.0.0-20151007035656-2152b45fa28a/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= @@ -696,8 +699,9 @@ github.com/opencontainers/runtime-tools v0.9.0/go.mod h1:r3f7wjNzSs2extwzU3Y+6pK github.com/opencontainers/selinux v1.5.1/go.mod h1:yTcKuYAh6R95iDpefGLQaPaRwJFwyzAJufJyiTt7s0g= github.com/opencontainers/selinux v1.6.0/go.mod h1:VVGKuOLlE7v4PJyT6h7mNWvq1rzqiriPsEqVhc+svHE= github.com/opencontainers/selinux v1.8.0/go.mod h1:RScLhm78qiWa2gbVCcGkC7tCGdgk3ogry1nUQF8Evvo= -github.com/opencontainers/selinux v1.8.1 h1:yvEZh7CsfnJNwKzG9ZeXwbvR05RAZsu5RS/3vA6qFTA= github.com/opencontainers/selinux v1.8.1/go.mod h1:RScLhm78qiWa2gbVCcGkC7tCGdgk3ogry1nUQF8Evvo= +github.com/opencontainers/selinux v1.8.2 h1:c4ca10UMgRcvZ6h0K4HtS15UaVSBEaE+iln2LVpAuGc= +github.com/opencontainers/selinux v1.8.2/go.mod h1:MUIHuUEvKB1wtJjQdOyYRgOnLD2xAPP8dBsCoU0KuF8= github.com/openshift/imagebuilder v1.2.2-0.20210415181909-87f3e48c2656 h1:WaxyNFpmIDu4i6so9r6LVFIbSaXqsj8oitMitt86ae4= github.com/openshift/imagebuilder v1.2.2-0.20210415181909-87f3e48c2656/go.mod h1:9aJRczxCH0mvT6XQ+5STAQaPWz7OsWcU5/mRkt8IWeo= github.com/ostreedev/ostree-go v0.0.0-20190702140239-759a8c1ac913 h1:TnbXhKzrTOyuvWrjI8W6pcoI9XPbLHFXCdN2dtUw7Rw= @@ -852,7 +856,6 @@ github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df/go.mod h1:JP3t17 github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae h1:4hwBBUfQCFe3Cym0ZtKyq7L16eZUtYKs+BaHDN6mAns= github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0= github.com/willf/bitset v1.1.11-0.20200630133818-d5bec3311243/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4= -github.com/willf/bitset v1.1.11 h1:N7Z7E9UvjW+sGsEl7k/SJrvY2reP1A07MrGuCjIOjRE= github.com/willf/bitset v1.1.11/go.mod h1:83CECat5yLh5zVOf4P1ErAgKA5UDvKtgyUABdr3+MjI= github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU= github.com/xeipuuv/gojsonpointer v0.0.0-20190809123943-df4f5c81cb3b h1:6cLsL+2FW6dRAdl5iMtHgRogVCff0QpRi9653YmdcJA= @@ -872,8 +875,9 @@ github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPS github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg= go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0= go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= +go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU= +go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg= go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1 h1:A/5uWzF44DlIgdm/PQFwfMkW0JX+cIcQi/SwLAmZP5M= go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk= @@ -1054,6 +1058,7 @@ golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200909081042-eff7692f9009/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200916030750-2334cc1a136f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200922070232-aee5d888a860/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201009025420-dfb3f7c4e634/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201101102859-da207088b7d1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= diff --git a/libpod/container_api.go b/libpod/container_api.go index 4ccb240e7..b75d0b41d 100644 --- a/libpod/container_api.go +++ b/libpod/container_api.go @@ -12,6 +12,7 @@ import ( "github.com/containers/podman/v3/libpod/define" "github.com/containers/podman/v3/libpod/events" "github.com/containers/podman/v3/pkg/signal" + "github.com/containers/storage/pkg/archive" "github.com/pkg/errors" "github.com/sirupsen/logrus" ) @@ -776,6 +777,9 @@ type ContainerCheckpointOptions struct { // ImportPrevious tells the API to restore container with two // images. One is TargetFile, the other is ImportPrevious. ImportPrevious string + // Compression tells the API which compression to use for + // the exported checkpoint archive. + Compression archive.Compression } // Checkpoint checkpoints a container diff --git a/libpod/container_internal_linux.go b/libpod/container_internal_linux.go index 1b2f5a496..a3acc3198 100644 --- a/libpod/container_internal_linux.go +++ b/libpod/container_internal_linux.go @@ -985,7 +985,7 @@ func (c *Container) exportCheckpoint(options ContainerCheckpointOptions) error { } input, err := archive.TarWithOptions(c.bundlePath(), &archive.TarOptions{ - Compression: archive.Gzip, + Compression: options.Compression, IncludeSourceDir: true, IncludeFiles: includeFiles, }) @@ -1668,17 +1668,16 @@ func (c *Container) generateResolvConf() (string, error) { return "", err } - // Ensure that the container's /etc/resolv.conf is compatible with its - // network configuration. - // TODO: set ipv6 enable bool more sanely - resolv, err := resolvconf.FilterResolvDNS(contents, true, c.config.CreateNetNS) - if err != nil { - return "", errors.Wrapf(err, "error parsing host resolv.conf") - } - + ipv6 := false // Check if CNI gave back and DNS servers for us to add in cniResponse := c.state.NetworkStatus for _, i := range cniResponse { + for _, ip := range i.IPs { + // Note: only using To16() does not work since it also returns a vaild ip for ipv4 + if ip.Address.IP.To4() == nil && ip.Address.IP.To16() != nil { + ipv6 = true + } + } if i.DNS.Nameservers != nil { cniNameServers = append(cniNameServers, i.DNS.Nameservers...) logrus.Debugf("adding nameserver(s) from cni response of '%q'", i.DNS.Nameservers) @@ -1689,6 +1688,25 @@ func (c *Container) generateResolvConf() (string, error) { } } + if c.config.NetMode.IsSlirp4netns() { + ctrNetworkSlipOpts := []string{} + if c.config.NetworkOptions != nil { + ctrNetworkSlipOpts = append(ctrNetworkSlipOpts, c.config.NetworkOptions["slirp4netns"]...) + } + slirpOpts, err := parseSlirp4netnsNetworkOptions(c.runtime, ctrNetworkSlipOpts) + if err != nil { + return "", err + } + ipv6 = slirpOpts.enableIPv6 + } + + // Ensure that the container's /etc/resolv.conf is compatible with its + // network configuration. + resolv, err := resolvconf.FilterResolvDNS(contents, ipv6, c.config.CreateNetNS) + if err != nil { + return "", errors.Wrapf(err, "error parsing host resolv.conf") + } + dns := make([]net.IP, 0, len(c.runtime.config.Containers.DNSServers)) for _, i := range c.runtime.config.Containers.DNSServers { result := net.ParseIP(i) diff --git a/libpod/events/filters.go b/libpod/events/filters.go index acfb96302..4d27e8fc4 100644 --- a/libpod/events/filters.go +++ b/libpod/events/filters.go @@ -97,18 +97,41 @@ func parseFilter(filter string) (string, string, error) { return filterSplit[0], filterSplit[1], nil } -func generateEventOptions(filters []string, since, until string) ([]EventFilter, error) { - options := make([]EventFilter, 0, len(filters)) +// applyFilters applies the EventFilter slices in sequence. Filters under the +// same key are disjunctive while each key must match (conjuctive). +func applyFilters(event *Event, filterMap map[string][]EventFilter) bool { + for _, filters := range filterMap { + success := false + for _, filter := range filters { + if filter(event) { + success = true + break + } + } + if !success { + return false + } + } + return true +} + +// generateEventFilter parses the specified filters into a filter map that can +// later on be used to filter events. Keys are conjunctive, values are +// disjunctive. +func generateEventFilters(filters []string, since, until string) (map[string][]EventFilter, error) { + filterMap := make(map[string][]EventFilter) for _, filter := range filters { key, val, err := parseFilter(filter) if err != nil { return nil, err } - funcFilter, err := generateEventFilter(key, val) + filterFunc, err := generateEventFilter(key, val) if err != nil { return nil, err } - options = append(options, funcFilter) + filterSlice := filterMap[key] + filterSlice = append(filterSlice, filterFunc) + filterMap[key] = filterSlice } if len(since) > 0 { @@ -116,7 +139,8 @@ func generateEventOptions(filters []string, since, until string) ([]EventFilter, if err != nil { return nil, errors.Wrapf(err, "unable to convert since time of %s", since) } - options = append(options, generateEventSinceOption(timeSince)) + filterFunc := generateEventSinceOption(timeSince) + filterMap["since"] = []EventFilter{filterFunc} } if len(until) > 0 { @@ -124,7 +148,8 @@ func generateEventOptions(filters []string, since, until string) ([]EventFilter, if err != nil { return nil, errors.Wrapf(err, "unable to convert until time of %s", until) } - options = append(options, generateEventUntilOption(timeUntil)) + filterFunc := generateEventUntilOption(timeUntil) + filterMap["until"] = []EventFilter{filterFunc} } - return options, nil + return filterMap, nil } diff --git a/libpod/events/journal_linux.go b/libpod/events/journal_linux.go index 23e5f15b1..7006290e9 100644 --- a/libpod/events/journal_linux.go +++ b/libpod/events/journal_linux.go @@ -69,9 +69,9 @@ func (e EventJournalD) Write(ee Event) error { // Read reads events from the journal and sends qualified events to the event channel func (e EventJournalD) Read(ctx context.Context, options ReadOptions) error { defer close(options.EventChannel) - eventOptions, err := generateEventOptions(options.Filters, options.Since, options.Until) + filterMap, err := generateEventFilters(options.Filters, options.Since, options.Until) if err != nil { - return errors.Wrapf(err, "failed to generate event options") + return errors.Wrapf(err, "failed to parse event filters") } var untilTime time.Time if len(options.Until) > 0 { @@ -159,11 +159,7 @@ func (e EventJournalD) Read(ctx context.Context, options ReadOptions) error { } continue } - include := true - for _, filter := range eventOptions { - include = include && filter(newEvent) - } - if include { + if applyFilters(newEvent, filterMap) { options.EventChannel <- newEvent } } diff --git a/libpod/events/logfile.go b/libpod/events/logfile.go index 0f00525e8..952444f2b 100644 --- a/libpod/events/logfile.go +++ b/libpod/events/logfile.go @@ -44,9 +44,9 @@ func (e EventLogFile) Write(ee Event) error { // Reads from the log file func (e EventLogFile) Read(ctx context.Context, options ReadOptions) error { defer close(options.EventChannel) - eventOptions, err := generateEventOptions(options.Filters, options.Since, options.Until) + filterMap, err := generateEventFilters(options.Filters, options.Since, options.Until) if err != nil { - return errors.Wrapf(err, "unable to generate event options") + return errors.Wrapf(err, "failed to parse event filters") } t, err := e.getTail(options) if err != nil { @@ -92,11 +92,7 @@ func (e EventLogFile) Read(ctx context.Context, options ReadOptions) error { default: return errors.Errorf("event type %s is not valid in %s", event.Type.String(), e.options.LogFilePath) } - include := true - for _, filter := range eventOptions { - include = include && filter(event) - } - if include && copy { + if copy && applyFilters(event, filterMap) { options.EventChannel <- event } } diff --git a/pkg/api/handlers/compat/containers_stats.go b/pkg/api/handlers/compat/containers_stats.go index 694b57bb1..851955207 100644 --- a/pkg/api/handlers/compat/containers_stats.go +++ b/pkg/api/handlers/compat/containers_stats.go @@ -22,7 +22,8 @@ func StatsContainer(w http.ResponseWriter, r *http.Request) { decoder := r.Context().Value("decoder").(*schema.Decoder) query := struct { - Stream bool `schema:"stream"` + Stream bool `schema:"stream"` + OneShot bool `schema:"one-shot"` //added schema for one shot }{ Stream: true, } @@ -30,6 +31,10 @@ func StatsContainer(w http.ResponseWriter, r *http.Request) { utils.Error(w, "Something went wrong.", http.StatusBadRequest, errors.Wrapf(err, "failed to parse parameters for %s", r.URL.String())) return } + if query.Stream && query.OneShot { // mismatch. one-shot can only be passed with stream=false + utils.Error(w, "invalid combination of stream and one-shot", http.StatusBadRequest, define.ErrInvalidArg) + return + } name := utils.GetName(r) ctnr, err := runtime.LookupContainer(name) @@ -56,6 +61,16 @@ func StatsContainer(w http.ResponseWriter, r *http.Request) { return } + coder := json.NewEncoder(w) + // Write header and content type. + w.WriteHeader(http.StatusOK) + w.Header().Add("Content-Type", "application/json") + if flusher, ok := w.(http.Flusher); ok { + flusher.Flush() + } + + // Setup JSON encoder for streaming. + coder.SetEscapeHTML(true) var preRead time.Time var preCPUStats CPUStats if query.Stream { @@ -75,17 +90,6 @@ func StatsContainer(w http.ResponseWriter, r *http.Request) { } } - // Write header and content type. - w.WriteHeader(http.StatusOK) - w.Header().Add("Content-Type", "application/json") - if flusher, ok := w.(http.Flusher); ok { - flusher.Flush() - } - - // Setup JSON encoder for streaming. - coder := json.NewEncoder(w) - coder.SetEscapeHTML(true) - streamLabel: // A label to flatten the scope select { case <-r.Context().Done(): @@ -199,7 +203,7 @@ streamLabel: // A label to flatten the scope flusher.Flush() } - if !query.Stream { + if !query.Stream || query.OneShot { return } diff --git a/pkg/api/handlers/compat/events.go b/pkg/api/handlers/compat/events.go index 405e616c5..9fbac91e0 100644 --- a/pkg/api/handlers/compat/events.go +++ b/pkg/api/handlers/compat/events.go @@ -75,7 +75,7 @@ func GetEvents(w http.ResponseWriter, r *http.Request) { coder := json.NewEncoder(w) coder.SetEscapeHTML(true) - for stream := true; stream; stream = query.Stream { + for { select { case err := <-errorChannel: if err != nil { diff --git a/pkg/api/handlers/compat/resize.go b/pkg/api/handlers/compat/resize.go index 23ed33a22..f65e313fc 100644 --- a/pkg/api/handlers/compat/resize.go +++ b/pkg/api/handlers/compat/resize.go @@ -46,20 +46,13 @@ func ResizeTTY(w http.ResponseWriter, r *http.Request) { utils.ContainerNotFound(w, name, err) return } - if state, err := ctnr.State(); err != nil { - utils.InternalServerError(w, errors.Wrapf(err, "cannot obtain container state")) - return - } else if state != define.ContainerStateRunning && !query.IgnoreNotRunning { - utils.Error(w, "Container not running", http.StatusConflict, - fmt.Errorf("container %q in wrong state %q", name, state.String())) - return - } - // If container is not running, ignore since this can be a race condition, and is expected if err := ctnr.AttachResize(sz); err != nil { - if errors.Cause(err) != define.ErrCtrStateInvalid || !query.IgnoreNotRunning { + if errors.Cause(err) != define.ErrCtrStateInvalid { utils.InternalServerError(w, errors.Wrapf(err, "cannot resize container")) - return + } else { + utils.Error(w, "Container not running", http.StatusConflict, err) } + return } // This is not a 204, even though we write nothing, for compatibility // reasons. diff --git a/pkg/api/handlers/libpod/swagger.go b/pkg/api/handlers/libpod/swagger.go index 9450a70d9..19eced986 100644 --- a/pkg/api/handlers/libpod/swagger.go +++ b/pkg/api/handlers/libpod/swagger.go @@ -95,7 +95,7 @@ type swagInfoResponse struct { // swagger:response NetworkRmReport type swagNetworkRmReport struct { // in:body - Body entities.NetworkRmReport + Body []entities.NetworkRmReport } // Network inspect diff --git a/pkg/api/server/handler_api.go b/pkg/api/server/handler_api.go index 28b8706a8..becc674c0 100644 --- a/pkg/api/server/handler_api.go +++ b/pkg/api/server/handler_api.go @@ -63,6 +63,12 @@ func (s *APIServer) APIHandler(h http.HandlerFunc) http.HandlerFunc { w.Header().Set("Libpod-API-Version", lv) w.Header().Set("Server", "Libpod/"+lv+" ("+runtime.GOOS+")") + if s.CorsHeaders != "" { + w.Header().Set("Access-Control-Allow-Origin", s.CorsHeaders) + w.Header().Set("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, X-Registry-Auth, Connection, Upgrade, X-Registry-Config") + w.Header().Set("Access-Control-Allow-Methods", "HEAD, GET, POST, DELETE, PUT, OPTIONS") + } + h(w, r) logrus.Debugf("APIHandler(%s) -- %s %s END", rid, r.Method, r.URL.String()) } diff --git a/pkg/api/server/register_containers.go b/pkg/api/server/register_containers.go index 536c4707a..88ebb4df5 100644 --- a/pkg/api/server/register_containers.go +++ b/pkg/api/server/register_containers.go @@ -375,6 +375,11 @@ func (s *APIServer) registerContainersHandlers(r *mux.Router) error { // type: boolean // default: true // description: Stream the output + // - in: query + // name: one-shot + // type: boolean + // default: false + // description: Provide a one-shot response in which preCPU stats are blank, resulting in a single cycle return. // produces: // - application/json // responses: @@ -1359,6 +1364,8 @@ func (s *APIServer) registerContainersHandlers(r *mux.Router) error { // $ref: "#/responses/ok" // 404: // $ref: "#/responses/NoSuchContainer" + // 409: + // $ref: "#/responses/ConflictError" // 500: // $ref: "#/responses/InternalError" r.HandleFunc(VersionedPath("/libpod/containers/{name}/resize"), s.APIHandler(compat.ResizeTTY)).Methods(http.MethodPost) diff --git a/pkg/api/server/register_networks.go b/pkg/api/server/register_networks.go index 9a5ccb789..4d9806316 100644 --- a/pkg/api/server/register_networks.go +++ b/pkg/api/server/register_networks.go @@ -241,7 +241,9 @@ func (s *APIServer) registerNetworkHandlers(r *mux.Router) error { // tags: // - networks // summary: List networks - // description: Display summary of network configurations + // description: | + // Display summary of network configurations. + // - In a 200 response, all of the fields named Bytes are returned as a Base64 encoded string. // parameters: // - in: query // name: filters diff --git a/pkg/api/server/server.go b/pkg/api/server/server.go index 972541bc6..1e8faf8f5 100644 --- a/pkg/api/server/server.go +++ b/pkg/api/server/server.go @@ -34,10 +34,12 @@ type APIServer struct { context.CancelFunc // Stop APIServer idleTracker *idle.Tracker // Track connections to support idle shutdown pprof *http.Server // Sidecar http server for providing performance data + CorsHeaders string // Inject CORS headers to each request } // Number of seconds to wait for next request, if exceeded shutdown server const ( + DefaultCorsHeaders = "" DefaultServiceDuration = 300 * time.Second UnlimitedServiceDuration = 0 * time.Second ) @@ -45,17 +47,22 @@ const ( // shutdownOnce ensures Shutdown() may safely be called from several go routines var shutdownOnce sync.Once +type Options struct { + Timeout time.Duration + CorsHeaders string +} + // NewServer will create and configure a new API server with all defaults func NewServer(runtime *libpod.Runtime) (*APIServer, error) { - return newServer(runtime, DefaultServiceDuration, nil) + return newServer(runtime, DefaultServiceDuration, nil, DefaultCorsHeaders) } // NewServerWithSettings will create and configure a new API server using provided settings -func NewServerWithSettings(runtime *libpod.Runtime, duration time.Duration, listener *net.Listener) (*APIServer, error) { - return newServer(runtime, duration, listener) +func NewServerWithSettings(runtime *libpod.Runtime, listener *net.Listener, opts Options) (*APIServer, error) { + return newServer(runtime, opts.Timeout, listener, opts.CorsHeaders) } -func newServer(runtime *libpod.Runtime, duration time.Duration, listener *net.Listener) (*APIServer, error) { +func newServer(runtime *libpod.Runtime, duration time.Duration, listener *net.Listener, corsHeaders string) (*APIServer, error) { // If listener not provided try socket activation protocol if listener == nil { if _, found := os.LookupEnv("LISTEN_PID"); !found { @@ -71,6 +78,11 @@ func newServer(runtime *libpod.Runtime, duration time.Duration, listener *net.Li } listener = &listeners[0] } + if corsHeaders == "" { + logrus.Debug("CORS Headers were not set") + } else { + logrus.Debugf("CORS Headers were set to %s", corsHeaders) + } logrus.Infof("API server listening on %q", (*listener).Addr()) router := mux.NewRouter().UseEncodedPath() @@ -88,6 +100,7 @@ func newServer(runtime *libpod.Runtime, duration time.Duration, listener *net.Li idleTracker: idle, Listener: *listener, Runtime: runtime, + CorsHeaders: corsHeaders, } router.NotFoundHandler = http.HandlerFunc( diff --git a/pkg/bindings/containers/attach.go b/pkg/bindings/containers/attach.go index fd8a7011d..adef1e7c8 100644 --- a/pkg/bindings/containers/attach.go +++ b/pkg/bindings/containers/attach.go @@ -138,7 +138,7 @@ func Attach(ctx context.Context, nameOrID string, stdin io.Reader, stdout io.Wri winCtx, winCancel := context.WithCancel(ctx) defer winCancel() - go attachHandleResize(ctx, winCtx, winChange, false, nameOrID, file) + attachHandleResize(ctx, winCtx, winChange, false, nameOrID, file) } // If we are attaching around a start, we need to "signal" @@ -327,32 +327,38 @@ func (f *rawFormatter) Format(entry *logrus.Entry) ([]byte, error) { return append(buffer, '\r'), nil } -// This is intended to be run as a goroutine, handling resizing for a container -// or exec session. +// This is intended to not be run as a goroutine, handling resizing for a container +// or exec session. It will call resize once and then starts a goroutine which calls resize on winChange func attachHandleResize(ctx, winCtx context.Context, winChange chan os.Signal, isExec bool, id string, file *os.File) { - // Prime the pump, we need one reset to ensure everything is ready - winChange <- sig.SIGWINCH - for { - select { - case <-winCtx.Done(): - return - case <-winChange: - w, h, err := terminal.GetSize(int(file.Fd())) - if err != nil { - logrus.Warnf("failed to obtain TTY size: %v", err) - } + resize := func() { + w, h, err := terminal.GetSize(int(file.Fd())) + if err != nil { + logrus.Warnf("failed to obtain TTY size: %v", err) + } - var resizeErr error - if isExec { - resizeErr = ResizeExecTTY(ctx, id, new(ResizeExecTTYOptions).WithHeight(h).WithWidth(w)) - } else { - resizeErr = ResizeContainerTTY(ctx, id, new(ResizeTTYOptions).WithHeight(h).WithWidth(w)) - } - if resizeErr != nil { - logrus.Warnf("failed to resize TTY: %v", resizeErr) - } + var resizeErr error + if isExec { + resizeErr = ResizeExecTTY(ctx, id, new(ResizeExecTTYOptions).WithHeight(h).WithWidth(w)) + } else { + resizeErr = ResizeContainerTTY(ctx, id, new(ResizeTTYOptions).WithHeight(h).WithWidth(w)) + } + if resizeErr != nil { + logrus.Warnf("failed to resize TTY: %v", resizeErr) } } + + resize() + + go func() { + for { + select { + case <-winCtx.Done(): + return + case <-winChange: + resize() + } + } + }() } // Configure the given terminal for raw mode @@ -457,7 +463,7 @@ func ExecStartAndAttach(ctx context.Context, sessionID string, options *ExecStar winCtx, winCancel := context.WithCancel(ctx) defer winCancel() - go attachHandleResize(ctx, winCtx, winChange, true, sessionID, terminalFile) + attachHandleResize(ctx, winCtx, winChange, true, sessionID, terminalFile) } if options.GetAttachInput() { diff --git a/pkg/checkpoint/checkpoint_restore.go b/pkg/checkpoint/checkpoint_restore.go index 7a8f71c66..0d45cab5f 100644 --- a/pkg/checkpoint/checkpoint_restore.go +++ b/pkg/checkpoint/checkpoint_restore.go @@ -11,6 +11,7 @@ import ( "github.com/containers/podman/v3/libpod" "github.com/containers/podman/v3/pkg/domain/entities" "github.com/containers/podman/v3/pkg/errorhandling" + "github.com/containers/podman/v3/pkg/specgen/generate" "github.com/containers/storage/pkg/archive" spec "github.com/opencontainers/runtime-spec/specs-go" "github.com/pkg/errors" @@ -95,6 +96,14 @@ func CRImportCheckpoint(ctx context.Context, runtime *libpod.Runtime, restoreOpt newName = true } + if len(restoreOptions.PublishPorts) > 0 { + ports, _, _, err := generate.ParsePortMapping(restoreOptions.PublishPorts) + if err != nil { + return nil, err + } + ctrConfig.PortMappings = ports + } + pullOptions := &libimage.PullOptions{} pullOptions.Writer = os.Stderr if _, err := runtime.LibimageRuntime().Pull(ctx, ctrConfig.RootfsImageName, config.PullPolicyMissing, pullOptions); err != nil { diff --git a/pkg/domain/entities/containers.go b/pkg/domain/entities/containers.go index eacc14d50..8ed9b9b61 100644 --- a/pkg/domain/entities/containers.go +++ b/pkg/domain/entities/containers.go @@ -9,6 +9,7 @@ import ( "github.com/containers/image/v5/types" "github.com/containers/podman/v3/libpod/define" "github.com/containers/podman/v3/pkg/specgen" + "github.com/containers/storage/pkg/archive" "github.com/cri-o/ocicni/pkg/ocicni" ) @@ -178,6 +179,7 @@ type CheckpointOptions struct { TCPEstablished bool PreCheckPoint bool WithPrevious bool + Compression archive.Compression } type CheckpointReport struct { @@ -197,6 +199,7 @@ type RestoreOptions struct { Name string TCPEstablished bool ImportPrevious string + PublishPorts []specgen.PortMapping } type RestoreReport struct { diff --git a/pkg/domain/entities/events.go b/pkg/domain/entities/events.go index 930ca53ae..5e7cc9ad1 100644 --- a/pkg/domain/entities/events.go +++ b/pkg/domain/entities/events.go @@ -30,29 +30,41 @@ func ConvertToLibpodEvent(e Event) *libpodEvents.Event { if err != nil { return nil } + image := e.Actor.Attributes["image"] + name := e.Actor.Attributes["name"] + details := e.Actor.Attributes + delete(details, "image") + delete(details, "name") + delete(details, "containerExitCode") return &libpodEvents.Event{ ContainerExitCode: exitCode, ID: e.Actor.ID, - Image: e.Actor.Attributes["image"], - Name: e.Actor.Attributes["name"], + Image: image, + Name: name, Status: status, Time: time.Unix(e.Time, e.TimeNano), Type: t, + Details: libpodEvents.Details{ + Attributes: details, + }, } } // ConvertToEntitiesEvent converts a libpod event to an entities one. func ConvertToEntitiesEvent(e libpodEvents.Event) *Event { + attributes := e.Details.Attributes + if attributes == nil { + attributes = make(map[string]string) + } + attributes["image"] = e.Image + attributes["name"] = e.Name + attributes["containerExitCode"] = strconv.Itoa(e.ContainerExitCode) return &Event{dockerEvents.Message{ Type: e.Type.String(), Action: e.Status.String(), Actor: dockerEvents.Actor{ - ID: e.ID, - Attributes: map[string]string{ - "image": e.Image, - "name": e.Name, - "containerExitCode": strconv.Itoa(e.ContainerExitCode), - }, + ID: e.ID, + Attributes: attributes, }, Scope: "local", Time: e.Time.Unix(), diff --git a/pkg/domain/entities/system.go b/pkg/domain/entities/system.go index 31a6185dc..cca4bf44e 100644 --- a/pkg/domain/entities/system.go +++ b/pkg/domain/entities/system.go @@ -11,9 +11,10 @@ import ( // ServiceOptions provides the input for starting an API Service type ServiceOptions struct { - URI string // Path to unix domain socket service should listen on - Timeout time.Duration // duration of inactivity the service should wait before shutting down - Command *cobra.Command // CLI command provided. Used in V1 code + URI string // Path to unix domain socket service should listen on + Timeout time.Duration // duration of inactivity the service should wait before shutting down + Command *cobra.Command // CLI command provided. Used in V1 code + CorsHeaders string // CORS headers } // SystemPruneOptions provides options to prune system. diff --git a/pkg/domain/infra/abi/containers.go b/pkg/domain/infra/abi/containers.go index 237a43441..4908e72f6 100644 --- a/pkg/domain/infra/abi/containers.go +++ b/pkg/domain/infra/abi/containers.go @@ -483,6 +483,7 @@ func (ic *ContainerEngine) ContainerCheckpoint(ctx context.Context, namesOrIds [ KeepRunning: options.LeaveRunning, PreCheckPoint: options.PreCheckPoint, WithPrevious: options.WithPrevious, + Compression: options.Compression, } if options.All { diff --git a/pkg/specgen/generate/pod_create.go b/pkg/specgen/generate/pod_create.go index 20151f016..07c56b799 100644 --- a/pkg/specgen/generate/pod_create.go +++ b/pkg/specgen/generate/pod_create.go @@ -125,7 +125,7 @@ func createPodOptions(p *specgen.PodSpecGenerator, rt *libpod.Runtime) ([]libpod options = append(options, libpod.WithPodUseImageHosts()) } if len(p.PortMappings) > 0 { - ports, _, _, err := parsePortMapping(p.PortMappings) + ports, _, _, err := ParsePortMapping(p.PortMappings) if err != nil { return nil, err } diff --git a/pkg/specgen/generate/ports.go b/pkg/specgen/generate/ports.go index 8745f0dad..c00ad19fb 100644 --- a/pkg/specgen/generate/ports.go +++ b/pkg/specgen/generate/ports.go @@ -24,7 +24,7 @@ const ( // Parse port maps to OCICNI port mappings. // Returns a set of OCICNI port mappings, and maps of utilized container and // host ports. -func parsePortMapping(portMappings []specgen.PortMapping) ([]ocicni.PortMapping, map[string]map[string]map[uint16]uint16, map[string]map[string]map[uint16]uint16, error) { +func ParsePortMapping(portMappings []specgen.PortMapping) ([]ocicni.PortMapping, map[string]map[string]map[uint16]uint16, map[string]map[string]map[uint16]uint16, error) { // First, we need to validate the ports passed in the specgen, and then // convert them into CNI port mappings. type tempMapping struct { @@ -254,7 +254,7 @@ func parsePortMapping(portMappings []specgen.PortMapping) ([]ocicni.PortMapping, // Make final port mappings for the container func createPortMappings(ctx context.Context, s *specgen.SpecGenerator, imageData *libimage.ImageData) ([]ocicni.PortMapping, error) { - finalMappings, containerPortValidate, hostPortValidate, err := parsePortMapping(s.PortMappings) + finalMappings, containerPortValidate, hostPortValidate, err := ParsePortMapping(s.PortMappings) if err != nil { return nil, err } diff --git a/pkg/systemd/generate/common.go b/pkg/systemd/generate/common.go index 1ee070888..e183125a7 100644 --- a/pkg/systemd/generate/common.go +++ b/pkg/systemd/generate/common.go @@ -60,7 +60,7 @@ func filterPodFlags(command []string, argCount int) []string { return processed } -// filterCommonContainerFlags removes --conmon-pidfile, --cidfile and --cgroups from the specified command. +// filterCommonContainerFlags removes --sdnotify, --rm and --cgroups from the specified command. // argCount is the number of last arguments which should not be filtered, e.g. the container entrypoint. func filterCommonContainerFlags(command []string, argCount int) []string { processed := []string{} @@ -68,11 +68,14 @@ func filterCommonContainerFlags(command []string, argCount int) []string { s := command[i] switch { - case s == "--conmon-pidfile", s == "--cidfile", s == "--cgroups": + case s == "--rm": + // Boolean flags support --flag and --flag={true,false}. + continue + case s == "--sdnotify", s == "--cgroups": i++ continue - case strings.HasPrefix(s, "--conmon-pidfile="), - strings.HasPrefix(s, "--cidfile="), + case strings.HasPrefix(s, "--sdnotify="), + strings.HasPrefix(s, "--rm="), strings.HasPrefix(s, "--cgroups="): continue } diff --git a/pkg/systemd/generate/common_test.go b/pkg/systemd/generate/common_test.go index fdcc9d21b..3e2ac015f 100644 --- a/pkg/systemd/generate/common_test.go +++ b/pkg/systemd/generate/common_test.go @@ -93,22 +93,22 @@ func TestFilterCommonContainerFlags(t *testing.T) { }, { []string{"podman", "run", "--conmon-pidfile", "foo", "alpine"}, - []string{"podman", "run", "alpine"}, + []string{"podman", "run", "--conmon-pidfile", "foo", "alpine"}, 1, }, { []string{"podman", "run", "--conmon-pidfile=foo", "alpine"}, - []string{"podman", "run", "alpine"}, + []string{"podman", "run", "--conmon-pidfile=foo", "alpine"}, 1, }, { []string{"podman", "run", "--cidfile", "foo", "alpine"}, - []string{"podman", "run", "alpine"}, + []string{"podman", "run", "--cidfile", "foo", "alpine"}, 1, }, { []string{"podman", "run", "--cidfile=foo", "alpine"}, - []string{"podman", "run", "alpine"}, + []string{"podman", "run", "--cidfile=foo", "alpine"}, 1, }, { @@ -122,25 +122,15 @@ func TestFilterCommonContainerFlags(t *testing.T) { 1, }, { - []string{"podman", "run", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo", "alpine"}, + []string{"podman", "run", "--cgroups=foo", "--rm", "alpine"}, []string{"podman", "run", "alpine"}, 1, }, { - []string{"podman", "run", "--cgroups=foo", "--conmon-pidfile=foo", "--cidfile=foo", "alpine"}, - []string{"podman", "run", "alpine"}, - 1, - }, - { - []string{"podman", "run", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo", "alpine", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo"}, - []string{"podman", "run", "alpine", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo"}, + []string{"podman", "run", "--cgroups", "--rm=bogus", "alpine", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo", "--rm"}, + []string{"podman", "run", "alpine", "--cgroups", "foo", "--conmon-pidfile", "foo", "--cidfile", "foo", "--rm"}, 7, }, - { - []string{"podman", "run", "--cgroups=foo", "--conmon-pidfile=foo", "--cidfile=foo", "alpine", "--cgroups=foo", "--conmon-pidfile=foo", "--cidfile=foo"}, - []string{"podman", "run", "alpine", "--cgroups=foo", "--conmon-pidfile=foo", "--cidfile=foo"}, - 4, - }, } for _, test := range tests { diff --git a/pkg/systemd/generate/containers.go b/pkg/systemd/generate/containers.go index 72f321347..0e6e1b4df 100644 --- a/pkg/systemd/generate/containers.go +++ b/pkg/systemd/generate/containers.go @@ -25,6 +25,10 @@ type containerInfo struct { ServiceName string // Name or ID of the container. ContainerNameOrID string + // Type of the unit. + Type string + // NotifyAccess of the unit. + NotifyAccess string // StopTimeout sets the timeout Podman waits before killing the container // during service stop. StopTimeout uint @@ -102,10 +106,19 @@ TimeoutStopSec={{{{.TimeoutStopSec}}}} ExecStartPre={{{{.ExecStartPre}}}} {{{{- end}}}} ExecStart={{{{.ExecStart}}}} +{{{{- if .ExecStop}}}} ExecStop={{{{.ExecStop}}}} +{{{{- end}}}} +{{{{- if .ExecStopPost}}}} ExecStopPost={{{{.ExecStopPost}}}} +{{{{- end}}}} +{{{{- if .PIDFile}}}} PIDFile={{{{.PIDFile}}}} -Type=forking +{{{{- end}}}} +Type={{{{.Type}}}} +{{{{- if .NotifyAccess}}}} +NotifyAccess={{{{.NotifyAccess}}}} +{{{{- end}}}} [Install] WantedBy=multi-user.target default.target @@ -208,6 +221,7 @@ func executeContainerTemplate(info *containerInfo, options entities.GenerateSyst info.Executable = executable } + info.Type = "forking" info.EnvVariable = define.EnvVariable info.ExecStart = "{{{{.Executable}}}} start {{{{.ContainerNameOrID}}}}" info.ExecStop = "{{{{.Executable}}}} stop {{{{if (ge .StopTimeout 0)}}}}-t {{{{.StopTimeout}}}}{{{{end}}}} {{{{.ContainerNameOrID}}}}" @@ -221,8 +235,12 @@ func executeContainerTemplate(info *containerInfo, options entities.GenerateSyst // invalid `info.CreateCommand`. Hence, we're doing a best effort unit // generation and don't try aiming at completeness. if options.New { - info.PIDFile = "%t/" + info.ServiceName + ".pid" - info.ContainerIDFile = "%t/" + info.ServiceName + ".ctr-id" + info.Type = "notify" + info.NotifyAccess = "all" + info.PIDFile = "" + info.ContainerIDFile = "" + info.ExecStop = "" + info.ExecStopPost = "" // The create command must at least have three arguments: // /usr/bin/podman run $IMAGE index := 0 @@ -245,9 +263,9 @@ func executeContainerTemplate(info *containerInfo, options entities.GenerateSyst } startCommand = append(startCommand, "run", - "--conmon-pidfile", "{{{{.PIDFile}}}}", - "--cidfile", "{{{{.ContainerIDFile}}}}", + "--sdnotify=conmon", "--cgroups=no-conmon", + "--rm", ) remainingCmd := info.CreateCommand[index:] @@ -336,11 +354,7 @@ func executeContainerTemplate(info *containerInfo, options entities.GenerateSyst startCommand = append(startCommand, remainingCmd...) startCommand = escapeSystemdArguments(startCommand) - - info.ExecStartPre = "/bin/rm -f {{{{.PIDFile}}}} {{{{.ContainerIDFile}}}}" info.ExecStart = strings.Join(startCommand, " ") - info.ExecStop = "{{{{.Executable}}}} {{{{if .RootFlags}}}}{{{{ .RootFlags}}}} {{{{end}}}}stop --ignore --cidfile {{{{.ContainerIDFile}}}} {{{{if (ge .StopTimeout 0)}}}}-t {{{{.StopTimeout}}}}{{{{end}}}}" - info.ExecStopPost = "{{{{.Executable}}}} {{{{if .RootFlags}}}}{{{{ .RootFlags}}}} {{{{end}}}}rm --ignore -f --cidfile {{{{.ContainerIDFile}}}}" } info.TimeoutStopSec = minTimeoutStopSec + info.StopTimeout diff --git a/pkg/systemd/generate/containers_test.go b/pkg/systemd/generate/containers_test.go index b1070fa52..12a8f3004 100644 --- a/pkg/systemd/generate/containers_test.go +++ b/pkg/systemd/generate/containers_test.go @@ -130,12 +130,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman container run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d --replace --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN "foo=arg \"with \" space" -ExecStop=/usr/bin/podman container stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman container rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman container run --sdnotify=conmon --cgroups=no-conmon --rm -d --replace --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN "foo=arg \"with \" space" +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -155,12 +152,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon --replace -d --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm --replace -d --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -180,12 +174,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon --pod-id-file %t/pod-foobar.pod-id-file --replace -d --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm --pod-id-file %t/pod-foobar.pod-id-file --replace -d --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -205,12 +196,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon --replace --detach --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm --replace --detach --name jadda-jadda --hostname hello-world awesome-image:latest command arg1 ... argN +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -230,12 +218,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.pid %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.pid --cidfile %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.ctr-id --cgroups=no-conmon -d awesome-image:latest -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.ctr-id -PIDFile=%t/container-639c53578af4d84b8800b4635fa4e680ee80fd67e0e6a2d4eea48d1e3230f401.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d awesome-image:latest +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -256,14 +241,11 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=102 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon ` + +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm ` + detachparam + ` awesome-image:latest -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 42 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -285,12 +267,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=102 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d --replace --name test -p 80:80 awesome-image:latest somecmd --detach=false -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 42 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d --replace --name test -p 80:80 awesome-image:latest somecmd --detach=false +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -310,12 +289,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=102 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman --events-backend none --runroot /root run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d awesome-image:latest -ExecStop=/usr/bin/podman --events-backend none --runroot /root stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 42 -ExecStopPost=/usr/bin/podman --events-backend none --runroot /root rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman --events-backend none --runroot /root run --sdnotify=conmon --cgroups=no-conmon --rm -d awesome-image:latest +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -335,12 +311,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman container run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d awesome-image:latest -ExecStop=/usr/bin/podman container stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman container rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman container run --sdnotify=conmon --cgroups=no-conmon --rm -d awesome-image:latest +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -360,12 +333,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d --replace --name test --log-driver=journald --log-opt=tag={{.Name}} awesome-image:latest -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d --replace --name test --log-driver=journald --log-opt=tag={{.Name}} awesome-image:latest +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -385,12 +355,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d --replace --name test awesome-image:latest sh -c "kill $$$$ && echo %%\\" -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d --replace --name test awesome-image:latest sh -c "kill $$$$ && echo %%\\" +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -410,12 +377,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d awesome-image:latest podman run --cgroups=foo --conmon-pidfile=foo --cidfile=foo alpine -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d --conmon-pidfile=foo --cidfile=foo awesome-image:latest podman run --cgroups=foo --conmon-pidfile=foo --cidfile=foo alpine +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -435,12 +399,9 @@ RequiresMountsFor=/var/run/containers/storage Environment=PODMAN_SYSTEMD_UNIT=%n Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon --pod-id-file %t/pod-foobar.pod-id-file -d awesome-image:latest podman run --cgroups=foo --conmon-pidfile=foo --cidfile=foo --pod-id-file /tmp/pod-foobar.pod-id-file alpine -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm --pod-id-file %t/pod-foobar.pod-id-file -d --conmon-pidfile=foo --cidfile=foo awesome-image:latest podman run --cgroups=foo --conmon-pidfile=foo --cidfile=foo --pod-id-file /tmp/pod-foobar.pod-id-file alpine +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -461,12 +422,9 @@ Environment=PODMAN_SYSTEMD_UNIT=%n Environment=FOO=abc "BAR=my test" USER=%%a Restart=always TimeoutStopSec=70 -ExecStartPre=/bin/rm -f %t/jadda-jadda.pid %t/jadda-jadda.ctr-id -ExecStart=/usr/bin/podman run --conmon-pidfile %t/jadda-jadda.pid --cidfile %t/jadda-jadda.ctr-id --cgroups=no-conmon -d --env FOO --env=BAR --env=MYENV=2 -e USER awesome-image:latest -ExecStop=/usr/bin/podman stop --ignore --cidfile %t/jadda-jadda.ctr-id -t 10 -ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/jadda-jadda.ctr-id -PIDFile=%t/jadda-jadda.pid -Type=forking +ExecStart=/usr/bin/podman run --sdnotify=conmon --cgroups=no-conmon --rm -d --env FOO --env=BAR --env=MYENV=2 -e USER awesome-image:latest +Type=notify +NotifyAccess=all [Install] WantedBy=multi-user.target default.target @@ -929,10 +887,10 @@ WantedBy=multi-user.target default.target } got, err := executeContainerTemplate(&test.info, opts) if (err != nil) != test.wantErr { - t.Errorf("CreateContainerSystemdUnit() error = \n%v, wantErr \n%v", err, test.wantErr) + t.Errorf("CreateContainerSystemdUnit() %s error = \n%v, wantErr \n%v", test.name, err, test.wantErr) return } - assert.Equal(t, test.want, got) + assert.Equal(t, test.want, got, test.name) }) } } diff --git a/test/apiv2/python/rest_api/fixtures/api_testcase.py b/test/apiv2/python/rest_api/fixtures/api_testcase.py index 8b771774b..155e93928 100644 --- a/test/apiv2/python/rest_api/fixtures/api_testcase.py +++ b/test/apiv2/python/rest_api/fixtures/api_testcase.py @@ -49,7 +49,7 @@ class APITestCase(unittest.TestCase): def setUp(self): super().setUp() - APITestCase.podman.run("run", "alpine", "/bin/ls", check=True) + APITestCase.podman.run("run", "-d", "alpine", "top", check=True) def tearDown(self) -> None: APITestCase.podman.run("pod", "rm", "--all", "--force", check=True) diff --git a/test/apiv2/python/rest_api/test_v2_0_0_container.py b/test/apiv2/python/rest_api/test_v2_0_0_container.py index ad096ed38..b4b3af2df 100644 --- a/test/apiv2/python/rest_api/test_v2_0_0_container.py +++ b/test/apiv2/python/rest_api/test_v2_0_0_container.py @@ -12,7 +12,7 @@ class ContainerTestCase(APITestCase): r = requests.get(self.uri("/containers/json"), timeout=5) self.assertEqual(r.status_code, 200, r.text) obj = r.json() - self.assertEqual(len(obj), 0) + self.assertEqual(len(obj), 1) def test_list_all(self): r = requests.get(self.uri("/containers/json?all=true")) @@ -30,9 +30,13 @@ class ContainerTestCase(APITestCase): self.assertIn(r.status_code, (200, 409), r.text) if r.status_code == 200: self.assertId(r.content) + r = requests.get(self.uri(self.resolve_container("/containers/{}/stats?stream=false&one-shot=true"))) + self.assertIn(r.status_code, (200, 409), r.text) + if r.status_code == 200: + self.assertId(r.content) def test_delete(self): - r = requests.delete(self.uri(self.resolve_container("/containers/{}"))) + r = requests.delete(self.uri(self.resolve_container("/containers/{}?force=true"))) self.assertEqual(r.status_code, 204, r.text) def test_stop(self): diff --git a/test/e2e/checkpoint_test.go b/test/e2e/checkpoint_test.go index 9d0049910..70a1d09ed 100644 --- a/test/e2e/checkpoint_test.go +++ b/test/e2e/checkpoint_test.go @@ -425,6 +425,106 @@ var _ = Describe("Podman checkpoint", func() { // Remove exported checkpoint os.Remove(fileName) }) + // This test does the same steps which are necessary for migrating + // a container from one host to another + It("podman checkpoint container with export and different compression algorithms", func() { + localRunString := getRunString([]string{"--rm", ALPINE, "top"}) + session := podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + cid := session.OutputToString() + fileName := "/tmp/checkpoint-" + cid + ".tar" + + // Checkpoint with the default algorithm + result := podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName}) + result.WaitWithDefaultTimeout() + + // As the container has been started with '--rm' it will be completely + // cleaned up after checkpointing. + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Restore container + result = podmanTest.Podman([]string{"container", "restore", "-i", fileName}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Checkpoint with the zstd algorithm + result = podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName, "--compress", "zstd"}) + result.WaitWithDefaultTimeout() + + // As the container has been started with '--rm' it will be completely + // cleaned up after checkpointing. + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Restore container + result = podmanTest.Podman([]string{"container", "restore", "-i", fileName}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Checkpoint with the none algorithm + result = podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName, "-c", "none"}) + result.WaitWithDefaultTimeout() + + // As the container has been started with '--rm' it will be completely + // cleaned up after checkpointing. + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Restore container + result = podmanTest.Podman([]string{"container", "restore", "-i", fileName}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Checkpoint with the gzip algorithm + result = podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName, "-c", "gzip"}) + result.WaitWithDefaultTimeout() + + // As the container has been started with '--rm' it will be completely + // cleaned up after checkpointing. + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Restore container + result = podmanTest.Podman([]string{"container", "restore", "-i", fileName}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Checkpoint with the non-existing algorithm + result = podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName, "-c", "non-existing"}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(125)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.NumberOfContainers()).To(Equal(1)) + + result = podmanTest.Podman([]string{"rm", "-fa"}) + result.WaitWithDefaultTimeout() + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Remove exported checkpoint + os.Remove(fileName) + }) It("podman checkpoint and restore container with root file-system changes", func() { // Start the container @@ -822,4 +922,58 @@ var _ = Describe("Podman checkpoint", func() { os.Remove(checkpointFileName) os.Remove(preCheckpointFileName) }) + + It("podman checkpoint and restore container with different port mappings", func() { + localRunString := getRunString([]string{"-p", "1234:6379", "--rm", redis}) + session := podmanTest.Podman(localRunString) + session.WaitWithDefaultTimeout() + Expect(session.ExitCode()).To(Equal(0)) + cid := session.OutputToString() + fileName := "/tmp/checkpoint-" + cid + ".tar.gz" + + // Open a network connection to the redis server via initial port mapping + conn, err := net.Dial("tcp", "localhost:1234") + if err != nil { + os.Exit(1) + } + conn.Close() + + // Checkpoint the container + result := podmanTest.Podman([]string{"container", "checkpoint", "-l", "-e", fileName}) + result.WaitWithDefaultTimeout() + + // As the container has been started with '--rm' it will be completely + // cleaned up after checkpointing. + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Restore container with different port mapping + result = podmanTest.Podman([]string{"container", "restore", "-p", "1235:6379", "-i", fileName}) + result.WaitWithDefaultTimeout() + + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) + Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) + + // Open a network connection to the redis server via initial port mapping + // This should fail + conn, err = net.Dial("tcp", "localhost:1234") + Expect(err.Error()).To(ContainSubstring("connection refused")) + // Open a network connection to the redis server via new port mapping + conn, err = net.Dial("tcp", "localhost:1235") + if err != nil { + os.Exit(1) + } + conn.Close() + + result = podmanTest.Podman([]string{"rm", "-fa"}) + result.WaitWithDefaultTimeout() + Expect(result.ExitCode()).To(Equal(0)) + Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) + + // Remove exported checkpoint + os.Remove(fileName) + }) }) diff --git a/test/e2e/events_test.go b/test/e2e/events_test.go index 4dbbe9dd8..cc7c4d996 100644 --- a/test/e2e/events_test.go +++ b/test/e2e/events_test.go @@ -8,6 +8,7 @@ import ( "sync" "time" + "github.com/containers/podman/v3/libpod/events" . "github.com/containers/podman/v3/test/utils" "github.com/containers/storage/pkg/stringid" . "github.com/onsi/ginkgo" @@ -134,12 +135,10 @@ var _ = Describe("Podman events", func() { jsonArr := test.OutputToStringArray() Expect(test.OutputToStringArray()).ShouldNot(BeEmpty()) - eventsMap := make(map[string]string) - err := json.Unmarshal([]byte(jsonArr[0]), &eventsMap) + event := events.Event{} + err := json.Unmarshal([]byte(jsonArr[0]), &event) Expect(err).ToNot(HaveOccurred()) - Expect(eventsMap).To(HaveKey("Status")) - test = podmanTest.Podman([]string{"events", "--stream=false", "--format", "{{json.}}"}) test.WaitWithDefaultTimeout() Expect(test).To(Exit(0)) @@ -147,11 +146,9 @@ var _ = Describe("Podman events", func() { jsonArr = test.OutputToStringArray() Expect(test.OutputToStringArray()).ShouldNot(BeEmpty()) - eventsMap = make(map[string]string) - err = json.Unmarshal([]byte(jsonArr[0]), &eventsMap) + event = events.Event{} + err = json.Unmarshal([]byte(jsonArr[0]), &event) Expect(err).ToNot(HaveOccurred()) - - Expect(eventsMap).To(HaveKey("Status")) }) It("podman events --until future", func() { diff --git a/test/e2e/generate_systemd_test.go b/test/e2e/generate_systemd_test.go index 75d778f10..e03d6899e 100644 --- a/test/e2e/generate_systemd_test.go +++ b/test/e2e/generate_systemd_test.go @@ -215,7 +215,6 @@ var _ = Describe("Podman generate systemd", func() { // Grepping the output (in addition to unit tests) Expect(session.OutputToString()).To(ContainSubstring("# container-foo.service")) Expect(session.OutputToString()).To(ContainSubstring(" --replace ")) - Expect(session.OutputToString()).To(ContainSubstring(" stop --ignore --cidfile %t/container-foo.ctr-id -t 42")) if !IsRemote() { // The podman commands in the unit should contain the root flags if generate systemd --new is used Expect(session.OutputToString()).To(ContainSubstring(" --runroot")) @@ -234,7 +233,6 @@ var _ = Describe("Podman generate systemd", func() { // Grepping the output (in addition to unit tests) Expect(session.OutputToString()).To(ContainSubstring("# container-foo.service")) Expect(session.OutputToString()).To(ContainSubstring(" --replace ")) - Expect(session.OutputToString()).To(ContainSubstring(" stop --ignore --cidfile %t/container-foo.ctr-id -t 42")) }) It("podman generate systemd --new without explicit detaching param", func() { @@ -247,7 +245,7 @@ var _ = Describe("Podman generate systemd", func() { Expect(session.ExitCode()).To(Equal(0)) // Grepping the output (in addition to unit tests) - Expect(session.OutputToString()).To(ContainSubstring("--cgroups=no-conmon -d")) + Expect(session.OutputToString()).To(ContainSubstring(" -d ")) }) It("podman generate systemd --new with explicit detaching param in middle", func() { diff --git a/test/system/030-run.bats b/test/system/030-run.bats index 2ea981a85..32fc85c4e 100644 --- a/test/system/030-run.bats +++ b/test/system/030-run.bats @@ -600,12 +600,12 @@ json-file | f echo "$randomcontent" > $testdir/content # Workdir does not exist on the image but is volume mounted. - run_podman run --rm --workdir /IamNotOnTheImage -v $testdir:/IamNotOnTheImage $IMAGE cat content + run_podman run --rm --workdir /IamNotOnTheImage -v $testdir:/IamNotOnTheImage:Z $IMAGE cat content is "$output" "$randomcontent" "cat random content" # Workdir does not exist on the image but is created by the runtime as it's # a subdir of a volume. - run_podman run --rm --workdir /IamNotOntheImage -v $testdir/content:/IamNotOntheImage/foo $IMAGE cat foo + run_podman run --rm --workdir /IamNotOntheImage -v $testdir/content:/IamNotOntheImage/foo:Z $IMAGE cat foo is "$output" "$randomcontent" "cat random content" # Make sure that running on a read-only rootfs works (#9230). @@ -702,6 +702,8 @@ EOF run_podman build -t nomtab $tmpdir run_podman run --rm nomtab stat -c %N /etc/mtab is "$output" "$expected" "/etc/mtab should be created" + + run_podman rmi nomtab } # vim: filetype=sh diff --git a/test/system/090-events.bats b/test/system/090-events.bats index 19bee5506..d889bd7f9 100644 --- a/test/system/090-events.bats +++ b/test/system/090-events.bats @@ -6,7 +6,6 @@ load helpers @test "events with a filter by label" { - skip_if_remote "FIXME: -remote does not include labels in event output" cname=test-$(random_string 30 | tr A-Z a-z) labelname=$(random_string 10) labelvalue=$(random_string 15) @@ -27,7 +26,7 @@ load helpers } @test "image events" { - skip_if_remote "FIXME: remove events on podman-remote seem to be broken" + skip_if_remote "remote does not support --events-backend" pushedDir=$PODMAN_TMPDIR/dir mkdir -p $pushedDir @@ -61,3 +60,30 @@ load helpers .*image remove $imageID $tag.*" \ "podman events" } + +function _events_disjunctive_filters() { + local backend=$1 + + # Regression test for #10507: make sure that filters with the same key are + # applied in disjunction. + t0=$(date --iso-8601=seconds) + run_podman $backend run --name foo --rm $IMAGE ls + run_podman $backend run --name bar --rm $IMAGE ls + run_podman $backend events --stream=false --since=$t0 --filter container=foo --filter container=bar --filter event=start + is "$output" ".* container start .* name=foo.* +.* container start .* name=bar.*" +} + +@test "events with disjunctive filters - file" { + skip_if_remote "remote does not support --events-backend" + _events_disjunctive_filters --events-backend=file +} + +@test "events with disjunctive filters - journald" { + skip_if_remote "remote does not support --events-backend" + _events_disjunctive_filters --events-backend=journald +} + +@test "events with disjunctive filters - default" { + _events_disjunctive_filters "" +} diff --git a/test/system/255-auto-update.bats b/test/system/255-auto-update.bats new file mode 100644 index 000000000..59f53f775 --- /dev/null +++ b/test/system/255-auto-update.bats @@ -0,0 +1,279 @@ +#!/usr/bin/env bats -*- bats -*- +# +# Tests for automatically update images for containerized services +# + +load helpers + +UNIT_DIR="/usr/lib/systemd/system" +SNAME_FILE=$BATS_TMPDIR/services + +function setup() { + skip_if_remote "systemd tests are meaningless over remote" + skip_if_rootless + + basic_setup +} + +function teardown() { + while read line; do + if [[ "$line" =~ "podman-auto-update" ]]; then + echo "Stop timer: $line.timer" + systemctl stop $line.timer + systemctl disable $line.timer + else + systemctl stop $line + fi + rm -f $UNIT_DIR/$line.{service,timer} + done < $SNAME_FILE + + rm -f $SNAME_FILE + run_podman ? rmi quay.io/libpod/alpine:latest + run_podman ? rmi quay.io/libpod/alpine_nginx:latest + run_podman ? rmi quay.io/libpod/localtest:latest + basic_teardown +} + +# This functions is used for handle the basic step in auto-update related +# tests. Including following steps: +# 1. Generate a random container name and echo it to output. +# 2. Tag the fake image before test +# 3. Start a container with io.containers.autoupdate +# 4. Generate the service file from the container +# 5. Remove the origin container +# 6. Start the container from service +function generate_service() { + target_img_basename=$1 + autoupdate=$2 + + # Please keep variable name for cname and ori_image. The + # scripts will use them directly in following tests. + cname=c_$(random_string) + target_img="quay.io/libpod/$target_img_basename:latest" + run_podman tag $IMAGE $target_img + if [[ -n "$autoupdate" ]]; then + label="--label io.containers.autoupdate=$autoupdate" + else + label="" + fi + run_podman run -d --name $cname $label $target_img top -d 120 + + run_podman generate systemd --new $cname + echo "$output" > "$UNIT_DIR/container-$cname.service" + echo "container-$cname" >> $SNAME_FILE + run_podman rm -f $cname + + systemctl daemon-reload + systemctl start container-$cname + systemctl status container-$cname + + run_podman inspect --format "{{.Image}}" $cname + ori_image=$output +} + +function _wait_service_ready() { + local sname=$1 + + local timeout=6 + while [[ $timeout -gt 1 ]]; do + run systemctl is-active $sname + if [[ $output == "active" ]]; then + return + fi + sleep 1 + let timeout=$timeout-1 + done + + # Print serivce status as debug information before failed the case + systemctl status $sname + die "Timed out waiting for $sname to start" +} + +function _confirm_update() { + local sname=$1 + + local timeout=6 + last_log="" + while [[ $timeout -gt 1 ]]; do + run journalctl -u $sname -n 10 + if [[ "$output" == "$last_log" ]]; then + return + fi + last_log=$output + sleep 1 + let timeout=$timeout-1 + done + + die "Timed out waiting for $sname to update" +} + +# This test can fail in dev. environment because of SELinux. +# quick fix: chcon -t container_runtime_exec_t ./bin/podman +@test "podman auto-update - label io.containers.autoupdate=image" { + run_podman images + generate_service alpine image + + _wait_service_ready container-$cname.service + run_podman ps -a + run_podman auto-update + is "$output" "Trying to pull.*" "Image is updated." + run_podman ps -a + _confirm_update container-$cname.service + run_podman inspect --format "{{.Image}}" $cname + [[ "$output" != "$ori_image" ]] +} + +@test "podman auto-update - label io.containers.autoupdate=disabled" { + generate_service alpine disabled + + _wait_service_ready container-$cname.service + run_podman ps -a + run_podman auto-update + is "$output" "" "Image is not updated with disabled." + run_podman ps -a + _confirm_update container-$cname.service + run_podman inspect --format "{{.Image}}" $cname + is "$output" "$ori_image" "Image hash should not changed." +} + +@test "podman auto-update - label io.containers.autoupdate=fakevalue" { + fakevalue=$(random_string) + generate_service alpine $fakevalue + + _wait_service_ready container-$cname.service + run_podman ps -a + run_podman ? auto-update + is "$output" ".*invalid auto-update policy.*" "invalid policy setup" + run_podman ps -a + _confirm_update container-$cname.service + run_podman inspect --format "{{.Image}}" $cname + is "$output" "$ori_image" "Image hash should not changed." +} + +@test "podman auto-update - label io.containers.autoupdate=local" { + generate_service localtest local + podman commit --change CMD=/bin/bash $cname quay.io/libpod/localtest:latest + + _wait_service_ready container-$cname.service + run_podman ps -a + run_podman auto-update + run_podman ps -a + _confirm_update container-$cname.service + run_podman inspect --format "{{.Image}}" $cname + [[ "$output" != "$ori_image" ]] +} + +@test "podman auto-update with multiple services" { + fakevalue=$(random_string) + run_podman inspect --format "{{.Id}}" $IMAGE + img_id="$output" + cnames=() + local -A expect_update + local -A will_update=([image]=1 [registry]=1 [local]=1) + + for auto_update in image registry "" disabled "''" $fakevalue local + do + img_base="alpine" + if [[ $auto_update == "registry" ]]; then + img_base="alpine_nginx" + elif [[ $auto_update == "local" ]]; then + img_base="localtest" + fi + generate_service $img_base $auto_update + cnames+=($cname) + if [[ $auto_update == "local" ]]; then + local_cname=$cname + fi + if [[ -n "$auto_update" && -n "${will_update[$auto_update]}" ]]; then + expect_update[$cname]=1 + fi + done + + # Only check the last service is started. Previous services should already actived. + _wait_service_ready container-$cname.service + run_podman commit --change CMD=/bin/bash $local_cname quay.io/libpod/localtest:latest + run_podman ? auto-update + update_log=$output + for cname in "${cnames[@]}"; do + _confirm_update container-$cname.service + done + count=0 + while read line; do + if [[ "$line" =~ "Trying to pull" ]]; then + ((count+=1)) + fi + done <<< "$update_log" + is "$update_log" ".*invalid auto-update policy.*" "invalid policy setup" + is "$update_log" ".*1 error occurred.*" "invalid policy setup" + is "$count" "2" "There are two images being updated from registry." + + for cname in "${!expect_update[@]}"; do + + is "$update_log" ".*$cname.*" "container with auto-update policy image updated" + done + + for cname in "${cnames[@]}"; do + run_podman inspect --format "{{.Image}}" $cname + if [[ -n "${expect_update[$cname]}" ]]; then + [[ "$output" != "$img_id" ]] + else + is "$output" "$img_id" "Image should not be changed." + fi + done +} + +@test "podman auto-update using systemd" { + generate_service alpine image + + cat >$UNIT_DIR/podman-auto-update-$cname.timer <<EOF +[Unit] +Description=Podman auto-update testing timer + +[Timer] +OnCalendar=*-*-* *:*:0/2 +Persistent=true + +[Install] +WantedBy=timers.target +EOF + cat >$UNIT_DIR/podman-auto-update-$cname.service <<EOF +[Unit] +Description=Podman auto-update testing service +Documentation=man:podman-auto-update(1) +Wants=network.target +After=network-online.target + +[Service] +Type=oneshot +ExecStart=/usr/bin/podman auto-update + +[Install] +WantedBy=multi-user.target default.target +EOF + + echo "podman-auto-update-$cname" >> $SNAME_FILE + systemctl enable --now podman-auto-update-$cname.timer + systemctl list-timers --all + + count=0 + failed_start=1 + while [ $count -lt 120 ]; do + run journalctl -n 15 -u podman-auto-update-$cname.service + if [[ "$output" =~ "Finished Podman auto-update testing service" ]]; then + failed_start=0 + break + fi + ((count+=1)) + sleep 1 + done + echo $output + + _confirm_update container-$cname.service + run_podman inspect --format "{{.Image}}" $cname + if [[ $failed_start == 1 ]]; then + die "Failed to get podman auto-update service finished" + fi + [[ "$output" != "$ori_image" ]] +} + +# vim: filetype=sh diff --git a/test/system/450-interactive.bats b/test/system/450-interactive.bats index a9bf52ee8..a2db39492 100644 --- a/test/system/450-interactive.bats +++ b/test/system/450-interactive.bats @@ -56,8 +56,7 @@ function teardown() { stty rows $rows cols $cols <$PODMAN_TEST_PTY # ...and make sure stty under podman reads that. - # FIXME: 'sleep 1' is needed for podman-remote; without it, there's - run_podman run -it --name mystty $IMAGE sh -c 'sleep 1;stty size' <$PODMAN_TEST_PTY + run_podman run -it --name mystty $IMAGE stty size <$PODMAN_TEST_PTY is "$output" "$rows $cols" "stty under podman reads the correct dimensions" } diff --git a/test/system/500-networking.bats b/test/system/500-networking.bats index 1cec50827..55ec80bb2 100644 --- a/test/system/500-networking.bats +++ b/test/system/500-networking.bats @@ -34,7 +34,7 @@ load helpers # Bind-mount this file with a different name to a container running httpd run_podman run -d --name myweb -p "$HOST_PORT:80" \ --restart always \ - -v $INDEX1:/var/www/index.txt \ + -v $INDEX1:/var/www/index.txt:Z \ -w /var/www \ $IMAGE /bin/busybox-extras httpd -f -p 80 cid=$output @@ -257,7 +257,7 @@ load helpers # Bind-mount this file with a different name to a container running httpd run_podman run -d --name myweb -p "$HOST_PORT:80" \ --network $netname \ - -v $INDEX1:/var/www/index.txt \ + -v $INDEX1:/var/www/index.txt:Z \ -w /var/www \ $IMAGE /bin/busybox-extras httpd -f -p 80 cid=$output @@ -329,4 +329,62 @@ load helpers run_podman network rm -f $mynetname } +@test "podman ipv6 in /etc/resolv.conf" { + ipv6_regex='([0-9A-Fa-f]{0,4}:){2,7}([0-9A-Fa-f]{0,4})(%\w+)?' + + # Make sure to read the correct /etc/resolv.conf file in case of systemd-resolved. + resolve_file=$(readlink -f /etc/resolv.conf) + if [[ "$resolve_file" == "/run/systemd/resolve/stub-resolv.conf" ]]; then + resolve_file="/run/systemd/resolve/resolv.conf" + fi + + # If the host doesn't have an ipv6 in resolv.conf skip this test. + # We should never modify resolv.conf on the host. + if ! grep -E "$ipv6_regex" "$resolve_file"; then + skip "This test needs an ipv6 nameserver in $resolve_file" + fi + + # ipv4 slirp + run_podman run --rm --network slirp4netns:enable_ipv6=false $IMAGE cat /etc/resolv.conf + if grep -E "$ipv6_regex" <<< $output; then + die "resolv.conf contains a ipv6 nameserver" + fi + + # ipv6 slirp + run_podman run --rm --network slirp4netns:enable_ipv6=true $IMAGE cat /etc/resolv.conf + # "is" does not like the ipv6 regex + if ! grep -E "$ipv6_regex" <<< $output; then + die "resolv.conf does not contain a ipv6 nameserver" + fi + + # ipv4 cni + local mysubnet=$(random_rfc1918_subnet) + local netname=testnet-$(random_string 10) + + run_podman network create --subnet $mysubnet.0/24 $netname + is "$output" ".*/cni/net.d/$netname.conflist" "output of 'network create'" + + run_podman run --rm --network $netname $IMAGE cat /etc/resolv.conf + if grep -E "$ipv6_regex" <<< $output; then + die "resolv.conf contains a ipv6 nameserver" + fi + + run_podman network rm -f $netname + + # ipv6 cni + mysubnet=fd00:4:4:4:4::/64 + netname=testnet-$(random_string 10) + + run_podman network create --subnet $mysubnet $netname + is "$output" ".*/cni/net.d/$netname.conflist" "output of 'network create'" + + run_podman run --rm --network $netname $IMAGE cat /etc/resolv.conf + # "is" does not like the ipv6 regex + if ! grep -E "$ipv6_regex" <<< $output; then + die "resolv.conf does not contain a ipv6 nameserver" + fi + + run_podman network rm -f $netname +} + # vim: filetype=sh diff --git a/troubleshooting.md b/troubleshooting.md index e320f20e7..ab9fffeb3 100644 --- a/troubleshooting.md +++ b/troubleshooting.md @@ -697,3 +697,32 @@ limits. This can happen when running a container from an image for another architecture than the one you are running on. For example, if a remote repository only has, and thus send you, a `linux/arm64` _OS/ARCH_ but you run on `linux/amd64` (as happened in https://github.com/openMF/community-app/issues/3323 due to https://github.com/timbru31/docker-ruby-node/issues/564). + +### 27) `Error: failed to create sshClient: Connection to bastion host (ssh://user@host:22/run/user/.../podman/podman.sock) failed.: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain` + +In some situations where the client is not on the same machine as where the podman daemon is running the client key could be using a cipher not supported by the host. This indicates an issue with one's SSH config. Until remedied using podman over ssh +with a pre-shared key will be impossible. + +#### Symptom + +The accepted ciphers per `/etc/crypto-policies/back-ends/openssh.config` are not one that was used to create the public/private key pair that was transferred over to the host for ssh authentication. + +You can confirm this is the case by attempting to connect to the host via `podman-remote info` from the client and simultaneously on the host running `journalctl -f` and watching for the error `userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]`. + +#### Solution + +Create a new key using a supported algorithm e.g. ecdsa: + +`ssh-keygen -t ecdsa -f ~/.ssh/podman` + +Then copy the new id over: + +`ssh-copy-id -i ~/.ssh/podman.pub user@host` + +And then re-add the connection (removing the old one if necessary): + +`podman-remote system connection add myuser --identity ~/.ssh/podman ssh://user@host/run/user/1000/podman/podman.sock` + +And now this should work: + +`podman-remote info` diff --git a/vendor/github.com/willf/bitset/.gitignore b/vendor/github.com/bits-and-blooms/bitset/.gitignore index 5c204d28b..5c204d28b 100644 --- a/vendor/github.com/willf/bitset/.gitignore +++ b/vendor/github.com/bits-and-blooms/bitset/.gitignore diff --git a/vendor/github.com/willf/bitset/.travis.yml b/vendor/github.com/bits-and-blooms/bitset/.travis.yml index 094aa5ce0..094aa5ce0 100644 --- a/vendor/github.com/willf/bitset/.travis.yml +++ b/vendor/github.com/bits-and-blooms/bitset/.travis.yml diff --git a/vendor/github.com/willf/bitset/LICENSE b/vendor/github.com/bits-and-blooms/bitset/LICENSE index 59cab8a93..59cab8a93 100644 --- a/vendor/github.com/willf/bitset/LICENSE +++ b/vendor/github.com/bits-and-blooms/bitset/LICENSE diff --git a/vendor/github.com/willf/bitset/README.md b/vendor/github.com/bits-and-blooms/bitset/README.md index 50338e71d..97e83071e 100644 --- a/vendor/github.com/willf/bitset/README.md +++ b/vendor/github.com/bits-and-blooms/bitset/README.md @@ -2,10 +2,9 @@ *Go language library to map between non-negative integers and boolean values* -[](https://github.com/willf/bitset/actions?query=workflow%3ATest) -[](https://coveralls.io/github/willf/bitset?branch=master) +[](https://github.com/willf/bitset/actions?query=workflow%3ATest) [](https://goreportcard.com/report/github.com/willf/bitset) -[](https://pkg.go.dev/github.com/willf/bitset?tab=doc) +[](https://pkg.go.dev/github.com/bits-and-blooms/bitset?tab=doc) ## Description @@ -30,7 +29,7 @@ import ( "fmt" "math/rand" - "github.com/willf/bitset" + "github.com/bits-and-blooms/bitset" ) func main() { @@ -63,7 +62,7 @@ func main() { As an alternative to BitSets, one should check out the 'big' package, which provides a (less set-theoretical) view of bitsets. -Package documentation is at: https://pkg.go.dev/github.com/willf/bitset?tab=doc +Package documentation is at: https://pkg.go.dev/github.com/bits-and-blooms/bitset?tab=doc ## Memory Usage @@ -78,7 +77,7 @@ It is possible that a later version will match the `math/bits` return signature ## Installation ```bash -go get github.com/willf/bitset +go get github.com/bits-and-blooms/bitset ``` ## Contributing diff --git a/vendor/github.com/willf/bitset/azure-pipelines.yml b/vendor/github.com/bits-and-blooms/bitset/azure-pipelines.yml index f9b295918..f9b295918 100644 --- a/vendor/github.com/willf/bitset/azure-pipelines.yml +++ b/vendor/github.com/bits-and-blooms/bitset/azure-pipelines.yml diff --git a/vendor/github.com/willf/bitset/bitset.go b/vendor/github.com/bits-and-blooms/bitset/bitset.go index 21e889da2..d688806a5 100644 --- a/vendor/github.com/willf/bitset/bitset.go +++ b/vendor/github.com/bits-and-blooms/bitset/bitset.go @@ -209,6 +209,27 @@ func (b *BitSet) Flip(i uint) *BitSet { return b } +// FlipRange bit in [start, end). +// If end>= Cap(), this function will panic. +// Warning: using a very large value for 'end' +// may lead to a memory shortage and a panic: the caller is responsible +// for providing sensible parameters in line with their memory capacity. +func (b *BitSet) FlipRange(start, end uint) *BitSet { + if start >= end { + return b + } + + b.extendSetMaybe(end - 1) + var startWord uint = start >> log2WordSize + var endWord uint = end >> log2WordSize + b.set[startWord] ^= ^(^uint64(0) << (start & (wordSize - 1))) + for i := startWord; i < endWord; i++ { + b.set[i] = ^b.set[i] + } + b.set[endWord] ^= ^uint64(0) >> (-end & (wordSize - 1)) + return b +} + // Shrink shrinks BitSet so that the provided value is the last possible // set value. It clears all bits > the provided index and reduces the size // and length of the set. @@ -519,7 +540,7 @@ func (b *BitSet) Copy(c *BitSet) (count uint) { } // Count (number of set bits). -// Also known as "popcount" or "popularity count". +// Also known as "popcount" or "population count". func (b *BitSet) Count() uint { if b != nil && b.set != nil { return uint(popcntSlice(b.set)) diff --git a/vendor/github.com/bits-and-blooms/bitset/go.mod b/vendor/github.com/bits-and-blooms/bitset/go.mod new file mode 100644 index 000000000..c43e4522b --- /dev/null +++ b/vendor/github.com/bits-and-blooms/bitset/go.mod @@ -0,0 +1,3 @@ +module github.com/bits-and-blooms/bitset + +go 1.14 diff --git a/vendor/github.com/willf/bitset/go.sum b/vendor/github.com/bits-and-blooms/bitset/go.sum index e69de29bb..e69de29bb 100644 --- a/vendor/github.com/willf/bitset/go.sum +++ b/vendor/github.com/bits-and-blooms/bitset/go.sum diff --git a/vendor/github.com/willf/bitset/popcnt.go b/vendor/github.com/bits-and-blooms/bitset/popcnt.go index 76577a838..76577a838 100644 --- a/vendor/github.com/willf/bitset/popcnt.go +++ b/vendor/github.com/bits-and-blooms/bitset/popcnt.go diff --git a/vendor/github.com/willf/bitset/popcnt_19.go b/vendor/github.com/bits-and-blooms/bitset/popcnt_19.go index fc8ff4f36..fc8ff4f36 100644 --- a/vendor/github.com/willf/bitset/popcnt_19.go +++ b/vendor/github.com/bits-and-blooms/bitset/popcnt_19.go diff --git a/vendor/github.com/willf/bitset/popcnt_amd64.go b/vendor/github.com/bits-and-blooms/bitset/popcnt_amd64.go index 4cf64f24a..4cf64f24a 100644 --- a/vendor/github.com/willf/bitset/popcnt_amd64.go +++ b/vendor/github.com/bits-and-blooms/bitset/popcnt_amd64.go diff --git a/vendor/github.com/willf/bitset/popcnt_amd64.s b/vendor/github.com/bits-and-blooms/bitset/popcnt_amd64.s index 666c0dcc1..666c0dcc1 100644 --- a/vendor/github.com/willf/bitset/popcnt_amd64.s +++ b/vendor/github.com/bits-and-blooms/bitset/popcnt_amd64.s diff --git a/vendor/github.com/willf/bitset/popcnt_generic.go b/vendor/github.com/bits-and-blooms/bitset/popcnt_generic.go index 21e0ff7b4..21e0ff7b4 100644 --- a/vendor/github.com/willf/bitset/popcnt_generic.go +++ b/vendor/github.com/bits-and-blooms/bitset/popcnt_generic.go diff --git a/vendor/github.com/willf/bitset/trailing_zeros_18.go b/vendor/github.com/bits-and-blooms/bitset/trailing_zeros_18.go index c52b61be9..c52b61be9 100644 --- a/vendor/github.com/willf/bitset/trailing_zeros_18.go +++ b/vendor/github.com/bits-and-blooms/bitset/trailing_zeros_18.go diff --git a/vendor/github.com/willf/bitset/trailing_zeros_19.go b/vendor/github.com/bits-and-blooms/bitset/trailing_zeros_19.go index 36a988e71..36a988e71 100644 --- a/vendor/github.com/willf/bitset/trailing_zeros_19.go +++ b/vendor/github.com/bits-and-blooms/bitset/trailing_zeros_19.go diff --git a/vendor/github.com/docker/docker/pkg/archive/archive.go b/vendor/github.com/docker/docker/pkg/archive/archive.go index 8d14b7869..50b83c62c 100644 --- a/vendor/github.com/docker/docker/pkg/archive/archive.go +++ b/vendor/github.com/docker/docker/pkg/archive/archive.go @@ -402,10 +402,24 @@ func fillGo18FileTypeBits(mode int64, fi os.FileInfo) int64 { // ReadSecurityXattrToTarHeader reads security.capability xattr from filesystem // to a tar header func ReadSecurityXattrToTarHeader(path string, hdr *tar.Header) error { + const ( + // Values based on linux/include/uapi/linux/capability.h + xattrCapsSz2 = 20 + versionOffset = 3 + vfsCapRevision2 = 2 + vfsCapRevision3 = 3 + ) capability, _ := system.Lgetxattr(path, "security.capability") if capability != nil { + length := len(capability) + if capability[versionOffset] == vfsCapRevision3 { + // Convert VFS_CAP_REVISION_3 to VFS_CAP_REVISION_2 as root UID makes no + // sense outside the user namespace the archive is built in. + capability[versionOffset] = vfsCapRevision2 + length = xattrCapsSz2 + } hdr.Xattrs = make(map[string]string) - hdr.Xattrs["security.capability"] = string(capability) + hdr.Xattrs["security.capability"] = string(capability[:length]) } return nil } diff --git a/vendor/github.com/onsi/ginkgo/CHANGELOG.md b/vendor/github.com/onsi/ginkgo/CHANGELOG.md index 50631e4a9..494abdbfb 100644 --- a/vendor/github.com/onsi/ginkgo/CHANGELOG.md +++ b/vendor/github.com/onsi/ginkgo/CHANGELOG.md @@ -1,3 +1,13 @@ +## 1.16.4 + +### Fixes +1.16.4 retracts 1.16.3. There are no code changes. The 1.16.3 tag was associated with the wrong commit and an attempt to change it after-the-fact has proven problematic. 1.16.4 retracts 1.16.3 in Ginkgo's go.mod and creates a new, correctly tagged, release. + +## 1.16.3 + +### Features +- Measure is now deprecated and emits a deprecation warning. + ## 1.16.2 ### Fixes diff --git a/vendor/github.com/onsi/ginkgo/RELEASING.md b/vendor/github.com/onsi/ginkgo/RELEASING.md index 1e298c2da..db3d234c1 100644 --- a/vendor/github.com/onsi/ginkgo/RELEASING.md +++ b/vendor/github.com/onsi/ginkgo/RELEASING.md @@ -8,7 +8,10 @@ A Ginkgo release is a tagged git sha and a GitHub release. To cut a release: - Fixes (fix version) - Maintenance (which in general should not be mentioned in `CHANGELOG.md` as they have no user impact) 1. Update `VERSION` in `config/config.go` -1. Create a commit with the version number as the commit message (e.g. `v1.3.0`) -1. Tag the commit with the version number as the tag name (e.g. `v1.3.0`) -1. Push the commit and tag to GitHub -1. Create a new [GitHub release](https://help.github.com/articles/creating-releases/) with the version number as the tag (e.g. `v1.3.0`). List the key changes in the release notes. +1. Commit, push, and release: + ``` + git commit -m "vM.m.p" + git push + gh release create "vM.m.p" + git fetch --tags origin master + ```
\ No newline at end of file diff --git a/vendor/github.com/onsi/ginkgo/config/config.go b/vendor/github.com/onsi/ginkgo/config/config.go index 88ac2b2c9..5f3f43969 100644 --- a/vendor/github.com/onsi/ginkgo/config/config.go +++ b/vendor/github.com/onsi/ginkgo/config/config.go @@ -20,7 +20,7 @@ import ( "fmt" ) -const VERSION = "1.16.3" +const VERSION = "1.16.4" type GinkgoConfigType struct { RandomSeed int64 diff --git a/vendor/github.com/onsi/ginkgo/go.mod b/vendor/github.com/onsi/ginkgo/go.mod index 664372fb6..86a5a97be 100644 --- a/vendor/github.com/onsi/ginkgo/go.mod +++ b/vendor/github.com/onsi/ginkgo/go.mod @@ -9,3 +9,5 @@ require ( golang.org/x/sys v0.0.0-20210112080510-489259a85091 golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e ) + +retract v1.16.3 // git tag accidentally associated with incorrect git commit diff --git a/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go b/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go index 54597398b..a91a116f8 100644 --- a/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go +++ b/vendor/github.com/opencontainers/selinux/go-selinux/selinux_linux.go @@ -16,9 +16,9 @@ import ( "strings" "sync" + "github.com/bits-and-blooms/bitset" "github.com/opencontainers/selinux/pkg/pwalk" "github.com/pkg/errors" - "github.com/willf/bitset" "golang.org/x/sys/unix" ) diff --git a/vendor/github.com/willf/bitset/go.mod b/vendor/github.com/willf/bitset/go.mod deleted file mode 100644 index 583ecab78..000000000 --- a/vendor/github.com/willf/bitset/go.mod +++ /dev/null @@ -1,3 +0,0 @@ -module github.com/willf/bitset - -go 1.14 diff --git a/vendor/go.etcd.io/bbolt/.gitignore b/vendor/go.etcd.io/bbolt/.gitignore index 3bcd8cbaf..18312f004 100644 --- a/vendor/go.etcd.io/bbolt/.gitignore +++ b/vendor/go.etcd.io/bbolt/.gitignore @@ -3,3 +3,5 @@ *.swp /bin/ cover.out +/.idea +*.iml diff --git a/vendor/go.etcd.io/bbolt/.travis.yml b/vendor/go.etcd.io/bbolt/.travis.yml index 257dfdfee..452601e49 100644 --- a/vendor/go.etcd.io/bbolt/.travis.yml +++ b/vendor/go.etcd.io/bbolt/.travis.yml @@ -4,9 +4,10 @@ go_import_path: go.etcd.io/bbolt sudo: false go: -- 1.12 +- 1.15 before_install: +- go get -v golang.org/x/sys/unix - go get -v honnef.co/go/tools/... - go get -v github.com/kisielk/errcheck diff --git a/vendor/go.etcd.io/bbolt/Makefile b/vendor/go.etcd.io/bbolt/Makefile index 2968aaa61..21ecf48f6 100644 --- a/vendor/go.etcd.io/bbolt/Makefile +++ b/vendor/go.etcd.io/bbolt/Makefile @@ -2,8 +2,6 @@ BRANCH=`git rev-parse --abbrev-ref HEAD` COMMIT=`git rev-parse --short HEAD` GOLDFLAGS="-X main.branch $(BRANCH) -X main.commit $(COMMIT)" -default: build - race: @TEST_FREELIST_TYPE=hashmap go test -v -race -test.run="TestSimulate_(100op|1000op)" @echo "array freelist test" diff --git a/vendor/go.etcd.io/bbolt/README.md b/vendor/go.etcd.io/bbolt/README.md index c9e64b1a6..f1b4a7b2b 100644 --- a/vendor/go.etcd.io/bbolt/README.md +++ b/vendor/go.etcd.io/bbolt/README.md @@ -908,12 +908,14 @@ Below is a list of public, open source projects that use Bolt: * [BoltStore](https://github.com/yosssi/boltstore) - Session store using Bolt. * [Boltdb Boilerplate](https://github.com/bobintornado/boltdb-boilerplate) - Boilerplate wrapper around bolt aiming to make simple calls one-liners. * [BoltDbWeb](https://github.com/evnix/boltdbweb) - A web based GUI for BoltDB files. +* [BoltDB Viewer](https://github.com/zc310/rich_boltdb) - A BoltDB Viewer Can run on Windows、Linux、Android system. * [bleve](http://www.blevesearch.com/) - A pure Go search engine similar to ElasticSearch that uses Bolt as the default storage backend. * [btcwallet](https://github.com/btcsuite/btcwallet) - A bitcoin wallet. * [buckets](https://github.com/joyrexus/buckets) - a bolt wrapper streamlining simple tx and key scans. * [cayley](https://github.com/google/cayley) - Cayley is an open-source graph database using Bolt as optional backend. * [ChainStore](https://github.com/pressly/chainstore) - Simple key-value interface to a variety of storage engines organized as a chain of operations. +* [🌰 Chestnut](https://github.com/jrapoport/chestnut) - Chestnut is encrypted storage for Go. * [Consul](https://github.com/hashicorp/consul) - Consul is service discovery and configuration made easy. Distributed, highly available, and datacenter-aware. * [DVID](https://github.com/janelia-flyem/dvid) - Added Bolt as optional storage engine and testing it against Basho-tuned leveldb. * [dcrwallet](https://github.com/decred/dcrwallet) - A wallet for the Decred cryptocurrency. @@ -938,9 +940,8 @@ Below is a list of public, open source projects that use Bolt: * [MetricBase](https://github.com/msiebuhr/MetricBase) - Single-binary version of Graphite. * [MuLiFS](https://github.com/dankomiocevic/mulifs) - Music Library Filesystem creates a filesystem to organise your music files. * [NATS](https://github.com/nats-io/nats-streaming-server) - NATS Streaming uses bbolt for message and metadata storage. -* [Operation Go: A Routine Mission](http://gocode.io) - An online programming game for Golang using Bolt for user accounts and a leaderboard. -* [photosite/session](https://godoc.org/bitbucket.org/kardianos/photosite/session) - Sessions for a photo viewing site. * [Prometheus Annotation Server](https://github.com/oliver006/prom_annotation_server) - Annotation server for PromDash & Prometheus service monitoring system. +* [Rain](https://github.com/cenkalti/rain) - BitTorrent client and library. * [reef-pi](https://github.com/reef-pi/reef-pi) - reef-pi is an award winning, modular, DIY reef tank controller using easy to learn electronics based on a Raspberry Pi. * [Request Baskets](https://github.com/darklynx/request-baskets) - A web service to collect arbitrary HTTP requests and inspect them via REST API or simple web UI, similar to [RequestBin](http://requestb.in/) service * [Seaweed File System](https://github.com/chrislusf/seaweedfs) - Highly scalable distributed key~file system with O(1) disk read. diff --git a/vendor/go.etcd.io/bbolt/bolt_unix.go b/vendor/go.etcd.io/bbolt/bolt_unix.go index 2938fed58..4e5f65ccc 100644 --- a/vendor/go.etcd.io/bbolt/bolt_unix.go +++ b/vendor/go.etcd.io/bbolt/bolt_unix.go @@ -7,6 +7,8 @@ import ( "syscall" "time" "unsafe" + + "golang.org/x/sys/unix" ) // flock acquires an advisory lock on a file descriptor. @@ -49,13 +51,13 @@ func funlock(db *DB) error { // mmap memory maps a DB's data file. func mmap(db *DB, sz int) error { // Map the data file to memory. - b, err := syscall.Mmap(int(db.file.Fd()), 0, sz, syscall.PROT_READ, syscall.MAP_SHARED|db.MmapFlags) + b, err := unix.Mmap(int(db.file.Fd()), 0, sz, syscall.PROT_READ, syscall.MAP_SHARED|db.MmapFlags) if err != nil { return err } // Advise the kernel that the mmap is accessed randomly. - err = madvise(b, syscall.MADV_RANDOM) + err = unix.Madvise(b, syscall.MADV_RANDOM) if err != nil && err != syscall.ENOSYS { // Ignore not implemented error in kernel because it still works. return fmt.Errorf("madvise: %s", err) @@ -76,18 +78,9 @@ func munmap(db *DB) error { } // Unmap using the original byte slice. - err := syscall.Munmap(db.dataref) + err := unix.Munmap(db.dataref) db.dataref = nil db.data = nil db.datasz = 0 return err } - -// NOTE: This function is copied from stdlib because it is not available on darwin. -func madvise(b []byte, advice int) (err error) { - _, _, e1 := syscall.Syscall(syscall.SYS_MADVISE, uintptr(unsafe.Pointer(&b[0])), uintptr(len(b)), uintptr(advice)) - if e1 != 0 { - err = e1 - } - return -} diff --git a/vendor/go.etcd.io/bbolt/compact.go b/vendor/go.etcd.io/bbolt/compact.go new file mode 100644 index 000000000..e4fe91b04 --- /dev/null +++ b/vendor/go.etcd.io/bbolt/compact.go @@ -0,0 +1,114 @@ +package bbolt + +// Compact will create a copy of the source DB and in the destination DB. This may +// reclaim space that the source database no longer has use for. txMaxSize can be +// used to limit the transactions size of this process and may trigger intermittent +// commits. A value of zero will ignore transaction sizes. +// TODO: merge with: https://github.com/etcd-io/etcd/blob/b7f0f52a16dbf83f18ca1d803f7892d750366a94/mvcc/backend/backend.go#L349 +func Compact(dst, src *DB, txMaxSize int64) error { + // commit regularly, or we'll run out of memory for large datasets if using one transaction. + var size int64 + tx, err := dst.Begin(true) + if err != nil { + return err + } + defer tx.Rollback() + + if err := walk(src, func(keys [][]byte, k, v []byte, seq uint64) error { + // On each key/value, check if we have exceeded tx size. + sz := int64(len(k) + len(v)) + if size+sz > txMaxSize && txMaxSize != 0 { + // Commit previous transaction. + if err := tx.Commit(); err != nil { + return err + } + + // Start new transaction. + tx, err = dst.Begin(true) + if err != nil { + return err + } + size = 0 + } + size += sz + + // Create bucket on the root transaction if this is the first level. + nk := len(keys) + if nk == 0 { + bkt, err := tx.CreateBucket(k) + if err != nil { + return err + } + if err := bkt.SetSequence(seq); err != nil { + return err + } + return nil + } + + // Create buckets on subsequent levels, if necessary. + b := tx.Bucket(keys[0]) + if nk > 1 { + for _, k := range keys[1:] { + b = b.Bucket(k) + } + } + + // Fill the entire page for best compaction. + b.FillPercent = 1.0 + + // If there is no value then this is a bucket call. + if v == nil { + bkt, err := b.CreateBucket(k) + if err != nil { + return err + } + if err := bkt.SetSequence(seq); err != nil { + return err + } + return nil + } + + // Otherwise treat it as a key/value pair. + return b.Put(k, v) + }); err != nil { + return err + } + + return tx.Commit() +} + +// walkFunc is the type of the function called for keys (buckets and "normal" +// values) discovered by Walk. keys is the list of keys to descend to the bucket +// owning the discovered key/value pair k/v. +type walkFunc func(keys [][]byte, k, v []byte, seq uint64) error + +// walk walks recursively the bolt database db, calling walkFn for each key it finds. +func walk(db *DB, walkFn walkFunc) error { + return db.View(func(tx *Tx) error { + return tx.ForEach(func(name []byte, b *Bucket) error { + return walkBucket(b, nil, name, nil, b.Sequence(), walkFn) + }) + }) +} + +func walkBucket(b *Bucket, keypath [][]byte, k, v []byte, seq uint64, fn walkFunc) error { + // Execute callback. + if err := fn(keypath, k, v, seq); err != nil { + return err + } + + // If this is not a bucket then stop. + if v != nil { + return nil + } + + // Iterate over each child key/value. + keypath = append(keypath, k) + return b.ForEach(func(k, v []byte) error { + if v == nil { + bkt := b.Bucket(k) + return walkBucket(bkt, keypath, k, nil, bkt.Sequence(), fn) + } + return walkBucket(b, keypath, k, v, b.Sequence(), fn) + }) +} diff --git a/vendor/go.etcd.io/bbolt/db.go b/vendor/go.etcd.io/bbolt/db.go index 80b0095cc..a798c390a 100644 --- a/vendor/go.etcd.io/bbolt/db.go +++ b/vendor/go.etcd.io/bbolt/db.go @@ -120,6 +120,12 @@ type DB struct { // of truncate() and fsync() when growing the data file. AllocSize int + // Mlock locks database file in memory when set to true. + // It prevents major page faults, however used memory can't be reclaimed. + // + // Supported only on Unix via mlock/munlock syscalls. + Mlock bool + path string openFile func(string, int, os.FileMode) (*os.File, error) file *os.File @@ -188,6 +194,7 @@ func Open(path string, mode os.FileMode, options *Options) (*DB, error) { db.MmapFlags = options.MmapFlags db.NoFreelistSync = options.NoFreelistSync db.FreelistType = options.FreelistType + db.Mlock = options.Mlock // Set default values for later DB operations. db.MaxBatchSize = DefaultMaxBatchSize @@ -337,7 +344,8 @@ func (db *DB) mmap(minsz int) error { } // Ensure the size is at least the minimum size. - var size = int(info.Size()) + fileSize := int(info.Size()) + var size = fileSize if size < minsz { size = minsz } @@ -346,6 +354,13 @@ func (db *DB) mmap(minsz int) error { return err } + if db.Mlock { + // Unlock db memory + if err := db.munlock(fileSize); err != nil { + return err + } + } + // Dereference all mmap references before unmapping. if db.rwtx != nil { db.rwtx.root.dereference() @@ -361,6 +376,13 @@ func (db *DB) mmap(minsz int) error { return err } + if db.Mlock { + // Don't allow swapping of data file + if err := db.mlock(fileSize); err != nil { + return err + } + } + // Save references to the meta pages. db.meta0 = db.page(0).meta() db.meta1 = db.page(1).meta() @@ -422,12 +444,36 @@ func (db *DB) mmapSize(size int) (int, error) { return int(sz), nil } +func (db *DB) munlock(fileSize int) error { + if err := munlock(db, fileSize); err != nil { + return fmt.Errorf("munlock error: " + err.Error()) + } + return nil +} + +func (db *DB) mlock(fileSize int) error { + if err := mlock(db, fileSize); err != nil { + return fmt.Errorf("mlock error: " + err.Error()) + } + return nil +} + +func (db *DB) mrelock(fileSizeFrom, fileSizeTo int) error { + if err := db.munlock(fileSizeFrom); err != nil { + return err + } + if err := db.mlock(fileSizeTo); err != nil { + return err + } + return nil +} + // init creates a new database file and initializes its meta pages. func (db *DB) init() error { // Create two meta pages on a buffer. buf := make([]byte, db.pageSize*4) for i := 0; i < 2; i++ { - p := db.pageInBuffer(buf[:], pgid(i)) + p := db.pageInBuffer(buf, pgid(i)) p.id = pgid(i) p.flags = metaPageFlag @@ -444,13 +490,13 @@ func (db *DB) init() error { } // Write an empty freelist at page 3. - p := db.pageInBuffer(buf[:], pgid(2)) + p := db.pageInBuffer(buf, pgid(2)) p.id = pgid(2) p.flags = freelistPageFlag p.count = 0 // Write an empty leaf page at page 4. - p = db.pageInBuffer(buf[:], pgid(3)) + p = db.pageInBuffer(buf, pgid(3)) p.id = pgid(3) p.flags = leafPageFlag p.count = 0 @@ -462,6 +508,7 @@ func (db *DB) init() error { if err := fdatasync(db); err != nil { return err } + db.filesz = len(buf) return nil } @@ -973,6 +1020,12 @@ func (db *DB) grow(sz int) error { if err := db.file.Sync(); err != nil { return fmt.Errorf("file sync error: %s", err) } + if db.Mlock { + // unlock old file and lock new one + if err := db.mrelock(db.filesz, sz); err != nil { + return fmt.Errorf("mlock/munlock error: %s", err) + } + } } db.filesz = sz @@ -1064,6 +1117,11 @@ type Options struct { // OpenFile is used to open files. It defaults to os.OpenFile. This option // is useful for writing hermetic tests. OpenFile func(string, int, os.FileMode) (*os.File, error) + + // Mlock locks database file in memory when set to true. + // It prevents potential page faults, however + // used memory can't be reclaimed. (UNIX only) + Mlock bool } // DefaultOptions represent the options used if nil options are passed into Open(). diff --git a/vendor/go.etcd.io/bbolt/freelist_hmap.go b/vendor/go.etcd.io/bbolt/freelist_hmap.go index 02ef2be04..dbd67a1e7 100644 --- a/vendor/go.etcd.io/bbolt/freelist_hmap.go +++ b/vendor/go.etcd.io/bbolt/freelist_hmap.go @@ -4,7 +4,7 @@ import "sort" // hashmapFreeCount returns count of free pages(hashmap version) func (f *freelist) hashmapFreeCount() int { - // use the forwardmap to get the total count + // use the forwardMap to get the total count count := 0 for _, size := range f.forwardMap { count += int(size) @@ -41,7 +41,7 @@ func (f *freelist) hashmapAllocate(txid txid, n int) pgid { for pid := range bm { // remove the initial - f.delSpan(pid, uint64(size)) + f.delSpan(pid, size) f.allocs[pid] = txid @@ -51,7 +51,7 @@ func (f *freelist) hashmapAllocate(txid txid, n int) pgid { f.addSpan(pid+pgid(n), remain) for i := pgid(0); i < pgid(n); i++ { - delete(f.cache, pid+pgid(i)) + delete(f.cache, pid+i) } return pid } diff --git a/vendor/go.etcd.io/bbolt/go.mod b/vendor/go.etcd.io/bbolt/go.mod index c2366daef..96355a69b 100644 --- a/vendor/go.etcd.io/bbolt/go.mod +++ b/vendor/go.etcd.io/bbolt/go.mod @@ -2,4 +2,4 @@ module go.etcd.io/bbolt go 1.12 -require golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5 +require golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d diff --git a/vendor/go.etcd.io/bbolt/go.sum b/vendor/go.etcd.io/bbolt/go.sum index 4ad15a488..c13f8f470 100644 --- a/vendor/go.etcd.io/bbolt/go.sum +++ b/vendor/go.etcd.io/bbolt/go.sum @@ -1,2 +1,2 @@ -golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5 h1:LfCXLvNmTYH9kEmVgqbnsWfruoXZIrh4YBgqVHtDvw0= -golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d h1:L/IKR6COd7ubZrs2oTnTi73IhgqJ71c9s80WsQnh0Es= +golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= diff --git a/vendor/go.etcd.io/bbolt/mlock_unix.go b/vendor/go.etcd.io/bbolt/mlock_unix.go new file mode 100644 index 000000000..6a6c7b353 --- /dev/null +++ b/vendor/go.etcd.io/bbolt/mlock_unix.go @@ -0,0 +1,36 @@ +// +build !windows + +package bbolt + +import "golang.org/x/sys/unix" + +// mlock locks memory of db file +func mlock(db *DB, fileSize int) error { + sizeToLock := fileSize + if sizeToLock > db.datasz { + // Can't lock more than mmaped slice + sizeToLock = db.datasz + } + if err := unix.Mlock(db.dataref[:sizeToLock]); err != nil { + return err + } + return nil +} + +//munlock unlocks memory of db file +func munlock(db *DB, fileSize int) error { + if db.dataref == nil { + return nil + } + + sizeToUnlock := fileSize + if sizeToUnlock > db.datasz { + // Can't unlock more than mmaped slice + sizeToUnlock = db.datasz + } + + if err := unix.Munlock(db.dataref[:sizeToUnlock]); err != nil { + return err + } + return nil +} diff --git a/vendor/go.etcd.io/bbolt/mlock_windows.go b/vendor/go.etcd.io/bbolt/mlock_windows.go new file mode 100644 index 000000000..b4a36a493 --- /dev/null +++ b/vendor/go.etcd.io/bbolt/mlock_windows.go @@ -0,0 +1,11 @@ +package bbolt + +// mlock locks memory of db file +func mlock(_ *DB, _ int) error { + panic("mlock is supported only on UNIX systems") +} + +//munlock unlocks memory of db file +func munlock(_ *DB, _ int) error { + panic("munlock is supported only on UNIX systems") +} diff --git a/vendor/go.etcd.io/bbolt/tx.go b/vendor/go.etcd.io/bbolt/tx.go index 4b1a64a8b..869d41200 100644 --- a/vendor/go.etcd.io/bbolt/tx.go +++ b/vendor/go.etcd.io/bbolt/tx.go @@ -188,7 +188,6 @@ func (tx *Tx) Commit() error { } // If strict mode is enabled then perform a consistency check. - // Only the first consistency error is reported in the panic. if tx.db.StrictMode { ch := tx.Check() var errs []string @@ -393,7 +392,7 @@ func (tx *Tx) CopyFile(path string, mode os.FileMode) error { return err } - err = tx.Copy(f) + _, err = tx.WriteTo(f) if err != nil { _ = f.Close() return err diff --git a/vendor/modules.txt b/vendor/modules.txt index 83cd57794..786096f45 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -36,6 +36,8 @@ github.com/VividCortex/ewma github.com/acarl005/stripansi # github.com/beorn7/perks v1.0.1 github.com/beorn7/perks/quantile +# github.com/bits-and-blooms/bitset v1.2.0 +github.com/bits-and-blooms/bitset # github.com/blang/semver v3.5.1+incompatible github.com/blang/semver # github.com/buger/goterm v0.0.0-20181115115552-c206103e1f37 @@ -278,7 +280,7 @@ github.com/docker/distribution/registry/client/auth/challenge github.com/docker/distribution/registry/client/transport github.com/docker/distribution/registry/storage/cache github.com/docker/distribution/registry/storage/cache/memory -# github.com/docker/docker v20.10.6+incompatible +# github.com/docker/docker v20.10.7+incompatible github.com/docker/docker/api github.com/docker/docker/api/types github.com/docker/docker/api/types/blkiodev @@ -446,7 +448,7 @@ github.com/nxadm/tail/ratelimiter github.com/nxadm/tail/util github.com/nxadm/tail/watch github.com/nxadm/tail/winfile -# github.com/onsi/ginkgo v1.16.3 +# github.com/onsi/ginkgo v1.16.4 github.com/onsi/ginkgo github.com/onsi/ginkgo/config github.com/onsi/ginkgo/extensions/table @@ -516,7 +518,7 @@ github.com/opencontainers/runtime-tools/generate github.com/opencontainers/runtime-tools/generate/seccomp github.com/opencontainers/runtime-tools/specerror github.com/opencontainers/runtime-tools/validate -# github.com/opencontainers/selinux v1.8.1 +# github.com/opencontainers/selinux v1.8.2 github.com/opencontainers/selinux/go-selinux github.com/opencontainers/selinux/go-selinux/label github.com/opencontainers/selinux/pkg/pwalk @@ -608,15 +610,13 @@ github.com/vishvananda/netlink github.com/vishvananda/netlink/nl # github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae github.com/vishvananda/netns -# github.com/willf/bitset v1.1.11 -github.com/willf/bitset # github.com/xeipuuv/gojsonpointer v0.0.0-20190809123943-df4f5c81cb3b github.com/xeipuuv/gojsonpointer # github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 github.com/xeipuuv/gojsonreference # github.com/xeipuuv/gojsonschema v1.2.0 github.com/xeipuuv/gojsonschema -# go.etcd.io/bbolt v1.3.5 +# go.etcd.io/bbolt v1.3.6 go.etcd.io/bbolt # go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1 go.mozilla.org/pkcs7 |