diff options
33 files changed, 316 insertions, 77 deletions
@@ -1721,6 +1721,8 @@ kernel [string](https://godoc.org/builtin#string) os [string](https://godoc.org/builtin#string) uptime [string](https://godoc.org/builtin#string) + +eventlogger [string](https://godoc.org/builtin#string) ### <a name="InfoPodmanBinary"></a>type InfoPodmanBinary InfoPodman provides details on the podman binary diff --git a/cmd/podman/cliconfig/config.go b/cmd/podman/cliconfig/config.go index 37d6d3908..d5098ee51 100644 --- a/cmd/podman/cliconfig/config.go +++ b/cmd/podman/cliconfig/config.go @@ -437,6 +437,7 @@ type RestoreValues struct { Import string Name string IgnoreRootfs bool + IgnoreStaticIP bool } type RmValues struct { diff --git a/cmd/podman/restore.go b/cmd/podman/restore.go index 3ae141d41..90d0b2dc4 100644 --- a/cmd/podman/restore.go +++ b/cmd/podman/restore.go @@ -46,6 +46,7 @@ func init() { flags.StringVarP(&restoreCommand.Import, "import", "i", "", "Restore from exported checkpoint archive (tar.gz)") flags.StringVarP(&restoreCommand.Name, "name", "n", "", "Specify new name for container restored from exported checkpoint (only works with --import)") flags.BoolVar(&restoreCommand.IgnoreRootfs, "ignore-rootfs", false, "Do not apply root file-system changes when importing from exported checkpoint") + flags.BoolVar(&restoreCommand.IgnoreStaticIP, "ignore-static-ip", false, "Ignore IP address set via --static-ip") markFlagHiddenForRemoteClient("latest", flags) } diff --git a/cmd/podman/varlink/io.podman.varlink b/cmd/podman/varlink/io.podman.varlink index f5f3250f7..b867dccc1 100644 --- a/cmd/podman/varlink/io.podman.varlink +++ b/cmd/podman/varlink/io.podman.varlink @@ -228,7 +228,8 @@ type InfoHost ( hostname: string, kernel: string, os: string, - uptime: string + uptime: string, + eventlogger: string ) # InfoGraphStatus describes the detailed status of the storage driver diff --git a/completions/bash/podman b/completions/bash/podman index c8f00a3cb..d2eb5b570 100644 --- a/completions/bash/podman +++ b/completions/bash/podman @@ -872,6 +872,7 @@ _podman_container_restore() { --latest --tcp-established --ignore-rootfs + --ignore-static-ip " case "$prev" in -i|--import) diff --git a/docs/podman-container-restore.1.md b/docs/podman-container-restore.1.md index 544a096d8..a49cb7421 100644 --- a/docs/podman-container-restore.1.md +++ b/docs/podman-container-restore.1.md @@ -67,6 +67,15 @@ from a checkpoint tar.gz file it is possible that it also contains all root file changes. With **--ignore-rootfs** it is possible to explicitly disable applying these root file-system changes to the restored container. +**--ignore-static-ip** + +If the container was started with **--ip** the restored container also tries to use that +IP address and restore fails if that IP address is already in use. This can happen, if +a container is restored multiple times from an exported checkpoint with **--name, -n**. + +Using **--ignore-static-ip** tells Podman to ignore the IP address if it was configured +with **--ip** during container creation. + ## EXAMPLE podman container restore mywebserver diff --git a/docs/podman-events.1.md b/docs/podman-events.1.md index 2097bb1f9..ed3faedfd 100644 --- a/docs/podman-events.1.md +++ b/docs/podman-events.1.md @@ -68,7 +68,7 @@ Print usage statement. **--format** -Format the output using the given Go template. An output value of *json* is not supported. +Format the output to JSON Lines or using the given Go template. **--filter**=*filter* @@ -134,6 +134,13 @@ $ sudo podman events --since 5m 2019-03-02 10:44:42.374637304 -0600 CST pod create ca731231718e (image=, name=webapp) ``` +Show podman events in JSON Lines format +``` +events --format json +{"ID":"683b0909d556a9c02fa8cd2b61c3531a965db42158627622d1a67b391964d519","Image":"localhost/myshdemo:latest","Name":"agitated_diffie","Status":"cleanup","Time":"2019-04-27T22:47:00.849932843-04:00","Type":"container"} +{"ID":"a0f8ab051bfd43f9c5141a8a2502139707e4b38d98ac0872e57c5315381e88ad","Image":"docker.io/library/alpine:latest","Name":"friendly_tereshkova","Status":"unmount","Time":"2019-04-28T13:43:38.063017276-04:00","Type":"container"} +``` + ## SEE ALSO podman(1) diff --git a/docs/podman.1.md b/docs/podman.1.md index bfb5a9aec..12b7866ca 100644 --- a/docs/podman.1.md +++ b/docs/podman.1.md @@ -223,7 +223,7 @@ When Podman runs in rootless mode, the file `$HOME/.config/containers/mounts.con ## Rootless mode Podman can also be used as non-root user. When podman runs in rootless mode, a user namespace is automatically created for the user, defined in /etc/subuid and /etc/subgid. -Containers created by a non-root user are not visible to other users and are not seen or managed by podman running as root. +Containers created by a non-root user are not visible to other users and are not seen or managed by Podman running as root. It is required to have multiple uids/gids set for an user. Be sure the user is present in the files `/etc/subuid` and `/etc/subgid`. @@ -244,6 +244,14 @@ Images are pulled under `XDG_DATA_HOME` when specified, otherwise in the home di Currently the slirp4netns package is required to be installed to create a network device, otherwise rootless containers need to run in the network namespace of the host. +### **NOTE:** Unsupported file systems in rootless mode + +The Overlay file system (OverlayFS) is not supported in rootless mode. The fuse-overlayfs package is a tool that provides the functionality of OverlayFS in user namespace that allows mounting file systems in rootless environments. It is recommended to install the fuse-overlayfs package and to enable it by adding `mount_program = "/usr/bin/fuse-overlayfs"` under `[storage.options]` in the `~/.config/containers/storage.conf` file. + +The Network File System (NFS) and other distributed file systems (for example: Lustre, Spectrum Scale, the General Parallel File System (GPFS)) are not supported when running in rootless mode as these file systems do not understand user namespace. However, rootless Podman can make use of an NFS Homedir by modifying the `~/.config/containers/storage.conf` to have the `graphroot` option point to a directory stored on local (Non NFS) storage. + +For more information, please refer to the [Podman Troubleshooting Page](https://github.com/containers/libpod/blob/master/troubleshooting.md). + ## SEE ALSO `containers-mounts.conf(5)`, `containers-registries.conf(5)`, `containers-storage.conf(5)`, `buildah(1)`, `libpod.conf(5)`, `oci-hooks(5)`, `policy.json(5)`, `subuid(5)`, `subgid(5)`, `slirp4netns(1)` diff --git a/libpod/container.go b/libpod/container.go index 2d96b1120..9c01d2adf 100644 --- a/libpod/container.go +++ b/libpod/container.go @@ -377,6 +377,9 @@ type ContainerConfig struct { RestartRetries uint `json:"restart_retries,omitempty"` // TODO log options for log drivers + // PostConfigureNetNS needed when a user namespace is created by an OCI runtime + // if the network namespace is created before the user namespace it will be + // owned by the wrong user namespace. PostConfigureNetNS bool `json:"postConfigureNetNS"` // OCIRuntime used to create the container diff --git a/libpod/container_api.go b/libpod/container_api.go index ef9c3f006..abcfcb271 100644 --- a/libpod/container_api.go +++ b/libpod/container_api.go @@ -773,6 +773,11 @@ type ContainerCheckpointOptions struct { // IgnoreRootfs tells the API to not export changes to // the container's root file-system (or to not import) IgnoreRootfs bool + // IgnoreStaticIP tells the API to ignore the IP set + // during 'podman run' with '--ip'. This is especially + // important to be able to restore a container multiple + // times with '--import --name'. + IgnoreStaticIP bool } // Checkpoint checkpoints a container diff --git a/libpod/container_internal.go b/libpod/container_internal.go index aba9c5b93..313f67963 100644 --- a/libpod/container_internal.go +++ b/libpod/container_internal.go @@ -1152,9 +1152,27 @@ func (c *Container) restartWithTimeout(ctx context.Context, timeout uint) (err e c.newContainerEvent(events.Restart) if c.state.State == define.ContainerStateRunning { + conmonPID := c.state.ConmonPID if err := c.stop(timeout); err != nil { return err } + // Old versions of conmon have a bug where they create the exit file before + // closing open file descriptors causing a race condition when restarting + // containers with open ports since we cannot bind the ports as they're not + // yet closed by conmon. + // + // Killing the old conmon PID is ~okay since it forces the FDs of old conmons + // to be closed, while it's a NOP for newer versions which should have + // exited already. + if conmonPID != 0 { + // Ignore errors from FindProcess() as conmon could already have exited. + p, err := os.FindProcess(conmonPID) + if p != nil && err == nil { + if err = p.Kill(); err != nil { + logrus.Debugf("error killing conmon process: %v", err) + } + } + } } defer func() { if err != nil { diff --git a/libpod/container_internal_linux.go b/libpod/container_internal_linux.go index 6dbd53fbf..d06c19b8c 100644 --- a/libpod/container_internal_linux.go +++ b/libpod/container_internal_linux.go @@ -63,12 +63,12 @@ func (c *Container) unmountSHM(mount string) error { // namespaces func (c *Container) prepare() (err error) { var ( - wg sync.WaitGroup - netNS ns.NetNS - networkStatus []*cnitypes.Result - createNetNSErr, mountStorageErr error - mountPoint string - tmpStateLock sync.Mutex + wg sync.WaitGroup + netNS ns.NetNS + networkStatus []*cnitypes.Result + createNetNSErr, mountStorageErr, rootlessSetupErr error + mountPoint string + tmpStateLock sync.Mutex ) wg.Add(2) @@ -87,6 +87,11 @@ func (c *Container) prepare() (err error) { c.state.NetNS = netNS c.state.NetworkStatus = networkStatus } + + // Setup rootless networking, requires c.state.NetNS to be set + if rootless.IsRootless() { + rootlessSetupErr = c.runtime.setupRootlessNetNS(c) + } } }() // Mount storage if not mounted @@ -132,6 +137,10 @@ func (c *Container) prepare() (err error) { return mountStorageErr } + if rootlessSetupErr != nil { + return rootlessSetupErr + } + // Save the container return c.save() } @@ -743,6 +752,14 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti return err } + // If a container is restored multiple times from an exported checkpoint with + // the help of '--import --name', the restore will fail if during 'podman run' + // a static container IP was set with '--ip'. The user can tell the restore + // process to ignore the static IP with '--ignore-static-ip' + if options.IgnoreStaticIP { + c.config.StaticIP = nil + } + // Read network configuration from checkpoint // Currently only one interface with one IP is supported. networkStatusFile, err := os.Open(filepath.Join(c.bundlePath(), "network.status")) @@ -752,7 +769,7 @@ func (c *Container) restore(ctx context.Context, options ContainerCheckpointOpti // TODO: This implicit restoring with or without IP depending on an // unrelated restore parameter (--name) does not seem like the // best solution. - if err == nil && options.Name == "" { + if err == nil && options.Name == "" && !options.IgnoreStaticIP { // The file with the network.status does exist. Let's restore the // container with the same IP address as during checkpointing. defer networkStatusFile.Close() diff --git a/libpod/events/config.go b/libpod/events/config.go index 96172d47b..453c64f8c 100644 --- a/libpod/events/config.go +++ b/libpod/events/config.go @@ -22,13 +22,13 @@ const ( type Event struct { // ContainerExitCode is for storing the exit code of a container which can // be used for "internal" event notification - ContainerExitCode int + ContainerExitCode int `json:",omitempty"` // ID can be for the container, image, volume, etc - ID string + ID string `json:",omitempty"` // Image used where applicable - Image string + Image string `json:",omitempty"` // Name where applicable - Name string + Name string `json:",omitempty"` // Status describes the event that occurred Status Status // Time the event occurred @@ -53,6 +53,8 @@ type Eventer interface { Write(event Event) error // Read an event from the backend Read(options ReadOptions) error + // String returns the type of event logger + String() string } // ReadOptions describe the attributes needed to read event logs diff --git a/libpod/events/journal_linux.go b/libpod/events/journal_linux.go index d5bce4334..ae96e3b3b 100644 --- a/libpod/events/journal_linux.go +++ b/libpod/events/journal_linux.go @@ -146,3 +146,8 @@ func newEventFromJournalEntry(entry *sdjournal.JournalEntry) (*Event, error) { / } return &newEvent, nil } + +// String returns a string representation of the logger +func (e EventJournalD) String() string { + return Journald.String() +} diff --git a/libpod/events/logfile.go b/libpod/events/logfile.go index 30d72b9fc..4b65b0ad0 100644 --- a/libpod/events/logfile.go +++ b/libpod/events/logfile.go @@ -71,3 +71,8 @@ func (e EventLogFile) Read(options ReadOptions) error { close(options.EventChannel) return nil } + +// String returns a string representation of the logger +func (e EventLogFile) String() string { + return LogFile.String() +} diff --git a/libpod/events/nullout.go b/libpod/events/nullout.go index b11afcf80..f3b36e609 100644 --- a/libpod/events/nullout.go +++ b/libpod/events/nullout.go @@ -17,6 +17,10 @@ func (e EventToNull) Read(options ReadOptions) error { // NewNullEventer returns a new null eventer. You should only do this for // the purposes on internal libpod testing. func NewNullEventer() Eventer { - e := EventToNull{} - return e + return EventToNull{} +} + +// String returns a string representation of the logger +func (e EventToNull) String() string { + return "none" } diff --git a/libpod/info.go b/libpod/info.go index 4a89fa648..9ab6f7e52 100644 --- a/libpod/info.go +++ b/libpod/info.go @@ -102,7 +102,7 @@ func (r *Runtime) hostInfo() (map[string]interface{}, error) { return nil, errors.Wrapf(err, "error getting hostname") } info["hostname"] = host - + info["eventlogger"] = r.eventer.String() return info, nil } diff --git a/libpod/networking_linux.go b/libpod/networking_linux.go index bef3f7739..73e839bda 100644 --- a/libpod/networking_linux.go +++ b/libpod/networking_linux.go @@ -103,9 +103,6 @@ func (r *Runtime) configureNetNS(ctr *Container, ctrNS ns.NetNS) ([]*cnitypes.Re // Create and configure a new network namespace for a container func (r *Runtime) createNetNS(ctr *Container) (n ns.NetNS, q []*cnitypes.Result, err error) { - if rootless.IsRootless() { - return nil, nil, errors.New("cannot configure a new network namespace in rootless mode, only --network=slirp4netns is supported") - } ctrNS, err := netns.NewNS() if err != nil { return nil, nil, errors.Wrapf(err, "error creating network namespace for container %s", ctr.ID()) @@ -123,7 +120,10 @@ func (r *Runtime) createNetNS(ctr *Container) (n ns.NetNS, q []*cnitypes.Result, logrus.Debugf("Made network namespace at %s for container %s", ctrNS.Path(), ctr.ID()) - networkStatus, err := r.configureNetNS(ctr, ctrNS) + networkStatus := []*cnitypes.Result{} + if !rootless.IsRootless() { + networkStatus, err = r.configureNetNS(ctr, ctrNS) + } return ctrNS, networkStatus, err } @@ -151,9 +151,6 @@ func checkSlirpFlags(path string) (bool, bool, error) { // Configure the network namespace for a rootless container func (r *Runtime) setupRootlessNetNS(ctr *Container) (err error) { - defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncR) - defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncW) - path := r.config.NetworkCmdPath if path == "" { @@ -177,7 +174,7 @@ func (r *Runtime) setupRootlessNetNS(ctr *Container) (err error) { cmdArgs := []string{} if havePortMapping { - cmdArgs = append(cmdArgs, "--api-socket", apiSocket, fmt.Sprintf("%d", ctr.state.PID)) + cmdArgs = append(cmdArgs, "--api-socket", apiSocket) } dhp, mtu, err := checkSlirpFlags(path) if err != nil { @@ -189,13 +186,27 @@ func (r *Runtime) setupRootlessNetNS(ctr *Container) (err error) { if mtu { cmdArgs = append(cmdArgs, "--mtu", "65520") } - cmdArgs = append(cmdArgs, "-c", "-e", "3", "-r", "4", fmt.Sprintf("%d", ctr.state.PID), "tap0") - cmd := exec.Command(path, cmdArgs...) + cmdArgs = append(cmdArgs, "-c", "-e", "3", "-r", "4") + if !ctr.config.PostConfigureNetNS { + ctr.rootlessSlirpSyncR, ctr.rootlessSlirpSyncW, err = os.Pipe() + if err != nil { + return errors.Wrapf(err, "failed to create rootless network sync pipe") + } + cmdArgs = append(cmdArgs, "--netns-type=path", ctr.state.NetNS.Path(), "tap0") + } else { + defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncR) + defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncW) + cmdArgs = append(cmdArgs, fmt.Sprintf("%d", ctr.state.PID), "tap0") + } + cmd := exec.Command(path, cmdArgs...) + logrus.Debugf("slirp4netns command: %s", strings.Join(cmd.Args, " ")) cmd.SysProcAttr = &syscall.SysProcAttr{ Setpgid: true, } + + // Leak one end of the pipe in slirp4netns, the other will be sent to conmon cmd.ExtraFiles = append(cmd.ExtraFiles, ctr.rootlessSlirpSyncR, syncW) if err := cmd.Start(); err != nil { @@ -410,22 +421,25 @@ func (r *Runtime) teardownNetNS(ctr *Container) error { } } - logrus.Debugf("Tearing down network namespace at %s for container %s", ctr.state.NetNS.Path(), ctr.ID()) + logrus.Debugf("Tearing down network namespace for container %s", ctr.ID()) - var requestedIP net.IP - if ctr.requestedIP != nil { - requestedIP = ctr.requestedIP - // cancel request for a specific IP in case the container is reused later - ctr.requestedIP = nil - } else { - requestedIP = ctr.config.StaticIP - } + // rootless containers do not use the CNI plugin + if !rootless.IsRootless() { + var requestedIP net.IP + if ctr.requestedIP != nil { + requestedIP = ctr.requestedIP + // cancel request for a specific IP in case the container is reused later + ctr.requestedIP = nil + } else { + requestedIP = ctr.config.StaticIP + } - podNetwork := r.getPodNetwork(ctr.ID(), ctr.Name(), ctr.state.NetNS.Path(), ctr.config.Networks, ctr.config.PortMappings, requestedIP) + podNetwork := r.getPodNetwork(ctr.ID(), ctr.Name(), ctr.state.NetNS.Path(), ctr.config.Networks, ctr.config.PortMappings, requestedIP) - // The network may have already been torn down, so don't fail here, just log - if err := r.netPlugin.TearDownPod(podNetwork); err != nil { - return errors.Wrapf(err, "error tearing down CNI namespace configuration for container %s", ctr.ID()) + // The network may have already been torn down, so don't fail here, just log + if err := r.netPlugin.TearDownPod(podNetwork); err != nil { + return errors.Wrapf(err, "error tearing down CNI namespace configuration for container %s", ctr.ID()) + } } // First unmount the namespace diff --git a/libpod/oci_internal_linux.go b/libpod/oci_internal_linux.go index 52cebefab..28e4b5b82 100644 --- a/libpod/oci_internal_linux.go +++ b/libpod/oci_internal_linux.go @@ -130,10 +130,16 @@ func (r *OCIRuntime) createOCIContainer(ctr *Container, restoreOptions *Containe } if ctr.config.NetMode.IsSlirp4netns() { - ctr.rootlessSlirpSyncR, ctr.rootlessSlirpSyncW, err = os.Pipe() - if err != nil { - return errors.Wrapf(err, "failed to create rootless network sync pipe") + if ctr.config.PostConfigureNetNS { + ctr.rootlessSlirpSyncR, ctr.rootlessSlirpSyncW, err = os.Pipe() + if err != nil { + return errors.Wrapf(err, "failed to create rootless network sync pipe") + } + } else { + defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncR) + defer errorhandling.CloseQuiet(ctr.rootlessSlirpSyncW) } + // Leak one end in conmon, the other one will be leaked into slirp4netns cmd.ExtraFiles = append(cmd.ExtraFiles, ctr.rootlessSlirpSyncW) } diff --git a/libpod/oci_linux.go b/libpod/oci_linux.go index 45365203e..4d8e36516 100644 --- a/libpod/oci_linux.go +++ b/libpod/oci_linux.go @@ -405,6 +405,14 @@ func (r *OCIRuntime) stopContainer(ctr *Container, timeout uint) error { stopSignal = uint(syscall.SIGTERM) } + defer func() { + // cleanup container networking + err = ctr.cleanupNetwork() + if err != nil { + logrus.Errorf("Error cleaning up container: %s network: %v", ctr.ID(), err) + } + }() + if timeout > 0 { if err := r.killContainer(ctr, stopSignal); err != nil { // Is the container gone? diff --git a/libpod/runtime.go b/libpod/runtime.go index ffdbc32f1..b66b68cfd 100644 --- a/libpod/runtime.go +++ b/libpod/runtime.go @@ -52,7 +52,7 @@ const ( var ( // InstallPrefix is the prefix where podman will be installed. // It can be overridden at build time. - installPrefix = "/usr/local" + installPrefix = "/usr" // EtcDir is the sysconfdir where podman should look for system config files. // It can be overridden at build time. etcDir = "/etc" diff --git a/libpod/runtime_pod_infra_linux.go b/libpod/runtime_pod_infra_linux.go index da35b7f93..20b841296 100644 --- a/libpod/runtime_pod_infra_linux.go +++ b/libpod/runtime_pod_infra_linux.go @@ -95,7 +95,9 @@ func (r *Runtime) makeInfraContainer(ctx context.Context, p *Pod, imgName, imgID if isRootless { netmode = "slirp4netns" } - options = append(options, WithNetNS(p.config.InfraContainer.PortBindings, isRootless, netmode, networks)) + // PostConfigureNetNS should not be set since user namespace sharing is not implemented + // and rootless networking no longer supports post configuration setup + options = append(options, WithNetNS(p.config.InfraContainer.PortBindings, false, netmode, networks)) return r.newContainer(ctx, g.Config, options...) } diff --git a/pkg/adapter/client.go b/pkg/adapter/client.go index 694d9f961..da6ff5fd0 100644 --- a/pkg/adapter/client.go +++ b/pkg/adapter/client.go @@ -16,7 +16,7 @@ var remoteEndpoint *Endpoint func (r RemoteRuntime) RemoteEndpoint() (remoteEndpoint *Endpoint, err error) { remoteConfigConnections, err := remoteclientconfig.ReadRemoteConfig(r.config) - if errors.Cause(err) != remoteclientconfig.ErrNoConfigationFile { + if err != nil && errors.Cause(err) != remoteclientconfig.ErrNoConfigationFile { return nil, err } // If the user defines an env variable for podman_varlink_bridge @@ -68,7 +68,6 @@ func (r RemoteRuntime) Connect() (*varlink.Connection, error) { if err != nil { return nil, err } - switch ep.Type { case DirectConnection: return varlink.NewConnection(ep.Connection) diff --git a/pkg/adapter/containers.go b/pkg/adapter/containers.go index 155454e21..b712bd9aa 100644 --- a/pkg/adapter/containers.go +++ b/pkg/adapter/containers.go @@ -539,6 +539,7 @@ func (r *LocalRuntime) Restore(ctx context.Context, c *cliconfig.RestoreValues) TargetFile: c.Import, Name: c.Name, IgnoreRootfs: c.IgnoreRootfs, + IgnoreStaticIP: c.IgnoreStaticIP, } filterFuncs = append(filterFuncs, func(c *libpod.Container) bool { diff --git a/pkg/adapter/runtime.go b/pkg/adapter/runtime.go index ee6913cc0..4a3b41297 100644 --- a/pkg/adapter/runtime.go +++ b/pkg/adapter/runtime.go @@ -5,22 +5,22 @@ package adapter import ( "bufio" "context" - "github.com/containers/libpod/libpod/define" "io" "io/ioutil" "os" "text/template" - "github.com/containers/libpod/cmd/podman/shared" - "github.com/containers/buildah" "github.com/containers/buildah/imagebuildah" + "github.com/containers/buildah/pkg/formats" "github.com/containers/buildah/pkg/parse" "github.com/containers/image/docker/reference" "github.com/containers/image/types" "github.com/containers/libpod/cmd/podman/cliconfig" "github.com/containers/libpod/cmd/podman/libpodruntime" + "github.com/containers/libpod/cmd/podman/shared" "github.com/containers/libpod/libpod" + "github.com/containers/libpod/libpod/define" "github.com/containers/libpod/libpod/events" "github.com/containers/libpod/libpod/image" "github.com/containers/libpod/pkg/rootless" @@ -351,9 +351,13 @@ func (r *LocalRuntime) Events(c *cliconfig.EventValues) error { fromStart bool eventsError error ) - tmpl, err := template.New("events").Parse(c.Format) - if err != nil { - return err + var tmpl *template.Template + if c.Format != formats.JSONString { + template, err := template.New("events").Parse(c.Format) + if err != nil { + return err + } + tmpl = template } if len(c.Since) > 0 || len(c.Until) > 0 { fromStart = true @@ -369,7 +373,15 @@ func (r *LocalRuntime) Events(c *cliconfig.EventValues) error { } w := bufio.NewWriter(os.Stdout) for event := range eventChannel { - if len(c.Format) > 0 { + if c.Format == formats.JSONString { + jsonStr, err := event.ToJSONString() + if err != nil { + return errors.Wrapf(err, "unable to format json") + } + if _, err := w.Write([]byte(jsonStr)); err != nil { + return err + } + } else if len(c.Format) > 0 { if err := tmpl.Execute(w, event); err != nil { return err } diff --git a/pkg/adapter/runtime_remote.go b/pkg/adapter/runtime_remote.go index 9fae39df0..828838bde 100644 --- a/pkg/adapter/runtime_remote.go +++ b/pkg/adapter/runtime_remote.go @@ -14,9 +14,8 @@ import ( "text/template" "time" - v1 "k8s.io/api/core/v1" - "github.com/containers/buildah/imagebuildah" + "github.com/containers/buildah/pkg/formats" "github.com/containers/image/docker/reference" "github.com/containers/image/types" "github.com/containers/libpod/cmd/podman/cliconfig" @@ -32,6 +31,7 @@ import ( "github.com/pkg/errors" "github.com/sirupsen/logrus" "github.com/varlink/go/varlink" + v1 "k8s.io/api/core/v1" ) // ImageRuntime is wrapper for image runtime @@ -820,9 +820,13 @@ func (r *LocalRuntime) Events(c *cliconfig.EventValues) error { } w := bufio.NewWriter(os.Stdout) - tmpl, err := template.New("events").Parse(c.Format) - if err != nil { - return err + var tmpl *template.Template + if c.Format != formats.JSONString { + template, err := template.New("events").Parse(c.Format) + if err != nil { + return err + } + tmpl = template } for { @@ -856,7 +860,15 @@ func (r *LocalRuntime) Events(c *cliconfig.EventValues) error { Time: eTime, Type: eType, } - if len(c.Format) > 0 { + if c.Format == formats.JSONString { + jsonStr, err := event.ToJSONString() + if err != nil { + return errors.Wrapf(err, "unable to format json") + } + if _, err := w.Write([]byte(jsonStr)); err != nil { + return err + } + } else if len(c.Format) > 0 { if err := tmpl.Execute(w, event); err != nil { return err } diff --git a/pkg/netns/netns_linux.go b/pkg/netns/netns_linux.go index 1d6fb873c..e8388055a 100644 --- a/pkg/netns/netns_linux.go +++ b/pkg/netns/netns_linux.go @@ -23,23 +23,42 @@ import ( "fmt" "os" "path" + "path/filepath" "runtime" "strings" "sync" "github.com/containernetworking/plugins/pkg/ns" + "github.com/containers/libpod/pkg/rootless" + "github.com/containers/libpod/pkg/util" "github.com/sirupsen/logrus" "golang.org/x/sys/unix" ) -const nsRunDir = "/var/run/netns" +// get NSRunDir returns the dir of where to create the netNS. When running +// rootless, it needs to be at a location writable by user. +func getNSRunDir() (string, error) { + if rootless.IsRootless() { + rootlessDir, err := util.GetRootlessRuntimeDir() + if err != nil { + return "", err + } + return filepath.Join(rootlessDir, "netns"), nil + } + return "/var/run/netns", nil +} // NewNS creates a new persistent (bind-mounted) network namespace and returns // an object representing that namespace, without switching to it. func NewNS() (ns.NetNS, error) { + nsRunDir, err := getNSRunDir() + if err != nil { + return nil, err + } + b := make([]byte, 16) - _, err := rand.Reader.Read(b) + _, err = rand.Reader.Read(b) if err != nil { return nil, fmt.Errorf("failed to generate random netns name: %v", err) } @@ -127,7 +146,7 @@ func NewNS() (ns.NetNS, error) { // Put this thread back to the orig ns, since it might get reused (pre go1.10) defer func() { if err := origNS.Set(); err != nil { - logrus.Errorf("unable to set namespace: %q", err) + logrus.Warnf("unable to set namespace: %q", err) } }() @@ -150,6 +169,11 @@ func NewNS() (ns.NetNS, error) { // UnmountNS unmounts the NS held by the netns object func UnmountNS(ns ns.NetNS) error { + nsRunDir, err := getNSRunDir() + if err != nil { + return err + } + nsPath := ns.Path() // Only unmount if it's been bind-mounted (don't touch namespaces in /proc...) if strings.HasPrefix(nsPath, nsRunDir) { diff --git a/pkg/spec/createconfig.go b/pkg/spec/createconfig.go index f21ae2831..289634a0d 100644 --- a/pkg/spec/createconfig.go +++ b/pkg/spec/createconfig.go @@ -270,7 +270,7 @@ func (c *CreateConfig) getContainerCreateOptions(runtime *libpod.Runtime, pod *l options = append(options, libpod.WithNetNSFrom(connectedCtr)) } else if !c.NetMode.IsHost() && !c.NetMode.IsNone() { hasUserns := c.UsernsMode.IsContainer() || c.UsernsMode.IsNS() || len(c.IDMappings.UIDMap) > 0 || len(c.IDMappings.GIDMap) > 0 - postConfigureNetNS := c.NetMode.IsSlirp4netns() || (hasUserns && !c.UsernsMode.IsHost()) + postConfigureNetNS := hasUserns && !c.UsernsMode.IsHost() options = append(options, libpod.WithNetNS(portBindings, postConfigureNetNS, string(c.NetMode), networks)) } diff --git a/pkg/util/utils_supported.go b/pkg/util/utils_supported.go index af55689a6..6449c6f85 100644 --- a/pkg/util/utils_supported.go +++ b/pkg/util/utils_supported.go @@ -26,7 +26,7 @@ func GetRootlessRuntimeDir() (string, error) { if runtimeDir == "" { tmpDir := filepath.Join("/run", "user", uid) if err := os.MkdirAll(tmpDir, 0700); err != nil { - logrus.Errorf("unable to make temp dir %s", tmpDir) + logrus.Debugf("unable to make temp dir %s", tmpDir) } st, err := os.Stat(tmpDir) if err == nil && int(st.Sys().(*syscall.Stat_t).Uid) == os.Geteuid() && st.Mode().Perm() == 0700 { @@ -36,7 +36,7 @@ func GetRootlessRuntimeDir() (string, error) { if runtimeDir == "" { tmpDir := filepath.Join(os.TempDir(), fmt.Sprintf("run-%s", uid)) if err := os.MkdirAll(tmpDir, 0700); err != nil { - logrus.Errorf("unable to make temp dir %s", tmpDir) + logrus.Debugf("unable to make temp dir %s", tmpDir) } st, err := os.Stat(tmpDir) if err == nil && int(st.Sys().(*syscall.Stat_t).Uid) == os.Geteuid() && st.Mode().Perm() == 0700 { diff --git a/pkg/varlinkapi/system.go b/pkg/varlinkapi/system.go index 9b5b3a5b1..2de785b79 100644 --- a/pkg/varlinkapi/system.go +++ b/pkg/varlinkapi/system.go @@ -61,6 +61,7 @@ func (i *LibpodAPI) GetInfo(call iopodman.VarlinkCall) error { Kernel: host["kernel"].(string), Os: host["os"].(string), Uptime: host["uptime"].(string), + Eventlogger: host["eventlogger"].(string), } podmanInfo.Host = infoHost store := info[1].Data diff --git a/test/e2e/checkpoint_test.go b/test/e2e/checkpoint_test.go index 0261acbd9..d37d7c7cc 100644 --- a/test/e2e/checkpoint_test.go +++ b/test/e2e/checkpoint_test.go @@ -364,8 +364,8 @@ var _ = Describe("Podman checkpoint", func() { // This test does the same steps which are necessary for migrating // a container from one host to another It("podman checkpoint container with export (migration)", func() { - // CRIU does not work with seccomp correctly on RHEL7 - session := podmanTest.Podman([]string{"run", "-it", "--security-opt", "seccomp=unconfined", "-d", ALPINE, "top"}) + localRunString := getRunString([]string{"--rm", ALPINE, "top"}) + session := podmanTest.Podman(localRunString) session.WaitWithDefaultTimeout() Expect(session.ExitCode()).To(Equal(0)) Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) @@ -377,13 +377,7 @@ var _ = Describe("Podman checkpoint", func() { Expect(result.ExitCode()).To(Equal(0)) Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) - Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Exited")) - - // Remove all containers to simulate migration - result = podmanTest.Podman([]string{"rm", "-fa"}) - result.WaitWithDefaultTimeout() - Expect(result.ExitCode()).To(Equal(0)) - Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) result = podmanTest.Podman([]string{"container", "restore", "-i", fileName}) result.WaitWithDefaultTimeout() @@ -392,8 +386,12 @@ var _ = Describe("Podman checkpoint", func() { Expect(podmanTest.NumberOfContainersRunning()).To(Equal(1)) Expect(podmanTest.GetContainerStatus()).To(ContainSubstring("Up")) - // Restore container a second time with different name - result = podmanTest.Podman([]string{"container", "restore", "-i", fileName, "-n", "restore_again"}) + // Restore container a second time with different name. + // Using '--ignore-static-ip' as for parallel test runs + // each containers gets a random IP address via '--ip'. + // '--ignore-static-ip' tells the restore to use the next + // available IP address. + result = podmanTest.Podman([]string{"container", "restore", "-i", fileName, "-n", "restore_again", "--ignore-static-ip"}) result.WaitWithDefaultTimeout() Expect(result.ExitCode()).To(Equal(0)) @@ -404,6 +402,7 @@ var _ = Describe("Podman checkpoint", func() { result.WaitWithDefaultTimeout() Expect(result.ExitCode()).To(Equal(0)) Expect(podmanTest.NumberOfContainersRunning()).To(Equal(0)) + Expect(podmanTest.NumberOfContainers()).To(Equal(0)) // Remove exported checkpoint os.Remove(fileName) diff --git a/test/e2e/events_test.go b/test/e2e/events_test.go index c5eedda3c..0636af74c 100644 --- a/test/e2e/events_test.go +++ b/test/e2e/events_test.go @@ -1,6 +1,7 @@ package integration import ( + "encoding/json" "fmt" "os" "strings" @@ -116,4 +117,25 @@ var _ = Describe("Podman events", func() { Expect(result.ExitCode()).To(BeZero()) }) + It("podman events format", func() { + info := GetHostDistributionInfo() + if info.Distribution != "fedora" { + Skip("need to verify images have correct packages for journald") + } + _, ec, _ := podmanTest.RunLsContainer("") + Expect(ec).To(Equal(0)) + test := podmanTest.Podman([]string{"events", "--stream=false", "--format", "json"}) + test.WaitWithDefaultTimeout() + fmt.Println(test.OutputToStringArray()) + jsonArr := test.OutputToStringArray() + Expect(len(jsonArr)).To(Not(BeZero())) + eventsMap := make(map[string]string) + err := json.Unmarshal([]byte(jsonArr[0]), &eventsMap) + if err != nil { + os.Exit(1) + } + _, exist := eventsMap["Status"] + Expect(exist).To(BeTrue()) + Expect(test.ExitCode()).To(BeZero()) + }) }) diff --git a/troubleshooting.md b/troubleshooting.md index 64aec475e..798d9cec4 100644 --- a/troubleshooting.md +++ b/troubleshooting.md @@ -315,3 +315,53 @@ Not doing this will cause Podman in the container to detect that temporary files This can cause Podman to reset container states and lose track of running containers. For running containers on the host from inside a container, we also recommend the [Podman remote client](remote_client.md), which only requires a single socket to be mounted into the container. + +### 13) Rootless 'podman build' fails EPERM on NFS: + +NFS enforces file creation on different UIDs on the server side and does not understand user namespace, which rootless Podman requires. +When a container root process like YUM attempts to create a file owned by a different UID, NFS Server denies the creation. +NFS is also a problem for the file locks when the storage is on it. Other distributed file systems (for example: Lustre, Spectrum Scale, the General Parallel File System (GPFS)) are also not supported when running in rootless mode as these file systems do not understand user namespace. + +#### Symptom +```console +$ podman build . +ERRO[0014] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: open /root/.bash_logout: permission denied +error creating build container: Error committing the finished image: error adding layer with blob "sha256:a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17": ApplyLayer exit status 1 stdout: stderr: open /root/.bash_logout: permission denied +``` + +#### Solution +Choose one of the following: + * Setup containers/storage in a different directory, not on an NFS share. + * Create a directory on a local file system. + * Edit `~/.config/containers/libpod.conf` and point the `volume_path` option to that local directory. + * Otherwise just run podman as root, via `sudo podman` + +### 14) Rootless 'podman build' fails when using OverlayFS: + +The Overlay file system (OverlayFS) requires the ability to call the `mknod` command when creating whiteout files +when extracting an image. However, a rootless user does not have the privileges to use `mknod` in this capacity. + +#### Symptom +```console +podman build --storage-driver overlay . +STEP 1: FROM docker.io/ubuntu:xenial +Getting image source signatures +Copying blob edf72af6d627 done +Copying blob 3e4f86211d23 done +Copying blob 8d3eac894db4 done +Copying blob f7277927d38a done +Copying config 5e13f8dd4c done +Writing manifest to image destination +Storing signatures +Error: error creating build container: Error committing the finished image: error adding layer with blob "sha256:8d3eac894db4dc4154377ad28643dfe6625ff0e54bcfa63e0d04921f1a8ef7f8": Error processing tar file(exit status 1): operation not permitted +$ podman build . +ERRO[0014] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: open /root/.bash_logout: permission denied +error creating build container: Error committing the finished image: error adding layer with blob "sha256:a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17": ApplyLayer exit status 1 stdout: stderr: open /root/.bash_logout: permission denied +``` + +#### Solution +Choose one of the following: + * Complete the build operation as a privileged user. + * Install and configure fuse-overlayfs. + * Install the fuse-overlayfs package for your Linux Distribution. + * Add `mount_program = "/usr/bin/fuse-overlayfs` under `[storage.options]` in your `~/.config/containers/storage.conf` file. |