summaryrefslogtreecommitdiff
path: root/libpod/container_internal.go
Commit message (Collapse)AuthorAge
* Add support for containers.confDaniel J Walsh2020-03-27
| | | | | | | vendor in c/common config pkg for containers.conf Signed-off-by: Qi Wang qiwan@redhat.com Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Attempt manual removal of CNI IP allocations on refreshMatthew Heon2020-03-19
| | | | | | | | | | | | | | We previously attempted to work within CNI to do this, without success. So let's do it manually, instead. We know where the files should live, so we can remove them ourselves instead. This solves issues around sudden reboots where containers do not have time to fully tear themselves down, and leave IP address allocations which, for various reasons, are not stored in tmpfs and persist through reboot. Fixes #5433 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Add structure for new exec session tracking to DBMatthew Heon2020-03-18
| | | | | | | | | | | | | | | | | | | | | | | As part of the rework of exec sessions, we need to address them independently of containers. In the new API, we need to be able to fetch them by their ID, regardless of what container they are associated with. Unfortunately, our existing exec sessions are tied to individual containers; there's no way to tell what container a session belongs to and retrieve it without getting every exec session for every container. This adds a pointer to the container an exec session is associated with to the database. The sessions themselves are still stored in the container. Exec-related APIs have been restructured to work with the new database representation. The originally monolithic API has been split into a number of smaller calls to allow more fine-grained control of lifecycle. Support for legacy exec sessions has been retained, but in a deprecated fashion; we should remove this in a few releases. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Revert "exec: get the exit code from sync pipe instead of file"Matthew Heon2020-03-09
| | | | | | | | | This reverts commit 4b72f9e4013411208751df2a92ab9f322d4da5b2. Continues what began with revert of d3d97a25e8c87cf741b2e24ac01ef84962137106 in previous commit. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Fix spelling mistakes in code found by codespellDaniel J Walsh2020-03-07
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* exec: get the exit code from sync pipe instead of filePeter Hunt2020-03-03
| | | | | | | | Before, we were getting the exit code from the file, in which we waited an arbitrary amount of time (5 seconds) for the file, and segfaulted if we didn't find it. instead, we should be a bit more certain conmon has sent the exit code. Luckily, it sends the exit code along the sync pipe fd, so we can read it from there Adapt the ExecContainer interface to pass along a channel to get the pid and exit code from conmon, to be able to read both from the pipe Signed-off-by: Peter Hunt <pehunt@redhat.com>
* Add basic deadlock detection for container start/removeMatthew Heon2020-02-24
| | | | | | | | | | | | | | | We can easily tell if we're going to deadlock by comparing lock IDs before actually taking the lock. Add a few checks for this in common places where deadlocks might occur. This does not yet cover pod operations, where detection is more difficult (and costly) due to the number of locks being involved being higher than 2. Also, add some error wrapping on the Podman side, so we can tell people to use `system renumber` when it occurs. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* populate resolv.conf with dnsname responses when in usernamespaceBrent Baude2020-02-20
| | | | | | | | when using usernamespace, dnsname respondes from cni were not making it into the containers /etc/resolv.conf because of a timing issue. this corrects that behavior. Fixes: #5256 Signed-off-by: Brent Baude <bbaude@redhat.com>
* Do not copy up when volume is not emptyMatthew Heon2020-02-12
| | | | | | | | | | | | | | | | | When Docker performs a copy up, it first verifies that the volume being copied into is empty; thus, for volumes that have been modified elsewhere (e.g. manually copying into then), the copy up will not be performed at all. Duplicate this behavior in Podman by checking if the volume is empty before copying. Furthermore, move setting copyup to false further up. This will prevent a potential race where copy up could happen more than once if Podman was killed after some files had been copied but before the DB was updated. This resolves CVE-2020-1726. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Tear down network when restarting containersMatthew Heon2020-02-02
| | | | | | | | | | | This makes restart a bit slower for root containers, but it does make it more consistent with `podman stop` and `podman start` on a container. Importantly, `podman restart` will now recreate firewall rules if they were somehow purged. Fixes #5051 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* make lint: enable gocriticValentin Rothberg2020-01-13
| | | | | | | `gocritic` is a powerful linter that helps in preventing certain kinds of errors as well as enforcing a coding style. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* address review comments before mergebaude2020-01-10
| | | | Signed-off-by: baude <bbaude@redhat.com>
* Initial commit on compatible APIJhon Honce2020-01-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Jhon Honce <jhonce@redhat.com> Create service command Use cd cmd/service && go build . $ systemd-socket-activate -l 8081 cmd/service/service & $ curl http://localhost:8081/v1.24/images/json Signed-off-by: Jhon Honce <jhonce@redhat.com> Correct Makefile Signed-off-by: Jhon Honce <jhonce@redhat.com> Two more stragglers Signed-off-by: Jhon Honce <jhonce@redhat.com> Report errors back as http headers Signed-off-by: Jhon Honce <jhonce@redhat.com> Split out handlers, updated output Output aligned to docker structures Signed-off-by: Jhon Honce <jhonce@redhat.com> Refactored routing, added more endpoints and types * Encapsulated all the routing information in the handler_* files. * Added more serviceapi/types, including podman additions. See Info Signed-off-by: Jhon Honce <jhonce@redhat.com> Cleaned up code, implemented info content * Move Content-Type check into serviceHandler * Custom 404 handler showing the url, mostly for debugging * Refactored images: better method names and explicit http codes * Added content to /info * Added podman fields to Info struct * Added Container struct Signed-off-by: Jhon Honce <jhonce@redhat.com> Add a bunch of endpoints containers: stop, pause, unpause, wait, rm images: tag, rmi, create (pull only) Signed-off-by: baude <bbaude@redhat.com> Add even more handlers * Add serviceapi/Error() to improve error handling * Better support for API return payloads * Renamed unimplemented to unsupported these are generic endpoints we don't intend to ever support. Swarm broken out since it uses different HTTP codes to signal that the node is not in a swarm. * Added more types * API Version broken out so it can be validated in the future Signed-off-by: Jhon Honce <jhonce@redhat.com> Refactor to introduce ServiceWriter Signed-off-by: Jhon Honce <jhonce@redhat.com> populate pods endpoints /libpod/pods/.. exists, kill, pause, prune, restart, remove, start, stop, unpause Signed-off-by: baude <bbaude@redhat.com> Add components to Version, fix Error body Signed-off-by: Jhon Honce <jhonce@redhat.com> Add images pull output, fix swarm routes * docker-py tests/integration/api_client_test.py pass 100% * docker-py tests/integration/api_image_test.py pass 4/16 + Test failures include services podman does not support Signed-off-by: Jhon Honce <jhonce@redhat.com> pods endpoint submission 2 add create and others; only top and stats is left. Signed-off-by: baude <bbaude@redhat.com> Update pull image to work from empty registry Signed-off-by: Jhon Honce <jhonce@redhat.com> pod create and container create first pass at pod and container create. the container create does not quite work yet but it is very close. pod create needs a partial rewrite. also broken off the DELETE (rm/rmi) to specific handler funcs. Signed-off-by: baude <bbaude@redhat.com> Add docker-py demos, GET .../containers/json * Update serviceapi/types to reflect libpod not podman * Refactored removeImage() to provide non-streaming return Signed-off-by: Jhon Honce <jhonce@redhat.com> create container part2 finished minimal config needed for create container. started demo.py for upcoming talk Signed-off-by: baude <bbaude@redhat.com> Stop server after honoring request * Remove casting for method calls * Improve WriteResponse() * Update Container API type to match docker API Signed-off-by: Jhon Honce <jhonce@redhat.com> fix namespace assumptions cleaned up namespace issues with libpod. Signed-off-by: baude <bbaude@redhat.com> wip Signed-off-by: baude <bbaude@redhat.com> Add sliding window when shutting down server * Added a Timeout rather than closing down service on each call * Added gorilla/schema dependency for Decode'ing query parameters * Improved error handling * Container logs returned and multiplexed for stdout and stderr * .../containers/{name}/logs?stdout=True&stderr=True * Container stats * .../containers/{name}/stats Signed-off-by: Jhon Honce <jhonce@redhat.com> Improve error handling * Add check for at least one std stream required for /containers/{id}/logs * Add check for state in /containers/{id}/top * Fill in more fields for /info * Fixed error checking in service start code Signed-off-by: Jhon Honce <jhonce@redhat.com> get rest of image tests for pass Signed-off-by: baude <bbaude@redhat.com> linting our content Signed-off-by: baude <bbaude@redhat.com> more linting Signed-off-by: baude <bbaude@redhat.com> more linting Signed-off-by: baude <bbaude@redhat.com> pruning Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]apiv2 pods migrate from using args in the url to using a json struct in body for pod create. Signed-off-by: baude <bbaude@redhat.com> fix handler_images prune prune's api changed slightly to deal with filters. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]enabled base container create tests enabling the base container create tests which allow us to get more into the stop, kill, etc tests. many new tests now pass. Signed-off-by: baude <bbaude@redhat.com> serviceapi errors: append error message to API message I dearly hope this is not breaking any other tests but debugging "Internal Server Error" is not helpful to any user. In case, it breaks tests, we can rever the commit - that's why it's a small one. Signed-off-by: Valentin Rothberg <rothberg@redhat.com> serviceAPI: add containers/prune endpoint Signed-off-by: Valentin Rothberg <rothberg@redhat.com> add `service` make target Also remove the non-functional sub-Makefile. Signed-off-by: Valentin Rothberg <rothberg@redhat.com> add make targets for testing the service * `sudo make run-service` for running the service. * `DOCKERPY_TEST="tests/integration/api_container_test.py::ListContainersTest" \ make run-docker-py-tests` for running a specific tests. Run all tests by leaving the env variable empty. Signed-off-by: Valentin Rothberg <rothberg@redhat.com> Split handlers and server packages The files were split to help contain bloat. The api/server package will contain all code related to the functioning of the server while api/handlers will have all the code related to implementing the end points. api/server/register_* will contain the methods for registering endpoints. Additionally, they will have the comments for generating the swagger spec file. See api/handlers/version.go for a small example handler, api/handlers/containers.go contains much more complex handlers. Signed-off-by: Jhon Honce <jhonce@redhat.com> [CI:DOCS]enabled more tests Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]libpod endpoints small refactor for libpod inclusion and began adding endpoints. Signed-off-by: baude <bbaude@redhat.com> Implement /build and /events * Include crypto libraries for future ssh work Signed-off-by: Jhon Honce <jhonce@redhat.com> [CI:DOCS]more image implementations convert from using for to query structs among other changes including new endpoints. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]add bindings for golang Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]add volume endpoints for libpod create, inspect, ls, prune, and rm Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]apiv2 healthcheck enablement wire up container healthchecks for the api. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]Add mount endpoints via the api, allow ability to mount a container and list container mounts. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]Add search endpoint add search endpoint with golang bindings Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]more apiv2 development misc population of methods, etc Signed-off-by: baude <bbaude@redhat.com> rebase cleanup and epoch reset Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]add more network endpoints also, add some initial error handling and convenience functions for standard endpoints. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]use helper funcs for bindings use the methods developed to make writing bindings less duplicative and easier to use. Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]add return info for prereview begin to add return info and status codes for errors so that we can review the apiv2 Signed-off-by: baude <bbaude@redhat.com> [CI:DOCS]first pass at adding swagger docs for api Signed-off-by: baude <bbaude@redhat.com>
* Ensure SizeRw is shown when a user does 'inspect --size -t container'.Neville Cain2019-12-27
| | | | | | | | | | | | | | Currently, if a user requests the size on a container (inspect --size -t container), the SizeRw does not show up if the value is 0. It's because InspectContainerData is defined as int64 and there is an omit when empty. We do want to display it even if the value is empty. I have changed the type of SizeRw to be a pointer to an int64 instead of an int64. It will allow us todistinguish the empty value to the missing value. I updated the test "podman inspect container with size" to ensure we check thatSizeRw is displayed correctly. Closes #4744 Signed-off-by: NevilleC <neville.cain@qonto.eu>
* if container is not in a pid namespace, stop all processesDaniel J Walsh2019-12-19
| | | | | | | | | When a container is in a PID namespace, it is enought to send the stop signal to the PID 1 of the namespace, only send signals to all processes in the container when the container is not in a pid namespace. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Reap exec sessions on cleanup and removalMatthew Heon2019-12-12
| | | | | | | | | | | | | | | | | | | | | | We currently rely on exec sessions being removed from the state by the Exec() API itself, on detecting the session stopping. This is not a reliable method, though. The Podman frontend for exec could be killed before the session ended, or another Podman process could be holding the lock and prevent update (most notable in `run --rm`, when a container with an active exec session is stopped). To resolve this, add a function to reap active exec sessions from the state, and use it on cleanup (to clear sessions after the container stops) and remove (to do the same when --rm is passed). This is a bit more complicated than it ought to be because Kata and company exist, and we can't guarantee the exec session has a PID on the host, so we have to plumb this through to the OCI runtime. Fixes #4666 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Add ContainerStateRemovingMatthew Heon2019-11-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When Libpod removes a container, there is the possibility that removal will not fully succeed. The most notable problems are storage issues, where the container cannot be removed from c/storage. When this occurs, we were faced with a choice. We can keep the container in the state, appearing in `podman ps` and available for other API operations, but likely unable to do any of them as it's been partially removed. Or we can remove it very early and clean up after it's already gone. We have, until now, used the second approach. The problem that arises is intermittent problems removing storage. We end up removing a container, failing to remove its storage, and ending up with a container permanently stuck in c/storage that we can't remove with the normal Podman CLI, can't use the name of, and generally can't interact with. A notable cause is when Podman is hit by a SIGKILL midway through removal, which can consistently cause `podman rm` to fail to remove storage. We now add a new state for containers that are in the process of being removed, ContainerStateRemoving. We set this at the beginning of the removal process. It notifies Podman that the container cannot be used anymore, but preserves it in the DB until it is fully removed. This will allow Remove to be run on these containers again, which should successfully remove storage if it fails. Fixes #3906 Signed-off-by: Matthew Heon <mheon@redhat.com>
* Also delete winsz fifoPeter Hunt2019-11-15
| | | | | | In conmon 2.0.3, we add another fifo to handle window resizing. This needs to be cleaned up for commands like restore, where the same path is used. Signed-off-by: Peter Hunt <pehunt@redhat.com>
* Add ensureState helper for checking container stateMatthew Heon2019-10-28
| | | | | | | | | | | | | | | | We have a lot of checks for container state scattered throughout libpod. Many of these need to ensure the container is in one of a given set of states so an operation may safely proceed. Previously there was no set way of doing this, so we'd use unique boolean logic for each one. Introduce a helper to standardize state checks. Note that this is only intended to replace checks for multiple states. A simple check for one state (ContainerStateRunning, for example) should remain a straight equality, and not use this new helper. Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add a MissingRuntime implementationMatthew Heon2019-10-15
| | | | | | | | | | | | | | | | | When a container is created with a given OCI runtime, but then it is uninstalled or removed from the configuration file, Libpod presently reacts very poorly. The EvictContainer code can potentially remove these containers, but we still can't see them in `podman ps` (aside from the massive logrus.Errorf messages they create). Providing a minimal OCI runtime implementation for missing runtimes allows us to behave better. We'll be able to retrieve containers from the database, though we still pop up an error for each missing runtime. For containers which are stopped, we can remove them as normal. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Unwrap errors before comparing themNalin Dahyabhai2019-10-14
| | | | | | | Unwrap errors before directly comparing them with errors defined by the storage and image libraries. Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
* Move OCI runtime implementation behind an interfaceMatthew Heon2019-10-10
| | | | | | | | | | | | For future work, we need multiple implementations of the OCI runtime, not just a Conmon-wrapped runtime matching the runc CLI. As part of this, do some refactoring on the interface for exec (move to a struct, not a massive list of arguments). Also, add 'all' support to Kill and Stop (supported by runc and used a bit internally for removing containers). Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* catch runc v2 errorbaude2019-10-09
| | | | | | | when runc returns an error about not being v2 complient, catch the error and logrus an actionable message for users. Signed-off-by: baude <bbaude@redhat.com>
* rootless: do not attempt a CNI refreshGiuseppe Scrivano2019-10-01
| | | | Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Merge pull request #4065 from mheon/unconditional_conmon_rmOpenShift Merge Robot2019-09-27
|\ | | | | Unconditionally remove conmon files before starting
| * Unconditionally remove conmon files before startingMatthew Heon2019-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've been seeing a lot of issues (ref: #4061, but there are others) where Podman hiccups on trying to start a container, because some temporary files have been retained and Conmon will not overwrite them. If we're calling start() we can safely assume that we really want those files gone so the container starts without error, so invoke the cleanup routine. It's relatively cheap (four file removes) so it shouldn't hurt us that much. Also contains a small simplification to the removeConmonFiles logic - we don't need to stat-then-remove when ignoring ENOENT is fine. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* | Force a CNI Delete on refreshing containersMatthew Heon2019-09-24
|/ | | | | | | | | | | | | | | | | | CNI expects that a DELETE be run before re-creating container networks. If a reboot occurs quickly enough that containers can't stop and clean up, that DELETE never happens, and Podman currently wipes the old network info and thinks the state has been entirely cleared. Unfortunately, that may not be the case on the CNI side. Some things - like IP address reservations - may not have been cleared. To solve this, manually re-run CNI Delete on refresh. If the container has already been deleted this seems harmless. If not, it should clear lingering state. Fixes: #3759 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Unmounting a container that is already unmounted is OKMatthew Heon2019-09-16
| | | | | | | | | | | | We should not be throwing errors because the operation we wanted to perform is already done. Now, it is definitely strange that a container is actually unmounted, but shows as mounted in the DB - if this reoccurs in a way where we can investigate, it's worth tearing into. Fixes #4033 Signed-off-by: Matthew Heon <mheon@redhat.com>
* Report errors when trying to pause rootless containersDaniel J Walsh2019-09-13
| | | | | | | | If you are running a rootless container on cgroupV1 you can not pause the container. We need to report the proper error if this happens. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* clean up after healthcheck execsbaude2019-09-12
| | | | | | | | | | when executing a healthcheck, we were not cleaning up after exec's use of a socket. we now remove the socket file and ignore if for reason it does not exist. Fixes: #3962 Signed-off-by: baude <bbaude@redhat.com>
* Merge pull request #3581 from mheon/no_cgroupsOpenShift Merge Robot2019-09-11
|\ | | | | Support running containers without CGroups
| * Add support for launching containers without CGroupsMatthew Heon2019-09-10
| | | | | | | | | | | | | | This is mostly used with Systemd, which really wants to manage CGroups itself when managing containers via unit file. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* | When first mounting any named volume, copy upMatthew Heon2019-09-09
|/ | | | | | | | | | | Previously, we only did this for volumes created at the same time as the container. However, this is not correct behavior - Docker does so for all named volumes, even those made with 'podman volume create' and mounted into a container later. Fixes #3945 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Correctly report errors on unmounting SHMMatthew Heon2019-09-05
| | | | | | | | | | | When we fail to remove a container's SHM, that's an error, and we need to report it as such. This may be part of our lingering storage woes. Also, remove MNT_DETACH. It may be another cause of the storage removal failures. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Add ability for volumes with options to mount/umountMatthew Heon2019-09-05
| | | | | | | | | | | | | When volume options and the local volume driver are specified, the volume is intended to be mounted using the 'mount' command. Supported options will be used to volume the volume before the first container using it starts, and unmount the volume after the last container using it dies. This should work for any local filesystem, though at present I've only tested with tmpfs and btrfs. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* generate systemd: support pods and geneartig filesValentin Rothberg2019-08-21
| | | | | | | | | | | Support generating systemd unit files for a pod. Podman generates one unit file for the pod including the PID file for the infra container's conmon process and one unit file for each container (excluding the infra container). Note that this change implies refactorings in the `pkg/systemdgen` API. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* container stop: kill conmonValentin Rothberg2019-08-05
| | | | | | | | | | | | | Old versions of conmon have a bug where they create the exit file before closing open file descriptors causing a race condition when restarting containers with open ports since we cannot bind the ports as they're not yet closed by conmon. Killing the old conmon PID is ~okay since it forces the FDs of old conmons to be closed, while it's a NOP for newer versions which should have exited already. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Ensure we generate a 'stopped' event on force-removeMatthew Heon2019-07-31
| | | | | | | | | | | | | | When forcibly removing a container, we are initiating an explicit stop of the container, which is not reflected in 'podman events'. Swap to using our standard 'stop()' function instead of a custom one for force-remove, and move the event into the internal stop function (so internal calls also register it). This does add one more database save() to `podman remove`. This should not be a terribly serious performance hit, and does have the desirable side effect of making things generally safer. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* podman: fix memleak caused by renaming and not deletingMatthew Heon2019-07-31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the exit file If the container exit code needs to be retained, it cannot be retained in tmpfs, because libpod runs in a memcg itself so it can't leave traces with a daemon-less design. This wasn't a memleak detectable by kmemleak for example. The kernel never lost track of the memory and there was no erroneous refcounting either. The reference count dependencies however are not easy to track because when a refcount is increased, there's no way to tell who's still holding the reference. In this case it was a single page of tmpfs pagecache holding a refcount that kept pinned a whole hierarchy of dying memcg, slab kmem, cgropups, unrechable kernfs nodes and the respective dentries and inodes. Such a problem wouldn't happen if the exit file was stored in a regular filesystem because the pagecache could be reclaimed in such case under memory pressure. The tmpfs page can be swapped out, but that's not enough to release the memcg with CONFIG_MEMCG_SWAP_ENABLED=y. No amount of more aggressive kernel slab shrinking could have solved this. Not even assigning slab kmem of dying cgroups to alive cgroup would fully solve this. The only way to free the memory of a dying cgroup when a struct page still references it, would be to loop over all "struct page" in the kernel to find which one is associated with the dying cgroup which is a O(N) operation (where N is the number of pages and can reach billions). Linking all the tmpfs pages to the memcg would cost less during memcg offlining, but it would waste lots of memory and CPU globally. So this can't be optimized in the kernel. A cronjob running this command can act as workaround and will allow all slab cache to be released, not just the single tmpfs pages. rm -f /run/libpod/exits/* This patch solved the memleak with a reproducer, booting with cgroup.memory=nokmem and with selinux disabled. The reason memcg kmem and selinux were disabled for testing of this fix, is because kmem greatly decreases the kernel effectiveness in reusing partial slab objects. cgroup.memory=nokmem is strongly recommended at least for workstation usage. selinux needs to be further analyzed because it causes further slab allocations. The upstream podman commit used for testing is 1fe2965e4f672674f7b66648e9973a0ed5434bb4 (v1.4.4). The upstream kernel commit used for testing is f16fea666898dbdd7812ce94068c76da3e3fcf1e (v5.2-rc6). Reported-by: Michele Baldessari <michele@redhat.com> Signed-off-by: Andrea Arcangeli <aarcange@redhat.com> <Applied with small tweaks to comments> Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Merge pull request #3233 from ↵OpenShift Merge Robot2019-07-29
|\ | | | | | | | | wking/fatal-requested-hook-directory-does-not-exist libpod/container_internal: Make all errors loading explicitly configured hook dirs fatal
| * libpod/container_internal: Make all errors loading explicitly configured ↵W. Trevor King2019-05-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | hook dirs fatal Remove this IsNotExist out which was added along with the rest of this block in f6a2b6bf2b (hooks: Add pre-create hooks for runtime-config manipulation, 2018-11-19, #1830). Besides the obvious "hook directory does not exist", it was swallowing the less-obvious "hook command does not exist". And either way, folks are likely going to want non-zero podman exits when we fail to load a hook directory they explicitly pointed us towards. Signed-off-by: W. Trevor King <wking@tremily.us>
* | Implement conmon execPeter Hunt2019-07-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This includes: Implement exec -i and fix some typos in description of -i docs pass failed runtime status to caller Add resize handling for a terminal connection Customize exec systemd-cgroup slice fix healthcheck fix top add --detach-keys Implement podman-remote exec (jhonce) * Cleanup some orphaned code (jhonce) adapt remote exec for conmon exec (pehunt) Fix healthcheck and exec to match docs Introduce two new OCIRuntime errors to more comprehensively describe situations in which the runtime can error Use these different errors in branching for exit code in healthcheck and exec Set conmon to use new api version Signed-off-by: Jhon Honce <jhonce@redhat.com> Signed-off-by: Peter Hunt <pehunt@redhat.com>
* | golangci-lint round #3baude2019-07-21
| | | | | | | | | | | | | | this is the third round of preparing to use the golangci-lint on our code base. Signed-off-by: baude <bbaude@redhat.com>
* | golangci-lint pass number 2baude2019-07-11
| | | | | | | | | | | | clean up and prepare to migrate to the golangci-linter Signed-off-by: baude <bbaude@redhat.com>
* | Merge pull request #3425 from adrianreber/restore-mount-labelOpenShift Merge Robot2019-07-08
|\ \ | | | | | | Set correct SELinux label on restored containers
| * | Provide correct SELinux mount-label for restored containerAdrian Reber2019-06-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restoring a container from a checkpoint archive creates a complete new root file-system. This file-system needs to have the correct SELinux label or most things in that restored container will fail. Running processes are not as problematic as newly exec()'d process (internally or via 'podman exec'). This patch tells the storage setup which label should be used to mount the container's root file-system. Signed-off-by: Adrian Reber <areber@redhat.com>
* | | code cleanupbaude2019-07-08
| | | | | | | | | | | | | | | | | | clean up code identified as problematic by golands inspection Signed-off-by: baude <bbaude@redhat.com>
* | | Merge pull request #3472 from haircommander/generate-volumesOpenShift Merge Robot2019-07-04
|\ \ \ | | | | | | | | generate kube with volumes
| * | | Improve parsing of mountsPeter Hunt2019-07-02
| | | | | | | | | | | | | | | | | | | | | | | | Specifically, we were needlessly doing a double lookup to find which config mounts were user volumes. Improve this by refactoring a bit of code from inspect Signed-off-by: Peter Hunt <pehunt@redhat.com>
* | | | trivial cleanups from golangbaude2019-07-03
| | | | | | | | | | | | | | | | | | | | | | | | the results of a code cleanup performed by the goland IDE. Signed-off-by: baude <bbaude@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-13 22:59:18 +0000