aboutsummaryrefslogtreecommitdiff
path: root/libpod/util.go
Commit message (Collapse)AuthorAge
* inotify: make sure to remove filesValentin Rothberg2021-12-09
| | | | | | | | | | | | | | | | Issue #11825 suggests that *rootless* Podman can run into situations where too many inotify fds are open. Indeed, rootless Podman has a slightly higher usage of inotify watchers than the root counterpart when using slirp4netns Make sure to not only close all watchers but to also remove the files from being watched. Otherwise, the fds only get closed when the files are removed. [NO NEW TESTS NEEDED] since we don't have a way to test it. Fixes: #11825 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* libpod: deduplicate ports in dbPaul Holzinger2021-10-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The OCICNI port format has one big problem: It does not support ranges. So if a users forwards a range of 1k ports with podman run -p 1001-2000 we have to store each of the thousand ports individually as array element. This bloats the db and makes the JSON encoding and decoding much slower. In many places we already use a better port struct type which supports ranges, e.g. `pkg/specgen` or the new network interface. Because of this we have to do many runtime conversions between the two port formats. If everything uses the new format we can skip the runtime conversions. This commit adds logic to replace all occurrences of the old format with the new one. The database will automatically migrate the ports to new format when the container config is read for the first time after the update. The `ParsePortMapping` function is `pkg/specgen/generate` has been reworked to better work with the new format. The new logic is able to deduplicate the given ports. This is necessary the ensure we store them efficiently in the DB. The new code should also be more performant than the old one. To prove that the code is fast enough I added go benchmarks. Parsing 1 million ports took less than 0.5 seconds on my laptop. Benchmark normalize PortMappings in specgen: Please note that the 1 million ports are actually 20x 50k ranges because we cannot have bigger ranges than 65535 ports. ``` $ go test -bench=. -benchmem ./pkg/specgen/generate/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/pkg/specgen/generate cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz BenchmarkParsePortMappingNoPorts-12 480821532 2.230 ns/op 0 B/op 0 allocs/op BenchmarkParsePortMapping1-12 38972 30183 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMapping100-12 18752 60688 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMapping1k-12 3104 331719 ns/op 223840 B/op 3018 allocs/op BenchmarkParsePortMapping10k-12 376 3122930 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMapping1m-12 3 390869926 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingReverse100-12 18940 63414 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMappingReverse1k-12 3015 362500 ns/op 223841 B/op 3018 allocs/op BenchmarkParsePortMappingReverse10k-12 343 3318135 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMappingReverse1m-12 3 403392469 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingRange1-12 37635 28756 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange100-12 39604 28935 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1k-12 38384 29921 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange10k-12 29479 40381 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1m-12 927 1279369 ns/op 143022 B/op 164 allocs/op PASS ok github.com/containers/podman/v3/pkg/specgen/generate 25.492s ``` Benchmark convert old port format to new one: ``` go test -bench=. -benchmem ./libpod/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/libpod cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz Benchmark_ocicniPortsToNetTypesPortsNoPorts-12 663526126 1.663 ns/op 0 B/op 0 allocs/op Benchmark_ocicniPortsToNetTypesPorts1-12 7858082 141.9 ns/op 72 B/op 2 allocs/op Benchmark_ocicniPortsToNetTypesPorts10-12 2065347 571.0 ns/op 536 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts100-12 138478 8641 ns/op 4216 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1k-12 9414 120964 ns/op 41080 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts10k-12 781 1490526 ns/op 401528 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1m-12 4 250579010 ns/op 40001656 B/op 4 allocs/op PASS ok github.com/containers/podman/v3/libpod 11.727s ``` Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* standardize logrus messages to upper caseDaniel J Walsh2021-09-22
| | | | | | | | Remove ERROR: Error stutter from logrus messages also. [ NO TESTS NEEDED] This is just code cleanup. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Drop OCICNI dependencyPaul Holzinger2021-09-15
| | | | | | | | | | | We do not use the ocicni code anymore so let's get rid of it. Only the port struct is used but we can copy this into libpod network types so we can debloat the binary. The next step is to remove the OCICNI port mapping form the container config and use the better PortMapping struct everywhere. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* podman inspect show exposed portsPaul Holzinger2021-08-24
| | | | | | | | | | | | | | Podman inspect has to show exposed ports to match docker. This requires storing the exposed ports in the container config. A exposed port is shown as `"80/tcp": null` while a forwarded port is shown as `"80/tcp": [{"HostIp": "", "HostPort": "8080" }]`. Also make sure to add the exposed ports to the new image when the container is commited. Fixes #10777 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* podman info: try qfile before equeryEd Santiago2021-08-03
| | | | | | | | | | | | | | | | | | podman info takes >20s on Gentoo, because equery is s..l..o..w. qfile is much faster and, I suspect, present in most Gentoo installations, so let's try it first. And, because packageVersion() was scarily unmaintainable, refactor it. Define a simple (string) list of packaging tools to query (rpm, dpkg, ...) and iterate until we find one that works. IMPORTANT NOTE: the Debian (and, presumably, Ubuntu) query does not include version number! There is no standard way on Debian to get a package version from a file path, you can only do it via pipes of chained commands, and I have no desire to implement that. Signed-off-by: Ed Santiago <santiago@redhat.com>
* Use seccomp_profile as default profile if defined in containers.confPablo Correa Gómez2021-04-28
| | | | | | | Edits `podman info` to provide the default seccomp profile detected in the output Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
* bump go module to v3Valentin Rothberg2021-02-22
| | | | | | | | | We missed bumping the go module, so let's do it now :) * Automated go code with github.com/sirkon/go-imports-rename * Manually via `vgrep podman/v2` the rest Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* remote exec: write conmon error on hijacked connectionValentin Rothberg2021-01-27
| | | | | | | | | | | | | | Make sure to write error from conmon on the hijacked http connection. This fixes issues where errors were not reported on the client side, for instance, when specified command was not found on the container. To future generations: I am sorry. The code is complex, and there are many interdependencies among the concurrent goroutines. I added more complexity on top but I don't have a good idea of how to reduce complexity in the available time. Fixes: #8281 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Add support for Gentoo file to package queryMichael Vetter2020-12-29
| | | | | | | | On Gentoo systems where `app-portage/gentoolkit` is installed the binary `equery` is used to query for information on which package a file belongs to. Signed-off-by: Michael Vetter <jubalh@iodoru.org>
* SpellingJosh Soref2020-12-22
| | | | Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* Add support for pacman package version queryxatier2020-12-16
| | | | | | | | | | - refactor package query logic to be package manager agnostic. - `pacman -Qo` is the equivalent to `rpm -qf` [1]. [1] https://wiki.archlinux.org/index.php/Pacman/Rosetta#Querying_specific_packages Signed-off-by: xatier <xatierlike@gmail.com>
* Stop excessive wrapping of errorsDaniel J Walsh2020-10-30
| | | | | | | | | | | | Most of the builtin golang functions like os.Stat and os.Open report errors including the file system object path. We should not wrap these errors and put the file path in a second time, causing stuttering of errors when they get presented to the user. This patch tries to cleanup a bunch of these errors. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Send HTTP Hijack headers after successful attachMatthew Heon2020-08-27
| | | | | | | | | | | | | | | | | | | | | | | | | Our previous flow was to perform a hijack before passing a connection into Libpod, and then Libpod would attach to the container's attach socket and begin forwarding traffic. A problem emerges: we write the attach header as soon as the attach complete. As soon as we write the header, the client assumes that all is ready, and sends a Start request. This Start may be processed *before* we successfully finish attaching, causing us to lose output. The solution is to handle hijacking inside Libpod. Unfortunately, this requires a downright extensive refactor of the Attach and HTTP Exec StartAndAttach code. I think the result is an improvement in some places (a lot more errors will be handled with a proper HTTP error code, before the hijack occurs) but other parts, like the relocation of printing container logs, are just *bad*. Still, we need this fixed now to get CI back into good shape... Fixes #7195 Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Switch all references to github.com/containers/libpod -> podmanDaniel J Walsh2020-07-28
| | | | Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Preserve passwd on container restartMatthew Heon2020-07-15
| | | | | | | | | | | | | | | We added code to create a `/etc/passwd` file that we bind-mount into the container in some cases (most notably, `--userns=keep-id` containers). This, unfortunately, was not persistent, so user-added users would be dropped on container restart. Changing where we store the file should fix this. Further, we want to ensure that lookups of users in the container use the right /etc/passwd if we replaced it. There was already logic to do this, but it only worked for user-added mounts; it's easy enough to alter it to use our mounts as well. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Include infra container information in `pod inspect`Matthew Heon2020-07-14
| | | | | | | | | | We had a field for this in the inspect data, but it was never being populated. Because of this, `podman pod inspect` stopped showing port bindings (and other infra container settings). Add code to populate the infra container inspect data, and add a test to ensure we don't regress again. Signed-off-by: Matthew Heon <mheon@redhat.com>
* move go module to v2Valentin Rothberg2020-07-06
| | | | | | | | | | | | | | | With the advent of Podman 2.0.0 we crossed the magical barrier of go modules. While we were able to continue importing all packages inside of the project, the project could not be vendored anymore from the outside. Move the go module to new major version and change all imports to `github.com/containers/libpod/v2`. The renaming of the imports was done via `gomove` [1]. [1] https://github.com/KSubedi/gomove Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* WIP V2 attach bindings and testJhon Honce2020-05-13
| | | | | | | | * Add ErrLostSync to report lost of sync when de-mux'ing stream * Add logus.SetLevel(logrus.DebugLevel) when `go test -v` given * Add context to debugging messages Signed-off-by: Jhon Honce <jhonce@redhat.com>
* v2 podman statsbaude2020-05-05
| | | | Signed-off-by: baude <bbaude@redhat.com>
* Improve APIv2 support for AttachMatthew Heon2020-04-13
| | | | | | | | | | | | | | A few major fixes here: - Support for attaching to Configured containers, to match Docker behavior. - Support for stream parameter has been improved (we now properly handle cases where it is not set). - Initial support for logs parameter has been added. - Setting attach streams when the container has a terminal is now supported. - Errors are properly reported once the hijack has begun. Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add support for containers.confDaniel J Walsh2020-03-27
| | | | | | | vendor in c/common config pkg for containers.conf Signed-off-by: Qi Wang qiwan@redhat.com Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Add an API for Attach over HTTP APIMatthew Heon2020-01-16
| | | | | | | | | | | | | | | | | | | | | | | | | | The new APIv2 branch provides an HTTP-based remote API to Podman. The requirements of this are, unfortunately, incompatible with the existing Attach API. For non-terminal attach, we need append a header to what was copied from the container, to multiplex STDOUT and STDERR; to do this with the old API, we'd need to copy into an intermediate buffer first, to handle the headers. To avoid this, provide a new API to handle all aspects of terminal and non-terminal attach, including closing the hijacked HTTP connection. This might be a bit too specific, but for now, it seems to be the simplest approach. At the same time, add a Resize endpoint. This needs to be a separate endpoint, so our existing channel approach does not work here. I wanted to rework the rest of attach at the same time (some parts of it, particularly how we start the Attach session and how we do resizing, are (in my opinion) handled much better here. That may still be on the table, but I wanted to avoid breaking existing APIs in this already massive change. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Add ContainerStateRemovingMatthew Heon2019-11-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When Libpod removes a container, there is the possibility that removal will not fully succeed. The most notable problems are storage issues, where the container cannot be removed from c/storage. When this occurs, we were faced with a choice. We can keep the container in the state, appearing in `podman ps` and available for other API operations, but likely unable to do any of them as it's been partially removed. Or we can remove it very early and clean up after it's already gone. We have, until now, used the second approach. The problem that arises is intermittent problems removing storage. We end up removing a container, failing to remove its storage, and ending up with a container permanently stuck in c/storage that we can't remove with the normal Podman CLI, can't use the name of, and generally can't interact with. A notable cause is when Podman is hit by a SIGKILL midway through removal, which can consistently cause `podman rm` to fail to remove storage. We now add a new state for containers that are in the process of being removed, ContainerStateRemoving. We set this at the beginning of the removal process. It notifies Podman that the container cannot be used anymore, but preserves it in the DB until it is fully removed. This will allow Remove to be run on these containers again, which should successfully remove storage if it fails. Fixes #3906 Signed-off-by: Matthew Heon <mheon@redhat.com>
* Set SELinux labels based on the security context in the kube.yamlDaniel J Walsh2019-11-05
| | | | | | | If the kube.yaml specifieds the SELinux type or Level, we need the container to be launched with the correct label. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* add libpod/configValentin Rothberg2019-10-31
| | | | | | | | | | | | Refactor the `RuntimeConfig` along with related code from libpod into libpod/config. Note that this is a first step of consolidating code into more coherent packages to make the code more maintainable and less prone to regressions on the long runs. Some libpod definitions were moved to `libpod/define` to resolve circular dependencies. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Set default seccomp.json file for podman play kubeDaniel J Walsh2019-10-29
| | | | | | | Currently podman play kube is not using the system default seccomp.json file. This PR will use the default or override location for podman play. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Return information about mount_program (fuse-overlayfs)Daniel J Walsh2019-09-06
| | | | | | | | | | We want to get podman info to tell us about the version of the mount program to help us diagnose issues users are having. Also if in rootless mode and slirp4netns is installed reveal package info on slirp4netns. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* libpod: avoid polling container statusGiuseppe Scrivano2019-09-04
| | | | | | | | | | | | | | | | | | | | | | | use the inotify backend to be notified on the container exit instead of polling continuosly the runtime. Polling the runtime slowns significantly down the podman execution time for short lived processes: $ time bin/podman run --rm -ti fedora true real 0m0.324s user 0m0.088s sys 0m0.064s from: $ time podman run --rm -ti fedora true real 0m4.199s user 0m5.339s sys 0m0.344s Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* code cleanupbaude2019-07-08
| | | | | | clean up code identified as problematic by golands inspection Signed-off-by: baude <bbaude@redhat.com>
* libpod removal from main (phase 2)baude2019-06-27
| | | | | | this is phase 2 for the removal of libpod from main. Signed-off-by: baude <bbaude@redhat.com>
* remove libpod from mainbaude2019-06-25
| | | | | | | | | | | | | the compilation demands of having libpod in main is a burden for the remote client compilations. to combat this, we should move the use of libpod structs, vars, constants, and functions into the adapter code where it will only be compiled by the local client. this should result in cleaner code organization and smaller binaries. it should also help if we ever need to compile the remote client on non-Linux operating systems natively (not cross-compiled). Signed-off-by: baude <bbaude@redhat.com>
* util: fix race condition in WaitForFileGiuseppe Scrivano2019-05-20
| | | | | | | | | | | enable polling also when using inotify. It is generally useful to have it as under high load inotify can lose notifications. It also solves a race condition where the file is created while the watcher is configured and it'd wait until the timeout and fail. Closes: https://github.com/containers/libpod/issues/2942 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Remove ulele/deepcopier in favor of JSON deep copyMatthew Heon2019-03-27
| | | | | | | | | | | | | We have a very high performance JSON library that doesn't need to perform code generation. Let's use it instead of our questionably performant, reflection-dependent deep copy library. Most changes because some functions can now return errors. Also converts cmd/podman to use jsoniter, instead of pkg/json, for increased performance. Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Fix build on non-LinuxMiloslav Trmač2018-12-06
| | | | | | Unrelated to the rest of the PR. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
* util: use fsnotify to wait for fileGiuseppe Scrivano2018-11-28
| | | | | | | | | | | prefer a fsnotify watcher to polling the file, we take advantage of inotify on Linux and react more promptly to the PID file being created. If the watcher cannot be created, then fallback to the old polling mechanism. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* exec: don't wait for pidfile when the runtime exitedGiuseppe Scrivano2018-11-27
| | | | | | | | | | | don't wait for the timeout to expire if the runtime process exited. I've noticed podman to hang on exit and keeping the container lock taken when the OCI runtime already exited. Additionally, it reduces the waiting time as we won't hit the 25 milliseconds waiting time in the worst case. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* create: fix writing cidfile when using rootlessGiuseppe Scrivano2018-10-23
| | | | | | | | | | prevent opening the same file twice, since we re-exec podman in rootless mode. While at it, also solve a possible race between the check for the file and writing to it. Another process could have created the file in the meanwhile and we would just end up overwriting it. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Add ability for ubuntu to be testedbaude2018-10-03
| | | | | | | | unfortunately the papr CI system cannot test ubuntu as a VM; therefore, this PR still keeps travis. but it does include fixes that will be required for running on modern versions of ubuntu. Signed-off-by: baude <bbaude@redhat.com>
* Refactor error checking in With*NSFromPod optionshaircommander2018-08-23
| | | | | | | Signed-off-by: haircommander <pehunt@redhat.com> Closes: #1187 Approved by: mheon
* Fix build on non-Linux OSesMatthew Heon2018-08-15
| | | | | | | Signed-off-by: Matthew Heon <matthew.heon@gmail.com> Closes: #1266 Approved by: baude
* Create pod CGroups when using the systemd cgroup driverMatthew Heon2018-08-15
| | | | | | | Signed-off-by: Matthew Heon <matthew.heon@gmail.com> Closes: #1266 Approved by: baude
* We need to sort mounts so that one mount does not over mount another.Daniel J Walsh2018-08-10
| | | | | | | | | | | | | | | | | Currently we add mounts from images, volumes and internal. We can accidently over mount an existing mount. This patch sorts the mounts to make sure a parent directory is always mounted before its content. Had to change the default propagation on image volume mounts from shared to private to stop mount points from leaking out of the container. Also switched from using some docker/docker/pkg to container/storage/pkg to remove some dependencies on Docker. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> Closes: #1243 Approved by: mheon
* Stage 4 Image cleanupbaude2018-03-23
| | | | | | | | | | Cull funcs from runtime_img.go which are no longer needed. Also, fix any remaining spots that use the old image technique. Signed-off-by: baude <bbaude@redhat.com> Closes: #532 Approved by: mheon
* Stage3 Image Librarybaude2018-03-14
| | | | | | | | | | | | | This represents the stage3 implementation for the image library. At this point, we are moving the image-centric functions to pkg/image including migration of args and object-oriented references. This is a not a one-for-one migration of funcs and some funcs will need to continue to reside in runtime_img as they are overly specific to libpod and probably not useful to others. Signed-off-by: baude <bbaude@redhat.com> Closes: #484 Approved by: baude
* Add tracking for exec session IDsMatthew Heon2018-03-01
| | | | | | | | | | | Exec sessions now have an ID generated and assigned to their PID and stored in the database state. This allows us to track what exec sessions are currently active. Signed-off-by: Matthew Heon <matthew.heon@gmail.com> Closes: #412 Approved by: baude
* Implement --image-volumes for create and runumohnani82018-02-22
| | | | | | | | | | | | | | --image-volumes tells podman what to do with the image volumes in the image config There are 3 options: bind, tmpfs, and ignore bind puts the volume contents in /var/lib/containers/storage/container-id/volumes/vol-dir and bind mounts it into the container at /vol-dir tmpfs mounts /vol-dir as a tmps into the container ignore doesn't mount the image volumes onto the container Signed-off-by: umohnani8 <umohnani@redhat.com> Closes: #377 Approved by: rhatdan
* podman run container id to filebaude2018-01-11
| | | | | | | | | | podman run --cidfile /tmp/foo writes the container's id to a file. Signed-off-by: baude <bbaude@redhat.com> Closes: #205 Approved by: rhatdan
* kpod statsbaude2017-12-13
| | | | | | | | | Move kpod stats to the libpod backend. Signed-off-by: baude <bbaude@redhat.com> Closes: #113 Approved by: baude
* remove unused functions and variablesSuraj Deshmukh2017-11-28
| | | | | | | Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com> Closes: #71 Approved by: rhatdan
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-14 21:28:59 +0000