summaryrefslogtreecommitdiff
path: root/pkg/specgen
Commit message (Collapse)AuthorAge
* Merge pull request #11890 from Luap99/portsOpenShift Merge Robot2021-11-06
|\ | | | | libpod: deduplicate ports in db
| * libpod: deduplicate ports in dbPaul Holzinger2021-10-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The OCICNI port format has one big problem: It does not support ranges. So if a users forwards a range of 1k ports with podman run -p 1001-2000 we have to store each of the thousand ports individually as array element. This bloats the db and makes the JSON encoding and decoding much slower. In many places we already use a better port struct type which supports ranges, e.g. `pkg/specgen` or the new network interface. Because of this we have to do many runtime conversions between the two port formats. If everything uses the new format we can skip the runtime conversions. This commit adds logic to replace all occurrences of the old format with the new one. The database will automatically migrate the ports to new format when the container config is read for the first time after the update. The `ParsePortMapping` function is `pkg/specgen/generate` has been reworked to better work with the new format. The new logic is able to deduplicate the given ports. This is necessary the ensure we store them efficiently in the DB. The new code should also be more performant than the old one. To prove that the code is fast enough I added go benchmarks. Parsing 1 million ports took less than 0.5 seconds on my laptop. Benchmark normalize PortMappings in specgen: Please note that the 1 million ports are actually 20x 50k ranges because we cannot have bigger ranges than 65535 ports. ``` $ go test -bench=. -benchmem ./pkg/specgen/generate/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/pkg/specgen/generate cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz BenchmarkParsePortMappingNoPorts-12 480821532 2.230 ns/op 0 B/op 0 allocs/op BenchmarkParsePortMapping1-12 38972 30183 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMapping100-12 18752 60688 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMapping1k-12 3104 331719 ns/op 223840 B/op 3018 allocs/op BenchmarkParsePortMapping10k-12 376 3122930 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMapping1m-12 3 390869926 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingReverse100-12 18940 63414 ns/op 141088 B/op 315 allocs/op BenchmarkParsePortMappingReverse1k-12 3015 362500 ns/op 223841 B/op 3018 allocs/op BenchmarkParsePortMappingReverse10k-12 343 3318135 ns/op 1223650 B/op 30027 allocs/op BenchmarkParsePortMappingReverse1m-12 3 403392469 ns/op 124593840 B/op 4000624 allocs/op BenchmarkParsePortMappingRange1-12 37635 28756 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange100-12 39604 28935 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1k-12 38384 29921 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange10k-12 29479 40381 ns/op 131584 B/op 9 allocs/op BenchmarkParsePortMappingRange1m-12 927 1279369 ns/op 143022 B/op 164 allocs/op PASS ok github.com/containers/podman/v3/pkg/specgen/generate 25.492s ``` Benchmark convert old port format to new one: ``` go test -bench=. -benchmem ./libpod/ goos: linux goarch: amd64 pkg: github.com/containers/podman/v3/libpod cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz Benchmark_ocicniPortsToNetTypesPortsNoPorts-12 663526126 1.663 ns/op 0 B/op 0 allocs/op Benchmark_ocicniPortsToNetTypesPorts1-12 7858082 141.9 ns/op 72 B/op 2 allocs/op Benchmark_ocicniPortsToNetTypesPorts10-12 2065347 571.0 ns/op 536 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts100-12 138478 8641 ns/op 4216 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1k-12 9414 120964 ns/op 41080 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts10k-12 781 1490526 ns/op 401528 B/op 4 allocs/op Benchmark_ocicniPortsToNetTypesPorts1m-12 4 250579010 ns/op 40001656 B/op 4 allocs/op PASS ok github.com/containers/podman/v3/libpod 11.727s ``` Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* | Fix swagger definition for the new mac address typePaul Holzinger2021-11-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The new mac address type broke the api docs. While we could successfully generate the swagger file it could not be viewed in a browser. The problem is that the swagger generation create two type definitions with the name `HardwareAddr` and this pointed back to itself. Thus the render process was stucked in an endless loop. To fix this manually rename the new type to MacAddress and overwrite the types to string because the json unmarshaller accepts the mac as string. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* | MAC address json unmarshal should allow stringsPaul Holzinger2021-11-03
| | | | | | | | | | | | | | | | | | Create a new mac address type which supports json marshal/unmarshal from and to string. This change is backwards compatible with the previous versions as the unmarshal method still accepts the old byte array or base64 encoded string. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* | volumes: be more tolerant and fix infinite loopValentin Rothberg2021-10-28
|/ | | | | | | | | | | | | | | | | | | | | | Make Podman more tolerant when parsing image volumes during container creation and further fix an infinite loop when checking them. Consider `VOLUME ['/etc/foo', '/etc/bar']` in a Containerfile. While it looks correct to the human eye, the single quotes are wrong and yield the two volumes to be `[/etc/foo,` and `/etc/bar]` in Podman and Docker. When running the container, it'll create a directory `bar]` in `/etc` and a directory `[` in `/` with two subdirectories `etc/foo,`. This behavior is surprising to me but how Docker behaves. We may improve on that in the future. Note that the correct way to syntax for volumes in a Containerfile is `VOLUME /A /B /C` or `VOLUME ["/A", "/B", "/C"]`; single quotes are not supported. This change restores this behavior without breaking container creation or ending up in an infinite loop. BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2014149 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* pod create: remove need for pause imageValentin Rothberg2021-10-26
| | | | | | | | | So far, the infra containers of pods required pulling down an image rendering pods not usable in disconnected environments. Instead, build an image locally which uses local pause binary. Fixes: #10354 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Add support to play kube for --log-optDaniel J Walsh2021-10-25
| | | | | | Fixes: https://github.com/containers/podman/issues/11727 Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Allow API to specify size and inode quotaDaniel J Walsh2021-10-18
| | | | | | | | | | | Fixes: https://github.com/containers/podman/issues/11016 [NO NEW TESTS NEEDED] We have no easy way to tests this in CI/CD systems. Requires quota to be setup on directories to work. Fixes: https://github.com/containers/podman/issues/11016 Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Fix codespell errorsDaniel J Walsh2021-10-14
| | | | | | | | Along with a couple of nits found by Ed. [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Merge pull request #11920 from jwhonce/issues/11891OpenShift Merge Robot2021-10-11
|\ | | | | Use SplitN(2) when copying env variables
| * Use SplitN(2) when copying env variablesJhon Honce2021-10-11
| | | | | | | | | | | | | | | | | | Environment variables whose value contained an equal sign where truncated Fixes #11891 Signed-off-by: Jhon Honce <jhonce@redhat.com>
* | fix testchenkang2021-10-11
| | | | | | | | Signed-off-by: chenkang <kongchen28@gmail.com>
* | Support readonly rootfs contains colonchenkang2021-10-11
|/ | | | | | Fix: https://github.com/containers/podman/issues/11913 Signed-off-by: chenkang <kongchen28@gmail.com>
* faster image inspectionValentin Rothberg2021-10-08
| | | | | | | | | | | | | | Vendor the latest HEAD in c/common to pull in changes for a faster inspection of images. Previously, only the size computation was optional, now the one for the parent image is as well. In many cases, the parent image is not needed but it takes around 10ms on my local machine. With this change, we cut off 10ms from many code paths, most importantly, container creation. [NO NEW TESTS NEEDED] Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Merge pull request #11816 from cdoern/podEventsBugOpenShift Merge Robot2021-10-02
|\ | | | | Pod Events Logging Fix
| * Pod Events Logging Fixcdoern2021-10-01
| | | | | | | | | | | | | | on create, libpod was only creating a new event if the pod had an infra container. now, pod creation triggers a new pod event with or without infra Signed-off-by: cdoern <cdoern@redhat.com>
* | Pod Volumes From Supportcdoern2021-10-01
|/ | | | | | | added support for a volumes from container. this flag just required movement of the volumes-from flag declaration out of the !IsInfra block, and minor modificaions to container_create.go Signed-off-by: cdoern <cdoern@redhat.com>
* Merge pull request #11707 from rhatdan/playOpenShift Merge Robot2021-10-01
|\ | | | | Add podman play kube --no-hosts options
| * Add podman play kube --no-hosts optionsDaniel J Walsh2021-10-01
| | | | | | | | | | | | | | | | | | This option will setup the containers to not modify their /etc/hosts file and just use the one from the image. Fixes: https://github.com/containers/podman/issues/9500 Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* | Merge pull request #11808 from vrothberg/cacheOpenShift Merge Robot2021-10-01
|\ \ | | | | | | pkg/specgen: cache image in generator
| * | pkg/specgen: cache image in generatorValentin Rothberg2021-10-01
| |/ | | | | | | | | | | | | | | | | | | To prevent expensive redundant lookups and inspects on the same image, cache the image in the generator. Note that once a given image has been inspected, subsequent calls will use the libimage-internal cache. [NO TESTS NEEDED] since it is no functional change. Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* | Merge pull request #11686 from cdoern/podDeviceOptionsOpenShift Merge Robot2021-10-01
|\ \ | |/ |/| Pod Device-Read-BPS support
| * Pod Device-Read-BPS supportcdoern2021-09-28
| | | | | | | | | | | | | | | | | | added the option for the user to specify a rate, in bytes, at which they would like to be able to read from the device being added to the pod. This is the first in a line of pod device options. WARNING: changed pod name json tag to pod_name to avoid confusion when marshaling with the containerspec's name Signed-off-by: cdoern <cdoern@redhat.com>
* | Merge pull request #11811 from umohnani8/swaggerOpenShift Merge Robot2021-09-30
|\ \ | | | | | | Add expose type map[uint16]string to description
| * | Add expose type map[uint16]string to descriptionUrvashi Mohnani2021-09-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Swagger-go doesn't generate the types correctly for some complicated structs. We are seeing this with the expose option for container create, it is showing up as any. Add a line to the description to highlight that the type is map[uint16]string. [NO TESTS NEEDED] Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
* | | Merge pull request #11794 from umohnani8/pidOpenShift Merge Robot2021-09-30
|\ \ \ | | | | | | | | Allow a value of -1 to set unlimited pids limit
| * | | Allow a value of -1 to set unlimited pids limitUrvashi Mohnani2021-09-29
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | Users can set --pids-limit to -1 now to set unlimited pids limit for a container - this matches the convention. [NO TESTS NEEDED] Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
* | | Merge pull request #11806 from giuseppe/play-kube-fix-cpu-limitsOpenShift Merge Robot2021-09-30
|\ \ \ | | | | | | | | kube: fix conversion from milliCPU to period/quota
| * | | kube: fix conversion from milliCPU to period/quotaGiuseppe Scrivano2021-09-30
| |/ / | | | | | | | | | | | | | | | Closes: https://github.com/containers/podman/issues/11803 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* / / Support selinux options with bind mounts play/genBrent Baude2021-09-30
|/ / | | | | | | | | | | | | | | | | | | | | | | When using play kube and generate kube, we need to support if bind mounts have selinux options. As kubernetes does not support selinux in this way, we tuck the selinux values into a pod annotation for generation of the kube yaml. Then on play, we check annotations to see if a value for the mount exists and apply it. Fixes BZ #1984081 Signed-off-by: Brent Baude <bbaude@redhat.com>
* | libpod: add GetConfigNoCopy()Valentin Rothberg2021-09-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new function to libpod to directly access the runtime configuration without creating an expensive deep copy. Further migrate a number of callers to this new function. This drops the number of calls to JSONDeepCopy from 4 to 1 in a simple `podman run --rm -d busybox top`. Future work: Please note that there are more callers of GetConfig() that can me migrated to GetConfigNoCopy(). [NO TESTS NEEDED] Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* | set --cni-config-dir for exit commandPaul Holzinger2021-09-28
| | | | | | | | Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* | standardize logrus messages to upper caseDaniel J Walsh2021-09-22
|/ | | | | | | | Remove ERROR: Error stutter from logrus messages also. [ NO TESTS NEEDED] This is just code cleanup. Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Pod Device Supportcdoern2021-09-20
| | | | | | | | | | added support for pod devices. The device gets added to the infra container and recreated in all containers that join the pod. This required a new container config item to keep track of the original device passed in by the user before the path was parsed into the container device. Signed-off-by: cdoern <cdoern@redhat.com>
* Drop OCICNI dependencyPaul Holzinger2021-09-15
| | | | | | | | | | | We do not use the ocicni code anymore so let's get rid of it. Only the port struct is used but we can copy this into libpod network types so we can debloat the binary. The next step is to remove the OCICNI port mapping form the container config and use the better PortMapping struct everywhere. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Merge pull request #11409 from cdoern/podVolumesOpenShift Merge Robot2021-09-15
|\ | | | | Pod Volumes Support
| * Pod Volumes Supportcdoern2021-09-14
| | | | | | | | | | | | | | | | | | added support for the --volume flag in pods using the new infra container design. users can specify all volume options they can with regular containers resolves #10379 Signed-off-by: cdoern <cdoern@redhat.com>
* | rootfs: Add support for rootfs-overlay and bump to buildah v1.22.1-0.202108flouthoc2021-09-14
|/ | | | | | | | | | Allows users to specify a readonly rootfs with :O, in exchange podman will create a writable overlay. bump builah to v1.22.1-0.20210823173221-da2b428c56ce [NO TESTS NEEDED] Signed-off-by: flouthoc <flouthoc.git@gmail.com>
* Add init containers to generate and play kubeUrvashi Mohnani2021-09-10
| | | | | | | | | | | | | | | Kubernetes has a concept of init containers that run and exit before the regular containers in a pod are started. We added init containers to podman pods as well. This patch adds support for generating init containers in the kube yaml when a pod we are converting had init containers. When playing a kube yaml, it detects an init container and creates such a container in podman accordingly. Note, only init containers created with the init type set to "always" will be generated as the "once" option deletes the init container after it has run and exited. Play kube will always creates init containers with the "always" init container type. Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
* InfraContainer Reworkcdoern2021-08-26
| | | | | | | | | | InfraContainer should go through the same creation process as regular containers. This change was from the cmd level down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state) Signed-off-by: cdoern <cdoern@redhat.com>
* Merge pull request #11314 from Luap99/expose-portsOpenShift Merge Robot2021-08-25
|\ | | | | podman inspect show exposed ports
| * podman inspect show exposed portsPaul Holzinger2021-08-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Podman inspect has to show exposed ports to match docker. This requires storing the exposed ports in the container config. A exposed port is shown as `"80/tcp": null` while a forwarded port is shown as `"80/tcp": [{"HostIp": "", "HostPort": "8080" }]`. Also make sure to add the exposed ports to the new image when the container is commited. Fixes #10777 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* | Network interfacePaul Holzinger2021-08-24
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement a new network interface to abstract CNI from libpod. The interface is implemented for the CNI backend but in the future we can add more backends. The code is structured in three new packages: - `libpod/network/types`: contains the interface definition and the necessary types for it. - `libpod/network/cni` contains the interface implementation for the CNI backend. - `libpod/network/util` a set of utility functions related to networking. The CNI package uses ginkgo style unit tests. To test Setup/Teardown the test must be run as root. Each test will run in their own namespace to make the test independent from the host environment. New features with the CNI backend: - The default network will be created in memory if it does not exists on disk. - It can set more than one static IP per container network. - Networks are loaded once from disk and only if this interface is used, e.g. for commands such as `podman info` networks are not loaded. This reduces unnecessary disk IO. This commit only adds the interface it is not wired into libpod. This requires a lot of breaking changes which will be done in a followup commit. Once this is integrated into libpod the current network code under `libpod/network` should be removed. Also the dependency on OCICNI should be dropped. Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* fix: unifiedOverlays should be assigned if no conflicts found.flouthoc2021-08-17
| | | | | | [NO TESTS NEEDED] Signed-off-by: flouthoc <flouthoc.git@gmail.com>
* libpod/option.go remove error stutter from wrap/wrafflouthoc2021-08-17
| | | | | [NO TESTS NEEDED] Signed-off-by: flouthoc <flouthoc.git@gmail.com>
* volume: move validating volume dest from client to server.flouthoc2021-08-16
| | | | | | [NO TESTS NEEDED] Signed-off-by: flouthoc <flouthoc.git@gmail.com>
* rename oneshot initcontainers to onceBrent Baude2021-08-12
| | | | | | | | | | after the init containers pr merged, it was suggested to use `once` instead of `oneshot` containers as it is more aligned with other terminiology used similarily. [NO TESTS NEEDED] Signed-off-by: Brent Baude <bbaude@redhat.com>
* Add support for pod inside of user namespace.Daniel J Walsh2021-08-09
| | | | | | | | | | | | | Add the --userns flag to podman pod create and keep track of the userns setting that pod was created with so that all containers created within the pod will inherit that userns setting. Specifically we need to be able to launch a pod with --userns=keep-id Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
* personality: Add support for setting execution domain.flouthoc2021-08-06
| | | | | | | | | | Execution domains tell Linux how to map signal numbers into signal actions. The execution domain system allows Linux to provide limited support for binaries compiled under other UNIX-like operating systems. Reference: https://man7.org/linux/man-pages/man2/personality.2.html Signed-off-by: flouthoc <flouthoc.git@gmail.com>
* implement init containers in podmanBrent Baude2021-08-04
| | | | | | | | | | | | | | this is the first pass at implementing init containers for podman pods. init containersare made popular by k8s as a way to run setup for pods before the pods standard containers run. unlike k8s, we support two styles of init containers: always and oneshot. always means the container stays in the pod and starts whenever a pod is started. this does not apply to pods restarting. oneshot means the container runs onetime when the pod starts and then is removed. Signed-off-by: Brent Baude <bbaude@redhat.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-12-29 14:21:34 +0000