diff options
Diffstat (limited to 'vendor/github.com/projectatomic/buildah')
25 files changed, 5610 insertions, 0 deletions
diff --git a/vendor/github.com/projectatomic/buildah/LICENSE b/vendor/github.com/projectatomic/buildah/LICENSE new file mode 100644 index 000000000..8dada3eda --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/vendor/github.com/projectatomic/buildah/README.md b/vendor/github.com/projectatomic/buildah/README.md new file mode 100644 index 000000000..ef430153d --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/README.md @@ -0,0 +1,79 @@ + + +# [Buildah](https://www.youtube.com/embed/YVk5NgSiUw8) - a tool that facilitates building OCI container images + +[](https://goreportcard.com/report/github.com/projectatomic/buildah) +[](https://travis-ci.org/projectatomic/buildah) + +Note: this package is in alpha, but is close to being feature-complete. + +The Buildah package provides a command line tool that can be used to +* create a working container, either from scratch or using an image as a starting point +* create an image, either from a working container or via the instructions in a Dockerfile +* images can be built in either the OCI image format or the traditional upstream docker image format +* mount a working container's root filesystem for manipulation +* unmount a working container's root filesystem +* use the updated contents of a container's root filesystem as a filesystem layer to create a new image +* delete a working container or an image + +**[Changelog](CHANGELOG.md)** + +**[Installation notes](install.md)** + +**[Troubleshooting Guide](troubleshooting.md)** + +**[Tutorials](docs/tutorials/README.md)** + +## Example + +From [`./examples/lighttpd.sh`](examples/lighttpd.sh): + +```bash +$ cat > lighttpd.sh <<"EOF" +#!/bin/bash -x + +ctr1=`buildah from ${1:-fedora}` + +## Get all updates and install our minimal httpd server +buildah run $ctr1 -- dnf update -y +buildah run $ctr1 -- dnf install -y lighttpd + +## Include some buildtime annotations +buildah config --annotation "com.example.build.host=$(uname -n)" $ctr1 + +## Run our server and expose the port +buildah config --cmd "/usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf" $ctr1 +buildah config --port 80 $ctr1 + +## Commit this container to an image name +buildah commit $ctr1 ${2:-$USER/lighttpd} +EOF + +$ chmod +x lighttpd.sh +$ sudo ./lighttpd.sh +``` + +## Commands +| Command | Description | +| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| [buildah-add(1)](/docs/buildah-add.md) | Add the contents of a file, URL, or a directory to the container. | +| [buildah-bud(1)](/docs/buildah-bud.md) | Build an image using instructions from Dockerfiles. | +| [buildah-commit(1)](/docs/buildah-commit.md) | Create an image from a working container. | +| [buildah-config(1)](/docs/buildah-config.md) | Update image configuration settings. | +| [buildah-containers(1)](/docs/buildah-containers.md) | List the working containers and their base images. | +| [buildah-copy(1)](/docs/buildah-copy.md) | Copies the contents of a file, URL, or directory into a container's working directory. | +| [buildah-from(1)](/docs/buildah-from.md) | Creates a new working container, either from scratch or using a specified image as a starting point. | +| [buildah-images(1)](/docs/buildah-images.md) | List images in local storage. | +| [buildah-inspect(1)](/docs/buildah-inspect.md) | Inspects the configuration of a container or image. | +| [buildah-mount(1)](/docs/buildah-mount.md) | Mount the working container's root filesystem. | +| [buildah-push(1)](/docs/buildah-push.md) | Push an image from local storage to elsewhere. | +| [buildah-rm(1)](/docs/buildah-rm.md) | Removes one or more working containers. | +| [buildah-rmi(1)](/docs/buildah-rmi.md) | Removes one or more images. | +| [buildah-run(1)](/docs/buildah-run.md) | Run a command inside of the container. | +| [buildah-tag(1)](/docs/buildah-tag.md) | Add an additional name to a local image. | +| [buildah-umount(1)](/docs/buildah-umount.md) | Unmount a working container's root file system. | +| [buildah-version(1)](/docs/buildah-version.md) | Display the Buildah Version Information | + +**Future goals include:** +* more CI tests +* additional CLI commands (?) diff --git a/vendor/github.com/projectatomic/buildah/add.go b/vendor/github.com/projectatomic/buildah/add.go new file mode 100644 index 000000000..4fab5a8d7 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/add.go @@ -0,0 +1,253 @@ +package buildah + +import ( + "io" + "net/http" + "net/url" + "os" + "path" + "path/filepath" + "strings" + "syscall" + "time" + + "github.com/containers/storage/pkg/archive" + "github.com/opencontainers/runtime-spec/specs-go" + "github.com/pkg/errors" + "github.com/projectatomic/libpod/pkg/chrootuser" + "github.com/sirupsen/logrus" +) + +//AddAndCopyOptions holds options for add and copy commands. +type AddAndCopyOptions struct { + Chown string +} + +// addURL copies the contents of the source URL to the destination. This is +// its own function so that deferred closes happen after we're done pulling +// down each item of potentially many. +func addURL(destination, srcurl string) error { + logrus.Debugf("saving %q to %q", srcurl, destination) + resp, err := http.Get(srcurl) + if err != nil { + return errors.Wrapf(err, "error getting %q", srcurl) + } + defer resp.Body.Close() + f, err := os.Create(destination) + if err != nil { + return errors.Wrapf(err, "error creating %q", destination) + } + if last := resp.Header.Get("Last-Modified"); last != "" { + if mtime, err2 := time.Parse(time.RFC1123, last); err2 != nil { + logrus.Debugf("error parsing Last-Modified time %q: %v", last, err2) + } else { + defer func() { + if err3 := os.Chtimes(destination, time.Now(), mtime); err3 != nil { + logrus.Debugf("error setting mtime to Last-Modified time %q: %v", last, err3) + } + }() + } + } + defer f.Close() + n, err := io.Copy(f, resp.Body) + if err != nil { + return errors.Wrapf(err, "error reading contents for %q", destination) + } + if resp.ContentLength >= 0 && n != resp.ContentLength { + return errors.Errorf("error reading contents for %q: wrong length (%d != %d)", destination, n, resp.ContentLength) + } + if err := f.Chmod(0600); err != nil { + return errors.Wrapf(err, "error setting permissions on %q", destination) + } + return nil +} + +// Add copies the contents of the specified sources into the container's root +// filesystem, optionally extracting contents of local files that look like +// non-empty archives. +func (b *Builder) Add(destination string, extract bool, options AddAndCopyOptions, source ...string) error { + mountPoint, err := b.Mount(b.MountLabel) + if err != nil { + return err + } + defer func() { + if err2 := b.Unmount(); err2 != nil { + logrus.Errorf("error unmounting container: %v", err2) + } + }() + // Find out which user (and group) the destination should belong to. + user, err := b.user(mountPoint, options.Chown) + if err != nil { + return err + } + dest := mountPoint + if destination != "" && filepath.IsAbs(destination) { + dest = filepath.Join(dest, destination) + } else { + if err = ensureDir(filepath.Join(dest, b.WorkDir()), user, 0755); err != nil { + return err + } + dest = filepath.Join(dest, b.WorkDir(), destination) + } + // If the destination was explicitly marked as a directory by ending it + // with a '/', create it so that we can be sure that it's a directory, + // and any files we're copying will be placed in the directory. + if len(destination) > 0 && destination[len(destination)-1] == os.PathSeparator { + if err = ensureDir(dest, user, 0755); err != nil { + return err + } + } + // Make sure the destination's parent directory is usable. + if destpfi, err2 := os.Stat(filepath.Dir(dest)); err2 == nil && !destpfi.IsDir() { + return errors.Errorf("%q already exists, but is not a subdirectory)", filepath.Dir(dest)) + } + // Now look at the destination itself. + destfi, err := os.Stat(dest) + if err != nil { + if !os.IsNotExist(err) { + return errors.Wrapf(err, "couldn't determine what %q is", dest) + } + destfi = nil + } + if len(source) > 1 && (destfi == nil || !destfi.IsDir()) { + return errors.Errorf("destination %q is not a directory", dest) + } + for _, src := range source { + if strings.HasPrefix(src, "http://") || strings.HasPrefix(src, "https://") { + // We assume that source is a file, and we're copying + // it to the destination. If the destination is + // already a directory, create a file inside of it. + // Otherwise, the destination is the file to which + // we'll save the contents. + url, err := url.Parse(src) + if err != nil { + return errors.Wrapf(err, "error parsing URL %q", src) + } + d := dest + if destfi != nil && destfi.IsDir() { + d = filepath.Join(dest, path.Base(url.Path)) + } + if err := addURL(d, src); err != nil { + return err + } + if err := setOwner("", d, user); err != nil { + return err + } + continue + } + + glob, err := filepath.Glob(src) + if err != nil { + return errors.Wrapf(err, "invalid glob %q", src) + } + if len(glob) == 0 { + return errors.Wrapf(syscall.ENOENT, "no files found matching %q", src) + } + for _, gsrc := range glob { + srcfi, err := os.Stat(gsrc) + if err != nil { + return errors.Wrapf(err, "error reading %q", gsrc) + } + if srcfi.IsDir() { + // The source is a directory, so copy the contents of + // the source directory into the target directory. Try + // to create it first, so that if there's a problem, + // we'll discover why that won't work. + if err = ensureDir(dest, user, 0755); err != nil { + return err + } + logrus.Debugf("copying %q to %q", gsrc+string(os.PathSeparator)+"*", dest+string(os.PathSeparator)+"*") + if err := copyWithTar(gsrc, dest); err != nil { + return errors.Wrapf(err, "error copying %q to %q", gsrc, dest) + } + if err := setOwner(gsrc, dest, user); err != nil { + return err + } + continue + } + if !extract || !archive.IsArchivePath(gsrc) { + // This source is a file, and either it's not an + // archive, or we don't care whether or not it's an + // archive. + d := dest + if destfi != nil && destfi.IsDir() { + d = filepath.Join(dest, filepath.Base(gsrc)) + } + // Copy the file, preserving attributes. + logrus.Debugf("copying %q to %q", gsrc, d) + if err := copyFileWithTar(gsrc, d); err != nil { + return errors.Wrapf(err, "error copying %q to %q", gsrc, d) + } + if err := setOwner(gsrc, d, user); err != nil { + return err + } + continue + } + // We're extracting an archive into the destination directory. + logrus.Debugf("extracting contents of %q into %q", gsrc, dest) + if err := untarPath(gsrc, dest); err != nil { + return errors.Wrapf(err, "error extracting %q into %q", gsrc, dest) + } + } + } + return nil +} + +// user returns the user (and group) information which the destination should belong to. +func (b *Builder) user(mountPoint string, userspec string) (specs.User, error) { + if userspec == "" { + userspec = b.User() + } + + uid, gid, err := chrootuser.GetUser(mountPoint, userspec) + u := specs.User{ + UID: uid, + GID: gid, + Username: userspec, + } + return u, err +} + +// setOwner sets the uid and gid owners of a given path. +func setOwner(src, dest string, user specs.User) error { + fid, err := os.Stat(dest) + if err != nil { + return errors.Wrapf(err, "error reading %q", dest) + } + if !fid.IsDir() || src == "" { + if err := os.Lchown(dest, int(user.UID), int(user.GID)); err != nil { + return errors.Wrapf(err, "error setting ownership of %q", dest) + } + return nil + } + err = filepath.Walk(src, func(p string, info os.FileInfo, we error) error { + relPath, err2 := filepath.Rel(src, p) + if err2 != nil { + return errors.Wrapf(err2, "error getting relative path of %q to set ownership on destination", p) + } + if relPath != "." { + absPath := filepath.Join(dest, relPath) + if err2 := os.Lchown(absPath, int(user.UID), int(user.GID)); err != nil { + return errors.Wrapf(err2, "error setting ownership of %q", absPath) + } + } + return nil + }) + if err != nil { + return errors.Wrapf(err, "error walking dir %q to set ownership", src) + } + return nil +} + +// ensureDir creates a directory if it doesn't exist, setting ownership and permissions as passed by user and perm. +func ensureDir(path string, user specs.User, perm os.FileMode) error { + if _, err := os.Stat(path); os.IsNotExist(err) { + if err := os.MkdirAll(path, perm); err != nil { + return errors.Wrapf(err, "error ensuring directory %q exists", path) + } + if err := os.Chown(path, int(user.UID), int(user.GID)); err != nil { + return errors.Wrapf(err, "error setting ownership of %q", path) + } + } + return nil +} diff --git a/vendor/github.com/projectatomic/buildah/buildah.go b/vendor/github.com/projectatomic/buildah/buildah.go new file mode 100644 index 000000000..9b55dc320 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/buildah.go @@ -0,0 +1,359 @@ +package buildah + +import ( + "context" + "encoding/json" + "io" + "io/ioutil" + "os" + "path/filepath" + + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/containers/storage/pkg/ioutils" + "github.com/opencontainers/image-spec/specs-go/v1" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/docker" +) + +const ( + // Package is the name of this package, used in help output and to + // identify working containers. + Package = "buildah" + // Version for the Package. Bump version in contrib/rpm/buildah.spec + // too. + Version = "0.16" + // The value we use to identify what type of information, currently a + // serialized Builder structure, we are using as per-container state. + // This should only be changed when we make incompatible changes to + // that data structure, as it's used to distinguish containers which + // are "ours" from ones that aren't. + containerType = Package + " 0.0.1" + // The file in the per-container directory which we use to store our + // per-container state. If it isn't there, then the container isn't + // one of our build containers. + stateFile = Package + ".json" +) + +const ( + // PullIfMissing is one of the values that BuilderOptions.PullPolicy + // can take, signalling that the source image should be pulled from a + // registry if a local copy of it is not already present. + PullIfMissing = iota + // PullAlways is one of the values that BuilderOptions.PullPolicy can + // take, signalling that a fresh, possibly updated, copy of the image + // should be pulled from a registry before the build proceeds. + PullAlways + // PullNever is one of the values that BuilderOptions.PullPolicy can + // take, signalling that the source image should not be pulled from a + // registry if a local copy of it is not already present. + PullNever +) + +// Builder objects are used to represent containers which are being used to +// build images. They also carry potential updates which will be applied to +// the image's configuration when the container's contents are used to build an +// image. +type Builder struct { + store storage.Store + + // Type is used to help identify a build container's metadata. It + // should not be modified. + Type string `json:"type"` + // FromImage is the name of the source image which was used to create + // the container, if one was used. It should not be modified. + FromImage string `json:"image,omitempty"` + // FromImageID is the ID of the source image which was used to create + // the container, if one was used. It should not be modified. + FromImageID string `json:"image-id"` + // Config is the source image's configuration. It should not be + // modified. + Config []byte `json:"config,omitempty"` + // Manifest is the source image's manifest. It should not be modified. + Manifest []byte `json:"manifest,omitempty"` + + // Container is the name of the build container. It should not be modified. + Container string `json:"container-name,omitempty"` + // ContainerID is the ID of the build container. It should not be modified. + ContainerID string `json:"container-id,omitempty"` + // MountPoint is the last location where the container's root + // filesystem was mounted. It should not be modified. + MountPoint string `json:"mountpoint,omitempty"` + // ProcessLabel is the SELinux process label associated with the container + ProcessLabel string `json:"process-label,omitempty"` + // MountLabel is the SELinux mount label associated with the container + MountLabel string `json:"mount-label,omitempty"` + + // ImageAnnotations is a set of key-value pairs which is stored in the + // image's manifest. + ImageAnnotations map[string]string `json:"annotations,omitempty"` + // ImageCreatedBy is a description of how this container was built. + ImageCreatedBy string `json:"created-by,omitempty"` + + // Image metadata and runtime settings, in multiple formats. + OCIv1 v1.Image `json:"ociv1,omitempty"` + Docker docker.V2Image `json:"docker,omitempty"` + // DefaultMountsFilePath is the file path holding the mounts to be mounted in "host-path:container-path" format + DefaultMountsFilePath string `json:"defaultMountsFilePath,omitempty"` + CommonBuildOpts *CommonBuildOptions +} + +// BuilderInfo are used as objects to display container information +type BuilderInfo struct { + Type string + FromImage string + FromImageID string + Config string + Manifest string + Container string + ContainerID string + MountPoint string + ProcessLabel string + MountLabel string + ImageAnnotations map[string]string + ImageCreatedBy string + OCIv1 v1.Image + Docker docker.V2Image + DefaultMountsFilePath string +} + +// GetBuildInfo gets a pointer to a Builder object and returns a BuilderInfo object from it. +// This is used in the inspect command to display Manifest and Config as string and not []byte. +func GetBuildInfo(b *Builder) BuilderInfo { + return BuilderInfo{ + Type: b.Type, + FromImage: b.FromImage, + FromImageID: b.FromImageID, + Config: string(b.Config), + Manifest: string(b.Manifest), + Container: b.Container, + ContainerID: b.ContainerID, + MountPoint: b.MountPoint, + ProcessLabel: b.ProcessLabel, + ImageAnnotations: b.ImageAnnotations, + ImageCreatedBy: b.ImageCreatedBy, + OCIv1: b.OCIv1, + Docker: b.Docker, + DefaultMountsFilePath: b.DefaultMountsFilePath, + } +} + +// CommonBuildOptions are reseources that can be defined by flags for both buildah from and bud +type CommonBuildOptions struct { + // AddHost is the list of hostnames to add to the resolv.conf + AddHost []string + //CgroupParent it the path to cgroups under which the cgroup for the container will be created. + CgroupParent string + //CPUPeriod limits the CPU CFS (Completely Fair Scheduler) period + CPUPeriod uint64 + //CPUQuota limits the CPU CFS (Completely Fair Scheduler) quota + CPUQuota int64 + //CPUShares (relative weight + CPUShares uint64 + //CPUSetCPUs in which to allow execution (0-3, 0,1) + CPUSetCPUs string + //CPUSetMems memory nodes (MEMs) in which to allow execution (0-3, 0,1). Only effective on NUMA systems. + CPUSetMems string + //Memory limit + Memory int64 + //MemorySwap limit value equal to memory plus swap. + MemorySwap int64 + //SecruityOpts modify the way container security is running + LabelOpts []string + SeccompProfilePath string + ApparmorProfile string + //ShmSize is the shared memory size + ShmSize string + //Ulimit options + Ulimit []string + //Volumes to bind mount into the container + Volumes []string +} + +// BuilderOptions are used to initialize a new Builder. +type BuilderOptions struct { + // FromImage is the name of the image which should be used as the + // starting point for the container. It can be set to an empty value + // or "scratch" to indicate that the container should not be based on + // an image. + FromImage string + // Container is a desired name for the build container. + Container string + // PullPolicy decides whether or not we should pull the image that + // we're using as a base image. It should be PullIfMissing, + // PullAlways, or PullNever. + PullPolicy int + // Registry is a value which is prepended to the image's name, if it + // needs to be pulled and the image name alone can not be resolved to a + // reference to a source image. No separator is implicitly added. + Registry string + // Transport is a value which is prepended to the image's name, if it + // needs to be pulled and the image name alone, or the image name and + // the registry together, can not be resolved to a reference to a + // source image. No separator is implicitly added. + Transport string + // Mount signals to NewBuilder() that the container should be mounted + // immediately. + Mount bool + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string + // ReportWriter is an io.Writer which will be used to log the reading + // of the source image from a registry, if we end up pulling the image. + ReportWriter io.Writer + // github.com/containers/image/types SystemContext to hold credentials + // and other authentication/authorization information. + SystemContext *types.SystemContext + // DefaultMountsFilePath is the file path holding the mounts to be mounted in "host-path:container-path" format + DefaultMountsFilePath string + CommonBuildOpts *CommonBuildOptions +} + +// ImportOptions are used to initialize a Builder from an existing container +// which was created elsewhere. +type ImportOptions struct { + // Container is the name of the build container. + Container string + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string +} + +// ImportFromImageOptions are used to initialize a Builder from an image. +type ImportFromImageOptions struct { + // Image is the name or ID of the image we'd like to examine. + Image string + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string + // github.com/containers/image/types SystemContext to hold information + // about which registries we should check for completing image names + // that don't include a domain portion. + SystemContext *types.SystemContext +} + +// NewBuilder creates a new build container. +func NewBuilder(ctx context.Context, store storage.Store, options BuilderOptions) (*Builder, error) { + return newBuilder(ctx, store, options) +} + +// ImportBuilder creates a new build configuration using an already-present +// container. +func ImportBuilder(ctx context.Context, store storage.Store, options ImportOptions) (*Builder, error) { + return importBuilder(ctx, store, options) +} + +// ImportBuilderFromImage creates a new builder configuration using an image. +// The returned object can be modified and examined, but it can not be saved +// or committed because it is not associated with a working container. +func ImportBuilderFromImage(ctx context.Context, store storage.Store, options ImportFromImageOptions) (*Builder, error) { + return importBuilderFromImage(ctx, store, options) +} + +// OpenBuilder loads information about a build container given its name or ID. +func OpenBuilder(store storage.Store, container string) (*Builder, error) { + cdir, err := store.ContainerDirectory(container) + if err != nil { + return nil, err + } + buildstate, err := ioutil.ReadFile(filepath.Join(cdir, stateFile)) + if err != nil { + return nil, err + } + b := &Builder{} + err = json.Unmarshal(buildstate, &b) + if err != nil { + return nil, err + } + if b.Type != containerType { + return nil, errors.Errorf("container is not a %s container", Package) + } + b.store = store + b.fixupConfig() + return b, nil +} + +// OpenBuilderByPath loads information about a build container given a +// path to the container's root filesystem +func OpenBuilderByPath(store storage.Store, path string) (*Builder, error) { + containers, err := store.Containers() + if err != nil { + return nil, err + } + abs, err := filepath.Abs(path) + if err != nil { + return nil, err + } + builderMatchesPath := func(b *Builder, path string) bool { + return (b.MountPoint == path) + } + for _, container := range containers { + cdir, err := store.ContainerDirectory(container.ID) + if err != nil { + return nil, err + } + buildstate, err := ioutil.ReadFile(filepath.Join(cdir, stateFile)) + if err != nil { + return nil, err + } + b := &Builder{} + err = json.Unmarshal(buildstate, &b) + if err == nil && b.Type == containerType && builderMatchesPath(b, abs) { + b.store = store + b.fixupConfig() + return b, nil + } + } + return nil, storage.ErrContainerUnknown +} + +// OpenAllBuilders loads all containers which have a state file that we use in +// their data directory, typically so that they can be listed. +func OpenAllBuilders(store storage.Store) (builders []*Builder, err error) { + containers, err := store.Containers() + if err != nil { + return nil, err + } + for _, container := range containers { + cdir, err := store.ContainerDirectory(container.ID) + if err != nil { + return nil, err + } + buildstate, err := ioutil.ReadFile(filepath.Join(cdir, stateFile)) + if err != nil && os.IsNotExist(err) { + continue + } + b := &Builder{} + err = json.Unmarshal(buildstate, &b) + if err == nil && b.Type == containerType { + b.store = store + b.fixupConfig() + builders = append(builders, b) + } + } + return builders, nil +} + +// Save saves the builder's current state to the build container's metadata. +// This should not need to be called directly, as other methods of the Builder +// object take care of saving their state. +func (b *Builder) Save() error { + buildstate, err := json.Marshal(b) + if err != nil { + return err + } + cdir, err := b.store.ContainerDirectory(b.ContainerID) + if err != nil { + return err + } + return ioutils.AtomicWriteFile(filepath.Join(cdir, stateFile), buildstate, 0600) +} diff --git a/vendor/github.com/projectatomic/buildah/commit.go b/vendor/github.com/projectatomic/buildah/commit.go new file mode 100644 index 000000000..a5b8aaf40 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/commit.go @@ -0,0 +1,154 @@ +package buildah + +import ( + "context" + "fmt" + "io" + "time" + + cp "github.com/containers/image/copy" + "github.com/containers/image/signature" + is "github.com/containers/image/storage" + "github.com/containers/image/transports" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/containers/storage/pkg/archive" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/util" + "github.com/sirupsen/logrus" +) + +// CommitOptions can be used to alter how an image is committed. +type CommitOptions struct { + // PreferredManifestType is the preferred type of image manifest. The + // image configuration format will be of a compatible type. + PreferredManifestType string + // Compression specifies the type of compression which is applied to + // layer blobs. The default is to not use compression, but + // archive.Gzip is recommended. + Compression archive.Compression + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string + // AdditionalTags is a list of additional names to add to the image, if + // the transport to which we're writing the image gives us a way to add + // them. + AdditionalTags []string + // ReportWriter is an io.Writer which will be used to log the writing + // of the new image. + ReportWriter io.Writer + // HistoryTimestamp is the timestamp used when creating new items in the + // image's history. If unset, the current time will be used. + HistoryTimestamp *time.Time + // github.com/containers/image/types SystemContext to hold credentials + // and other authentication/authorization information. + SystemContext *types.SystemContext +} + +// PushOptions can be used to alter how an image is copied somewhere. +type PushOptions struct { + // Compression specifies the type of compression which is applied to + // layer blobs. The default is to not use compression, but + // archive.Gzip is recommended. + Compression archive.Compression + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string + // ReportWriter is an io.Writer which will be used to log the writing + // of the new image. + ReportWriter io.Writer + // Store is the local storage store which holds the source image. + Store storage.Store + // github.com/containers/image/types SystemContext to hold credentials + // and other authentication/authorization information. + SystemContext *types.SystemContext + // ManifestType is the format to use when saving the imge using the 'dir' transport + // possible options are oci, v2s1, and v2s2 + ManifestType string +} + +// Commit writes the contents of the container, along with its updated +// configuration, to a new image in the specified location, and if we know how, +// add any additional tags that were specified. +func (b *Builder) Commit(ctx context.Context, dest types.ImageReference, options CommitOptions) error { + policy, err := signature.DefaultPolicy(getSystemContext(options.SystemContext, options.SignaturePolicyPath)) + if err != nil { + return errors.Wrapf(err, "error obtaining default signature policy") + } + policyContext, err := signature.NewPolicyContext(policy) + if err != nil { + return errors.Wrapf(err, "error creating new signature policy context") + } + defer func() { + if err2 := policyContext.Destroy(); err2 != nil { + logrus.Debugf("error destroying signature policy context: %v", err2) + } + }() + // Check if we're keeping everything in local storage. If so, we can take certain shortcuts. + _, destIsStorage := dest.Transport().(is.StoreTransport) + exporting := !destIsStorage + src, err := b.makeImageRef(options.PreferredManifestType, exporting, options.Compression, options.HistoryTimestamp) + if err != nil { + return errors.Wrapf(err, "error computing layer digests and building metadata") + } + // "Copy" our image to where it needs to be. + err = cp.Image(ctx, policyContext, dest, src, getCopyOptions(options.ReportWriter, nil, options.SystemContext, "")) + if err != nil { + return errors.Wrapf(err, "error copying layers and metadata") + } + if len(options.AdditionalTags) > 0 { + switch dest.Transport().Name() { + case is.Transport.Name(): + img, err := is.Transport.GetStoreImage(b.store, dest) + if err != nil { + return errors.Wrapf(err, "error locating just-written image %q", transports.ImageName(dest)) + } + err = util.AddImageNames(b.store, img, options.AdditionalTags) + if err != nil { + return errors.Wrapf(err, "error setting image names to %v", append(img.Names, options.AdditionalTags...)) + } + logrus.Debugf("assigned names %v to image %q", img.Names, img.ID) + default: + logrus.Warnf("don't know how to add tags to images stored in %q transport", dest.Transport().Name()) + } + } + + img, err := is.Transport.GetStoreImage(b.store, dest) + if err == nil { + fmt.Printf("%s\n", img.ID) + } + return nil +} + +// Push copies the contents of the image to a new location. +func Push(ctx context.Context, image string, dest types.ImageReference, options PushOptions) error { + systemContext := getSystemContext(options.SystemContext, options.SignaturePolicyPath) + policy, err := signature.DefaultPolicy(systemContext) + if err != nil { + return errors.Wrapf(err, "error obtaining default signature policy") + } + policyContext, err := signature.NewPolicyContext(policy) + if err != nil { + return errors.Wrapf(err, "error creating new signature policy context") + } + // Look up the image. + src, err := is.Transport.ParseStoreReference(options.Store, image) + if err != nil { + return errors.Wrapf(err, "error parsing reference to image %q", image) + } + // Copy everything. + err = cp.Image(ctx, policyContext, dest, src, getCopyOptions(options.ReportWriter, nil, options.SystemContext, options.ManifestType)) + if err != nil { + return errors.Wrapf(err, "error copying layers and metadata") + } + if options.ReportWriter != nil { + fmt.Fprintf(options.ReportWriter, "\n") + } + return nil +} diff --git a/vendor/github.com/projectatomic/buildah/common.go b/vendor/github.com/projectatomic/buildah/common.go new file mode 100644 index 000000000..18c960003 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/common.go @@ -0,0 +1,28 @@ +package buildah + +import ( + "io" + + cp "github.com/containers/image/copy" + "github.com/containers/image/types" +) + +func getCopyOptions(reportWriter io.Writer, sourceSystemContext *types.SystemContext, destinationSystemContext *types.SystemContext, manifestType string) *cp.Options { + return &cp.Options{ + ReportWriter: reportWriter, + SourceCtx: sourceSystemContext, + DestinationCtx: destinationSystemContext, + ForceManifestMIMEType: manifestType, + } +} + +func getSystemContext(defaults *types.SystemContext, signaturePolicyPath string) *types.SystemContext { + sc := &types.SystemContext{} + if defaults != nil { + *sc = *defaults + } + if signaturePolicyPath != "" { + sc.SignaturePolicyPath = signaturePolicyPath + } + return sc +} diff --git a/vendor/github.com/projectatomic/buildah/config.go b/vendor/github.com/projectatomic/buildah/config.go new file mode 100644 index 000000000..efbb133de --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/config.go @@ -0,0 +1,610 @@ +package buildah + +import ( + "encoding/json" + "path/filepath" + "runtime" + "strings" + "time" + + digest "github.com/opencontainers/go-digest" + ociv1 "github.com/opencontainers/image-spec/specs-go/v1" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/docker" +) + +// makeOCIv1Image builds the best OCIv1 image structure we can from the +// contents of the docker image structure. +func makeOCIv1Image(dimage *docker.V2Image) (ociv1.Image, error) { + config := dimage.Config + if config == nil { + config = &dimage.ContainerConfig + } + dcreated := dimage.Created.UTC() + image := ociv1.Image{ + Created: &dcreated, + Author: dimage.Author, + Architecture: dimage.Architecture, + OS: dimage.OS, + Config: ociv1.ImageConfig{ + User: config.User, + ExposedPorts: map[string]struct{}{}, + Env: config.Env, + Entrypoint: config.Entrypoint, + Cmd: config.Cmd, + Volumes: config.Volumes, + WorkingDir: config.WorkingDir, + Labels: config.Labels, + StopSignal: config.StopSignal, + }, + RootFS: ociv1.RootFS{ + Type: "", + DiffIDs: []digest.Digest{}, + }, + History: []ociv1.History{}, + } + for port, what := range config.ExposedPorts { + image.Config.ExposedPorts[string(port)] = what + } + RootFS := docker.V2S2RootFS{} + if dimage.RootFS != nil { + RootFS = *dimage.RootFS + } + if RootFS.Type == docker.TypeLayers { + image.RootFS.Type = docker.TypeLayers + image.RootFS.DiffIDs = append(image.RootFS.DiffIDs, RootFS.DiffIDs...) + } + for _, history := range dimage.History { + hcreated := history.Created.UTC() + ohistory := ociv1.History{ + Created: &hcreated, + CreatedBy: history.CreatedBy, + Author: history.Author, + Comment: history.Comment, + EmptyLayer: history.EmptyLayer, + } + image.History = append(image.History, ohistory) + } + return image, nil +} + +// makeDockerV2S2Image builds the best docker image structure we can from the +// contents of the OCI image structure. +func makeDockerV2S2Image(oimage *ociv1.Image) (docker.V2Image, error) { + image := docker.V2Image{ + V1Image: docker.V1Image{Created: oimage.Created.UTC(), + Author: oimage.Author, + Architecture: oimage.Architecture, + OS: oimage.OS, + ContainerConfig: docker.Config{ + User: oimage.Config.User, + ExposedPorts: docker.PortSet{}, + Env: oimage.Config.Env, + Entrypoint: oimage.Config.Entrypoint, + Cmd: oimage.Config.Cmd, + Volumes: oimage.Config.Volumes, + WorkingDir: oimage.Config.WorkingDir, + Labels: oimage.Config.Labels, + StopSignal: oimage.Config.StopSignal, + }, + }, + RootFS: &docker.V2S2RootFS{ + Type: "", + DiffIDs: []digest.Digest{}, + }, + History: []docker.V2S2History{}, + } + for port, what := range oimage.Config.ExposedPorts { + image.ContainerConfig.ExposedPorts[docker.Port(port)] = what + } + if oimage.RootFS.Type == docker.TypeLayers { + image.RootFS.Type = docker.TypeLayers + image.RootFS.DiffIDs = append(image.RootFS.DiffIDs, oimage.RootFS.DiffIDs...) + } + for _, history := range oimage.History { + dhistory := docker.V2S2History{ + Created: history.Created.UTC(), + CreatedBy: history.CreatedBy, + Author: history.Author, + Comment: history.Comment, + EmptyLayer: history.EmptyLayer, + } + image.History = append(image.History, dhistory) + } + image.Config = &image.ContainerConfig + return image, nil +} + +// makeDockerV2S1Image builds the best docker image structure we can from the +// contents of the V2S1 image structure. +func makeDockerV2S1Image(manifest docker.V2S1Manifest) (docker.V2Image, error) { + // Treat the most recent (first) item in the history as a description of the image. + if len(manifest.History) == 0 { + return docker.V2Image{}, errors.Errorf("error parsing image configuration from manifest") + } + dimage := docker.V2Image{} + err := json.Unmarshal([]byte(manifest.History[0].V1Compatibility), &dimage) + if err != nil { + return docker.V2Image{}, err + } + if dimage.DockerVersion == "" { + return docker.V2Image{}, errors.Errorf("error parsing image configuration from history") + } + // The DiffID list is intended to contain the sums of _uncompressed_ blobs, and these are most + // likely compressed, so leave the list empty to avoid potential confusion later on. We can + // construct a list with the correct values when we prep layers for pushing, so we don't lose. + // information by leaving this part undone. + rootFS := &docker.V2S2RootFS{ + Type: docker.TypeLayers, + DiffIDs: []digest.Digest{}, + } + // Build a filesystem history. + history := []docker.V2S2History{} + lastID := "" + for i := range manifest.History { + // Decode the compatibility field. + dcompat := docker.V1Compatibility{} + if err = json.Unmarshal([]byte(manifest.History[i].V1Compatibility), &dcompat); err != nil { + return docker.V2Image{}, errors.Errorf("error parsing image compatibility data (%q) from history", manifest.History[i].V1Compatibility) + } + // Skip this history item if it shares the ID of the last one + // that we saw, since the image library will do the same. + if i > 0 && dcompat.ID == lastID { + continue + } + lastID = dcompat.ID + // Construct a new history item using the recovered information. + createdBy := "" + if len(dcompat.ContainerConfig.Cmd) > 0 { + createdBy = strings.Join(dcompat.ContainerConfig.Cmd, " ") + } + h := docker.V2S2History{ + Created: dcompat.Created.UTC(), + Author: dcompat.Author, + CreatedBy: createdBy, + Comment: dcompat.Comment, + EmptyLayer: dcompat.ThrowAway, + } + // Prepend this layer to the list, because a v2s1 format manifest's list is in reverse order + // compared to v2s2, which lists earlier layers before later ones. + history = append([]docker.V2S2History{h}, history...) + } + dimage.RootFS = rootFS + dimage.History = history + return dimage, nil +} + +func (b *Builder) initConfig() { + image := ociv1.Image{} + dimage := docker.V2Image{} + if len(b.Config) > 0 { + // Try to parse the image configuration. If we fail start over from scratch. + if err := json.Unmarshal(b.Config, &dimage); err == nil && dimage.DockerVersion != "" { + if image, err = makeOCIv1Image(&dimage); err != nil { + image = ociv1.Image{} + } + } else { + if err := json.Unmarshal(b.Config, &image); err != nil { + if dimage, err = makeDockerV2S2Image(&image); err != nil { + dimage = docker.V2Image{} + } + } + } + b.OCIv1 = image + b.Docker = dimage + } else { + // Try to dig out the image configuration from the manifest. + manifest := docker.V2S1Manifest{} + if err := json.Unmarshal(b.Manifest, &manifest); err == nil && manifest.SchemaVersion == 1 { + if dimage, err = makeDockerV2S1Image(manifest); err == nil { + if image, err = makeOCIv1Image(&dimage); err != nil { + image = ociv1.Image{} + } + } + } + b.OCIv1 = image + b.Docker = dimage + } + if len(b.Manifest) > 0 { + // Attempt to recover format-specific data from the manifest. + v1Manifest := ociv1.Manifest{} + if json.Unmarshal(b.Manifest, &v1Manifest) == nil { + b.ImageAnnotations = v1Manifest.Annotations + } + } + b.fixupConfig() +} + +func (b *Builder) fixupConfig() { + if b.Docker.Config != nil { + // Prefer image-level settings over those from the container it was built from. + b.Docker.ContainerConfig = *b.Docker.Config + } + b.Docker.Config = &b.Docker.ContainerConfig + b.Docker.DockerVersion = "" + now := time.Now().UTC() + if b.Docker.Created.IsZero() { + b.Docker.Created = now + } + if b.OCIv1.Created == nil || b.OCIv1.Created.IsZero() { + b.OCIv1.Created = &now + } + if b.OS() == "" { + b.SetOS(runtime.GOOS) + } + if b.Architecture() == "" { + b.SetArchitecture(runtime.GOARCH) + } + if b.WorkDir() == "" { + b.SetWorkDir(string(filepath.Separator)) + } +} + +// Annotations returns a set of key-value pairs from the image's manifest. +func (b *Builder) Annotations() map[string]string { + return copyStringStringMap(b.ImageAnnotations) +} + +// SetAnnotation adds or overwrites a key's value from the image's manifest. +// Note: this setting is not present in the Docker v2 image format, so it is +// discarded when writing images using Docker v2 formats. +func (b *Builder) SetAnnotation(key, value string) { + if b.ImageAnnotations == nil { + b.ImageAnnotations = map[string]string{} + } + b.ImageAnnotations[key] = value +} + +// UnsetAnnotation removes a key and its value from the image's manifest, if +// it's present. +func (b *Builder) UnsetAnnotation(key string) { + delete(b.ImageAnnotations, key) +} + +// ClearAnnotations removes all keys and their values from the image's +// manifest. +func (b *Builder) ClearAnnotations() { + b.ImageAnnotations = map[string]string{} +} + +// CreatedBy returns a description of how this image was built. +func (b *Builder) CreatedBy() string { + return b.ImageCreatedBy +} + +// SetCreatedBy sets the description of how this image was built. +func (b *Builder) SetCreatedBy(how string) { + b.ImageCreatedBy = how +} + +// OS returns a name of the OS on which the container, or a container built +// using an image built from this container, is intended to be run. +func (b *Builder) OS() string { + return b.OCIv1.OS +} + +// SetOS sets the name of the OS on which the container, or a container built +// using an image built from this container, is intended to be run. +func (b *Builder) SetOS(os string) { + b.OCIv1.OS = os + b.Docker.OS = os +} + +// Architecture returns a name of the architecture on which the container, or a +// container built using an image built from this container, is intended to be +// run. +func (b *Builder) Architecture() string { + return b.OCIv1.Architecture +} + +// SetArchitecture sets the name of the architecture on which the container, or +// a container built using an image built from this container, is intended to +// be run. +func (b *Builder) SetArchitecture(arch string) { + b.OCIv1.Architecture = arch + b.Docker.Architecture = arch +} + +// Maintainer returns contact information for the person who built the image. +func (b *Builder) Maintainer() string { + return b.OCIv1.Author +} + +// SetMaintainer sets contact information for the person who built the image. +func (b *Builder) SetMaintainer(who string) { + b.OCIv1.Author = who + b.Docker.Author = who +} + +// User returns information about the user as whom the container, or a +// container built using an image built from this container, should be run. +func (b *Builder) User() string { + return b.OCIv1.Config.User +} + +// SetUser sets information about the user as whom the container, or a +// container built using an image built from this container, should be run. +// Acceptable forms are a user name or ID, optionally followed by a colon and a +// group name or ID. +func (b *Builder) SetUser(spec string) { + b.OCIv1.Config.User = spec + b.Docker.Config.User = spec +} + +// WorkDir returns the default working directory for running commands in the +// container, or in a container built using an image built from this container. +func (b *Builder) WorkDir() string { + return b.OCIv1.Config.WorkingDir +} + +// SetWorkDir sets the location of the default working directory for running +// commands in the container, or in a container built using an image built from +// this container. +func (b *Builder) SetWorkDir(there string) { + b.OCIv1.Config.WorkingDir = there + b.Docker.Config.WorkingDir = there +} + +// Shell returns the default shell for running commands in the +// container, or in a container built using an image built from this container. +func (b *Builder) Shell() []string { + return b.Docker.Config.Shell +} + +// SetShell sets the default shell for running +// commands in the container, or in a container built using an image built from +// this container. +// Note: this setting is not present in the OCIv1 image format, so it is +// discarded when writing images using OCIv1 formats. +func (b *Builder) SetShell(shell []string) { + b.Docker.Config.Shell = shell +} + +// Env returns a list of key-value pairs to be set when running commands in the +// container, or in a container built using an image built from this container. +func (b *Builder) Env() []string { + return copyStringSlice(b.OCIv1.Config.Env) +} + +// SetEnv adds or overwrites a value to the set of environment strings which +// should be set when running commands in the container, or in a container +// built using an image built from this container. +func (b *Builder) SetEnv(k string, v string) { + reset := func(s *[]string) { + n := []string{} + for i := range *s { + if !strings.HasPrefix((*s)[i], k+"=") { + n = append(n, (*s)[i]) + } + } + n = append(n, k+"="+v) + *s = n + } + reset(&b.OCIv1.Config.Env) + reset(&b.Docker.Config.Env) +} + +// UnsetEnv removes a value from the set of environment strings which should be +// set when running commands in this container, or in a container built using +// an image built from this container. +func (b *Builder) UnsetEnv(k string) { + unset := func(s *[]string) { + n := []string{} + for i := range *s { + if !strings.HasPrefix((*s)[i], k+"=") { + n = append(n, (*s)[i]) + } + } + *s = n + } + unset(&b.OCIv1.Config.Env) + unset(&b.Docker.Config.Env) +} + +// ClearEnv removes all values from the set of environment strings which should +// be set when running commands in this container, or in a container built +// using an image built from this container. +func (b *Builder) ClearEnv() { + b.OCIv1.Config.Env = []string{} + b.Docker.Config.Env = []string{} +} + +// Cmd returns the default command, or command parameters if an Entrypoint is +// set, to use when running a container built from an image built from this +// container. +func (b *Builder) Cmd() []string { + return copyStringSlice(b.OCIv1.Config.Cmd) +} + +// SetCmd sets the default command, or command parameters if an Entrypoint is +// set, to use when running a container built from an image built from this +// container. +func (b *Builder) SetCmd(cmd []string) { + b.OCIv1.Config.Cmd = copyStringSlice(cmd) + b.Docker.Config.Cmd = copyStringSlice(cmd) +} + +// Entrypoint returns the command to be run for containers built from images +// built from this container. +func (b *Builder) Entrypoint() []string { + return copyStringSlice(b.OCIv1.Config.Entrypoint) +} + +// SetEntrypoint sets the command to be run for in containers built from images +// built from this container. +func (b *Builder) SetEntrypoint(ep []string) { + b.OCIv1.Config.Entrypoint = copyStringSlice(ep) + b.Docker.Config.Entrypoint = copyStringSlice(ep) +} + +// Labels returns a set of key-value pairs from the image's runtime +// configuration. +func (b *Builder) Labels() map[string]string { + return copyStringStringMap(b.OCIv1.Config.Labels) +} + +// SetLabel adds or overwrites a key's value from the image's runtime +// configuration. +func (b *Builder) SetLabel(k string, v string) { + if b.OCIv1.Config.Labels == nil { + b.OCIv1.Config.Labels = map[string]string{} + } + b.OCIv1.Config.Labels[k] = v + if b.Docker.Config.Labels == nil { + b.Docker.Config.Labels = map[string]string{} + } + b.Docker.Config.Labels[k] = v +} + +// UnsetLabel removes a key and its value from the image's runtime +// configuration, if it's present. +func (b *Builder) UnsetLabel(k string) { + delete(b.OCIv1.Config.Labels, k) + delete(b.Docker.Config.Labels, k) +} + +// ClearLabels removes all keys and their values from the image's runtime +// configuration. +func (b *Builder) ClearLabels() { + b.OCIv1.Config.Labels = map[string]string{} + b.Docker.Config.Labels = map[string]string{} +} + +// Ports returns the set of ports which should be exposed when a container +// based on an image built from this container is run. +func (b *Builder) Ports() []string { + p := []string{} + for k := range b.OCIv1.Config.ExposedPorts { + p = append(p, k) + } + return p +} + +// SetPort adds or overwrites an exported port in the set of ports which should +// be exposed when a container based on an image built from this container is +// run. +func (b *Builder) SetPort(p string) { + if b.OCIv1.Config.ExposedPorts == nil { + b.OCIv1.Config.ExposedPorts = map[string]struct{}{} + } + b.OCIv1.Config.ExposedPorts[p] = struct{}{} + if b.Docker.Config.ExposedPorts == nil { + b.Docker.Config.ExposedPorts = make(docker.PortSet) + } + b.Docker.Config.ExposedPorts[docker.Port(p)] = struct{}{} +} + +// UnsetPort removes an exposed port from the set of ports which should be +// exposed when a container based on an image built from this container is run. +func (b *Builder) UnsetPort(p string) { + delete(b.OCIv1.Config.ExposedPorts, p) + delete(b.Docker.Config.ExposedPorts, docker.Port(p)) +} + +// ClearPorts empties the set of ports which should be exposed when a container +// based on an image built from this container is run. +func (b *Builder) ClearPorts() { + b.OCIv1.Config.ExposedPorts = map[string]struct{}{} + b.Docker.Config.ExposedPorts = docker.PortSet{} +} + +// Volumes returns a list of filesystem locations which should be mounted from +// outside of the container when a container built from an image built from +// this container is run. +func (b *Builder) Volumes() []string { + v := []string{} + for k := range b.OCIv1.Config.Volumes { + v = append(v, k) + } + return v +} + +// AddVolume adds a location to the image's list of locations which should be +// mounted from outside of the container when a container based on an image +// built from this container is run. +func (b *Builder) AddVolume(v string) { + if b.OCIv1.Config.Volumes == nil { + b.OCIv1.Config.Volumes = map[string]struct{}{} + } + b.OCIv1.Config.Volumes[v] = struct{}{} + if b.Docker.Config.Volumes == nil { + b.Docker.Config.Volumes = map[string]struct{}{} + } + b.Docker.Config.Volumes[v] = struct{}{} +} + +// RemoveVolume removes a location from the list of locations which should be +// mounted from outside of the container when a container based on an image +// built from this container is run. +func (b *Builder) RemoveVolume(v string) { + delete(b.OCIv1.Config.Volumes, v) + delete(b.Docker.Config.Volumes, v) +} + +// ClearVolumes removes all locations from the image's list of locations which +// should be mounted from outside of the container when a container based on an +// image built from this container is run. +func (b *Builder) ClearVolumes() { + b.OCIv1.Config.Volumes = map[string]struct{}{} + b.Docker.Config.Volumes = map[string]struct{}{} +} + +// Hostname returns the hostname which will be set in the container and in +// containers built using images built from the container. +func (b *Builder) Hostname() string { + return b.Docker.Config.Hostname +} + +// SetHostname sets the hostname which will be set in the container and in +// containers built using images built from the container. +// Note: this setting is not present in the OCIv1 image format, so it is +// discarded when writing images using OCIv1 formats. +func (b *Builder) SetHostname(name string) { + b.Docker.Config.Hostname = name +} + +// Domainname returns the domainname which will be set in the container and in +// containers built using images built from the container. +func (b *Builder) Domainname() string { + return b.Docker.Config.Domainname +} + +// SetDomainname sets the domainname which will be set in the container and in +// containers built using images built from the container. +// Note: this setting is not present in the OCIv1 image format, so it is +// discarded when writing images using OCIv1 formats. +func (b *Builder) SetDomainname(name string) { + b.Docker.Config.Domainname = name +} + +// SetDefaultMountsFilePath sets the mounts file path for testing purposes +func (b *Builder) SetDefaultMountsFilePath(path string) { + b.DefaultMountsFilePath = path +} + +// Comment returns the comment which will be set in the container and in +// containers built using images built from the container +func (b *Builder) Comment() string { + return b.Docker.Comment +} + +// SetComment sets the Comment which will be set in the container and in +// containers built using images built from the container. +// Note: this setting is not present in the OCIv1 image format, so it is +// discarded when writing images using OCIv1 formats. +func (b *Builder) SetComment(comment string) { + b.Docker.Comment = comment +} + +// StopSignal returns the signal which will be set in the container and in +// containers built using images buiilt from the container +func (b *Builder) StopSignal() string { + return b.Docker.Config.StopSignal +} + +// SetStopSignal sets the signal which will be set in the container and in +// containers built using images built from the container. +func (b *Builder) SetStopSignal(stopSignal string) { + b.OCIv1.Config.StopSignal = stopSignal + b.Docker.Config.StopSignal = stopSignal +} diff --git a/vendor/github.com/projectatomic/buildah/delete.go b/vendor/github.com/projectatomic/buildah/delete.go new file mode 100644 index 000000000..8de774ff9 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/delete.go @@ -0,0 +1,18 @@ +package buildah + +import ( + "github.com/opencontainers/selinux/go-selinux/label" + "github.com/pkg/errors" +) + +// Delete removes the working container. The buildah.Builder object should not +// be used after this method is called. +func (b *Builder) Delete() error { + if err := b.store.DeleteContainer(b.ContainerID); err != nil { + return errors.Wrapf(err, "error deleting build container") + } + b.MountPoint = "" + b.Container = "" + b.ContainerID = "" + return label.ReleaseLabel(b.ProcessLabel) +} diff --git a/vendor/github.com/projectatomic/buildah/docker/types.go b/vendor/github.com/projectatomic/buildah/docker/types.go new file mode 100644 index 000000000..9890eaf93 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/docker/types.go @@ -0,0 +1,271 @@ +package docker + +// +// Types extracted from Docker +// + +import ( + "time" + + "github.com/containers/image/pkg/strslice" + "github.com/opencontainers/go-digest" +) + +// github.com/moby/moby/image/rootfs.go +const TypeLayers = "layers" + +// github.com/docker/distribution/manifest/schema2/manifest.go +const V2S2MediaTypeManifest = "application/vnd.docker.distribution.manifest.v2+json" + +// github.com/docker/distribution/manifest/schema2/manifest.go +const V2S2MediaTypeImageConfig = "application/vnd.docker.container.image.v1+json" + +// github.com/docker/distribution/manifest/schema2/manifest.go +const V2S2MediaTypeLayer = "application/vnd.docker.image.rootfs.diff.tar.gzip" + +// github.com/docker/distribution/manifest/schema2/manifest.go +const V2S2MediaTypeUncompressedLayer = "application/vnd.docker.image.rootfs.diff.tar" + +// github.com/moby/moby/image/rootfs.go +// RootFS describes images root filesystem +// This is currently a placeholder that only supports layers. In the future +// this can be made into an interface that supports different implementations. +type V2S2RootFS struct { + Type string `json:"type"` + DiffIDs []digest.Digest `json:"diff_ids,omitempty"` +} + +// github.com/moby/moby/image/image.go +// History stores build commands that were used to create an image +type V2S2History struct { + // Created is the timestamp at which the image was created + Created time.Time `json:"created"` + // Author is the name of the author that was specified when committing the image + Author string `json:"author,omitempty"` + // CreatedBy keeps the Dockerfile command used while building the image + CreatedBy string `json:"created_by,omitempty"` + // Comment is the commit message that was set when committing the image + Comment string `json:"comment,omitempty"` + // EmptyLayer is set to true if this history item did not generate a + // layer. Otherwise, the history item is associated with the next + // layer in the RootFS section. + EmptyLayer bool `json:"empty_layer,omitempty"` +} + +// github.com/moby/moby/image/image.go +// ID is the content-addressable ID of an image. +type ID digest.Digest + +// github.com/moby/moby/api/types/container/config.go +// HealthConfig holds configuration settings for the HEALTHCHECK feature. +type HealthConfig struct { + // Test is the test to perform to check that the container is healthy. + // An empty slice means to inherit the default. + // The options are: + // {} : inherit healthcheck + // {"NONE"} : disable healthcheck + // {"CMD", args...} : exec arguments directly + // {"CMD-SHELL", command} : run command with system's default shell + Test []string `json:",omitempty"` + + // Zero means to inherit. Durations are expressed as integer nanoseconds. + Interval time.Duration `json:",omitempty"` // Interval is the time to wait between checks. + Timeout time.Duration `json:",omitempty"` // Timeout is the time to wait before considering the check to have hung. + + // Retries is the number of consecutive failures needed to consider a container as unhealthy. + // Zero means inherit. + Retries int `json:",omitempty"` +} + +// github.com/docker/go-connections/nat/nat.go +// PortSet is a collection of structs indexed by Port +type PortSet map[Port]struct{} + +// github.com/docker/go-connections/nat/nat.go +// Port is a string containing port number and protocol in the format "80/tcp" +type Port string + +// github.com/moby/moby/api/types/container/config.go +// Config contains the configuration data about a container. +// It should hold only portable information about the container. +// Here, "portable" means "independent from the host we are running on". +// Non-portable information *should* appear in HostConfig. +// All fields added to this struct must be marked `omitempty` to keep getting +// predictable hashes from the old `v1Compatibility` configuration. +type Config struct { + Hostname string // Hostname + Domainname string // Domainname + User string // User that will run the command(s) inside the container, also support user:group + AttachStdin bool // Attach the standard input, makes possible user interaction + AttachStdout bool // Attach the standard output + AttachStderr bool // Attach the standard error + ExposedPorts PortSet `json:",omitempty"` // List of exposed ports + Tty bool // Attach standard streams to a tty, including stdin if it is not closed. + OpenStdin bool // Open stdin + StdinOnce bool // If true, close stdin after the 1 attached client disconnects. + Env []string // List of environment variable to set in the container + Cmd strslice.StrSlice // Command to run when starting the container + Healthcheck *HealthConfig `json:",omitempty"` // Healthcheck describes how to check the container is healthy + ArgsEscaped bool `json:",omitempty"` // True if command is already escaped (Windows specific) + Image string // Name of the image as it was passed by the operator (e.g. could be symbolic) + Volumes map[string]struct{} // List of volumes (mounts) used for the container + WorkingDir string // Current directory (PWD) in the command will be launched + Entrypoint strslice.StrSlice // Entrypoint to run when starting the container + NetworkDisabled bool `json:",omitempty"` // Is network disabled + MacAddress string `json:",omitempty"` // Mac Address of the container + OnBuild []string // ONBUILD metadata that were defined on the image Dockerfile + Labels map[string]string // List of labels set to this container + StopSignal string `json:",omitempty"` // Signal to stop a container + StopTimeout *int `json:",omitempty"` // Timeout (in seconds) to stop a container + Shell strslice.StrSlice `json:",omitempty"` // Shell for shell-form of RUN, CMD, ENTRYPOINT +} + +// github.com/docker/distribution/manifest/schema1/config_builder.go +// For non-top-level layers, create fake V1Compatibility strings that +// fit the format and don't collide with anything else, but don't +// result in runnable images on their own. +type V1Compatibility struct { + ID string `json:"id"` + Parent string `json:"parent,omitempty"` + Comment string `json:"comment,omitempty"` + Created time.Time `json:"created"` + ContainerConfig struct { + Cmd []string + } `json:"container_config,omitempty"` + Author string `json:"author,omitempty"` + ThrowAway bool `json:"throwaway,omitempty"` +} + +// github.com/moby/moby/image/image.go +// V1Image stores the V1 image configuration. +type V1Image struct { + // ID is a unique 64 character identifier of the image + ID string `json:"id,omitempty"` + // Parent is the ID of the parent image + Parent string `json:"parent,omitempty"` + // Comment is the commit message that was set when committing the image + Comment string `json:"comment,omitempty"` + // Created is the timestamp at which the image was created + Created time.Time `json:"created"` + // Container is the id of the container used to commit + Container string `json:"container,omitempty"` + // ContainerConfig is the configuration of the container that is committed into the image + ContainerConfig Config `json:"container_config,omitempty"` + // DockerVersion specifies the version of Docker that was used to build the image + DockerVersion string `json:"docker_version,omitempty"` + // Author is the name of the author that was specified when committing the image + Author string `json:"author,omitempty"` + // Config is the configuration of the container received from the client + Config *Config `json:"config,omitempty"` + // Architecture is the hardware that the image is build and runs on + Architecture string `json:"architecture,omitempty"` + // OS is the operating system used to build and run the image + OS string `json:"os,omitempty"` + // Size is the total size of the image including all layers it is composed of + Size int64 `json:",omitempty"` +} + +// github.com/moby/moby/image/image.go +// Image stores the image configuration +type V2Image struct { + V1Image + Parent ID `json:"parent,omitempty"` + RootFS *V2S2RootFS `json:"rootfs,omitempty"` + History []V2S2History `json:"history,omitempty"` + OSVersion string `json:"os.version,omitempty"` + OSFeatures []string `json:"os.features,omitempty"` + + // rawJSON caches the immutable JSON associated with this image. + rawJSON []byte + + // computedID is the ID computed from the hash of the image config. + // Not to be confused with the legacy V1 ID in V1Image. + computedID ID +} + +// github.com/docker/distribution/manifest/versioned.go +// Versioned provides a struct with the manifest schemaVersion and mediaType. +// Incoming content with unknown schema version can be decoded against this +// struct to check the version. +type V2Versioned struct { + // SchemaVersion is the image manifest schema that this image follows + SchemaVersion int `json:"schemaVersion"` + + // MediaType is the media type of this schema. + MediaType string `json:"mediaType,omitempty"` +} + +// github.com/docker/distribution/manifest/schema1/manifest.go +// FSLayer is a container struct for BlobSums defined in an image manifest +type V2S1FSLayer struct { + // BlobSum is the tarsum of the referenced filesystem image layer + BlobSum digest.Digest `json:"blobSum"` +} + +// github.com/docker/distribution/manifest/schema1/manifest.go +// History stores unstructured v1 compatibility information +type V2S1History struct { + // V1Compatibility is the raw v1 compatibility information + V1Compatibility string `json:"v1Compatibility"` +} + +// github.com/docker/distribution/manifest/schema1/manifest.go +// Manifest provides the base accessible fields for working with V2 image +// format in the registry. +type V2S1Manifest struct { + V2Versioned + + // Name is the name of the image's repository + Name string `json:"name"` + + // Tag is the tag of the image specified by this manifest + Tag string `json:"tag"` + + // Architecture is the host architecture on which this image is intended to + // run + Architecture string `json:"architecture"` + + // FSLayers is a list of filesystem layer blobSums contained in this image + FSLayers []V2S1FSLayer `json:"fsLayers"` + + // History is a list of unstructured historical data for v1 compatibility + History []V2S1History `json:"history"` +} + +// github.com/docker/distribution/blobs.go +// Descriptor describes targeted content. Used in conjunction with a blob +// store, a descriptor can be used to fetch, store and target any kind of +// blob. The struct also describes the wire protocol format. Fields should +// only be added but never changed. +type V2S2Descriptor struct { + // MediaType describe the type of the content. All text based formats are + // encoded as utf-8. + MediaType string `json:"mediaType,omitempty"` + + // Size in bytes of content. + Size int64 `json:"size,omitempty"` + + // Digest uniquely identifies the content. A byte stream can be verified + // against against this digest. + Digest digest.Digest `json:"digest,omitempty"` + + // URLs contains the source URLs of this content. + URLs []string `json:"urls,omitempty"` + + // NOTE: Before adding a field here, please ensure that all + // other options have been exhausted. Much of the type relationships + // depend on the simplicity of this type. +} + +// github.com/docker/distribution/manifest/schema2/manifest.go +// Manifest defines a schema2 manifest. +type V2S2Manifest struct { + V2Versioned + + // Config references the image configuration as a blob. + Config V2S2Descriptor `json:"config"` + + // Layers lists descriptors for the layers referenced by the + // configuration. + Layers []V2S2Descriptor `json:"layers"` +} diff --git a/vendor/github.com/projectatomic/buildah/image.go b/vendor/github.com/projectatomic/buildah/image.go new file mode 100644 index 000000000..e5a49f1f9 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/image.go @@ -0,0 +1,529 @@ +package buildah + +import ( + "bytes" + "context" + "encoding/json" + "io" + "io/ioutil" + "os" + "path/filepath" + "time" + + "github.com/containers/image/docker/reference" + "github.com/containers/image/image" + is "github.com/containers/image/storage" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/containers/storage/pkg/archive" + "github.com/containers/storage/pkg/ioutils" + digest "github.com/opencontainers/go-digest" + specs "github.com/opencontainers/image-spec/specs-go" + "github.com/opencontainers/image-spec/specs-go/v1" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/docker" + "github.com/sirupsen/logrus" +) + +const ( + // OCIv1ImageManifest is the MIME type of an OCIv1 image manifest, + // suitable for specifying as a value of the PreferredManifestType + // member of a CommitOptions structure. It is also the default. + OCIv1ImageManifest = v1.MediaTypeImageManifest + // Dockerv2ImageManifest is the MIME type of a Docker v2s2 image + // manifest, suitable for specifying as a value of the + // PreferredManifestType member of a CommitOptions structure. + Dockerv2ImageManifest = docker.V2S2MediaTypeManifest +) + +type containerImageRef struct { + store storage.Store + compression archive.Compression + name reference.Named + names []string + layerID string + oconfig []byte + dconfig []byte + created time.Time + createdBy string + annotations map[string]string + preferredManifestType string + exporting bool +} + +type containerImageSource struct { + path string + ref *containerImageRef + store storage.Store + layerID string + names []string + compression archive.Compression + config []byte + configDigest digest.Digest + manifest []byte + manifestType string + exporting bool +} + +func (i *containerImageRef) NewImage(ctx context.Context, sc *types.SystemContext) (types.ImageCloser, error) { + src, err := i.NewImageSource(ctx, sc) + if err != nil { + return nil, err + } + return image.FromSource(ctx, sc, src) +} + +func expectedOCIDiffIDs(image v1.Image) int { + expected := 0 + for _, history := range image.History { + if !history.EmptyLayer { + expected = expected + 1 + } + } + return expected +} + +func expectedDockerDiffIDs(image docker.V2Image) int { + expected := 0 + for _, history := range image.History { + if !history.EmptyLayer { + expected = expected + 1 + } + } + return expected +} + +func (i *containerImageRef) NewImageSource(ctx context.Context, sc *types.SystemContext) (src types.ImageSource, err error) { + // Decide which type of manifest and configuration output we're going to provide. + manifestType := i.preferredManifestType + // If it's not a format we support, return an error. + if manifestType != v1.MediaTypeImageManifest && manifestType != docker.V2S2MediaTypeManifest { + return nil, errors.Errorf("no supported manifest types (attempted to use %q, only know %q and %q)", + manifestType, v1.MediaTypeImageManifest, docker.V2S2MediaTypeManifest) + } + // Start building the list of layers using the read-write layer. + layers := []string{} + layerID := i.layerID + layer, err := i.store.Layer(layerID) + if err != nil { + return nil, errors.Wrapf(err, "unable to read layer %q", layerID) + } + // Walk the list of parent layers, prepending each as we go. + for layer != nil { + layers = append(append([]string{}, layerID), layers...) + layerID = layer.Parent + if layerID == "" { + err = nil + break + } + layer, err = i.store.Layer(layerID) + if err != nil { + return nil, errors.Wrapf(err, "unable to read layer %q", layerID) + } + } + logrus.Debugf("layer list: %q", layers) + + // Make a temporary directory to hold blobs. + path, err := ioutil.TempDir(os.TempDir(), Package) + if err != nil { + return nil, err + } + logrus.Debugf("using %q to hold temporary data", path) + defer func() { + if src == nil { + err2 := os.RemoveAll(path) + if err2 != nil { + logrus.Errorf("error removing %q: %v", path, err) + } + } + }() + + // Build fresh copies of the configurations so that we don't mess with the values in the Builder + // object itself. + oimage := v1.Image{} + err = json.Unmarshal(i.oconfig, &oimage) + if err != nil { + return nil, err + } + created := i.created + oimage.Created = &created + dimage := docker.V2Image{} + err = json.Unmarshal(i.dconfig, &dimage) + if err != nil { + return nil, err + } + dimage.Created = created + + // Start building manifests. + omanifest := v1.Manifest{ + Versioned: specs.Versioned{ + SchemaVersion: 2, + }, + Config: v1.Descriptor{ + MediaType: v1.MediaTypeImageConfig, + }, + Layers: []v1.Descriptor{}, + Annotations: i.annotations, + } + dmanifest := docker.V2S2Manifest{ + V2Versioned: docker.V2Versioned{ + SchemaVersion: 2, + MediaType: docker.V2S2MediaTypeManifest, + }, + Config: docker.V2S2Descriptor{ + MediaType: docker.V2S2MediaTypeImageConfig, + }, + Layers: []docker.V2S2Descriptor{}, + } + + oimage.RootFS.Type = docker.TypeLayers + oimage.RootFS.DiffIDs = []digest.Digest{} + dimage.RootFS = &docker.V2S2RootFS{} + dimage.RootFS.Type = docker.TypeLayers + dimage.RootFS.DiffIDs = []digest.Digest{} + + // Extract each layer and compute its digests, both compressed (if requested) and uncompressed. + for _, layerID := range layers { + // The default layer media type assumes no compression. + omediaType := v1.MediaTypeImageLayer + dmediaType := docker.V2S2MediaTypeUncompressedLayer + // If we're not re-exporting the data, reuse the blobsum and diff IDs. + if !i.exporting && layerID != i.layerID { + layer, err2 := i.store.Layer(layerID) + if err2 != nil { + return nil, errors.Wrapf(err, "unable to locate layer %q", layerID) + } + if layer.UncompressedDigest == "" { + return nil, errors.Errorf("unable to look up size of layer %q", layerID) + } + layerBlobSum := layer.UncompressedDigest + layerBlobSize := layer.UncompressedSize + // Note this layer in the manifest, using the uncompressed blobsum. + olayerDescriptor := v1.Descriptor{ + MediaType: omediaType, + Digest: layerBlobSum, + Size: layerBlobSize, + } + omanifest.Layers = append(omanifest.Layers, olayerDescriptor) + dlayerDescriptor := docker.V2S2Descriptor{ + MediaType: dmediaType, + Digest: layerBlobSum, + Size: layerBlobSize, + } + dmanifest.Layers = append(dmanifest.Layers, dlayerDescriptor) + // Note this layer in the list of diffIDs, again using the uncompressed blobsum. + oimage.RootFS.DiffIDs = append(oimage.RootFS.DiffIDs, layerBlobSum) + dimage.RootFS.DiffIDs = append(dimage.RootFS.DiffIDs, layerBlobSum) + continue + } + // Figure out if we need to change the media type, in case we're using compression. + if i.compression != archive.Uncompressed { + switch i.compression { + case archive.Gzip: + omediaType = v1.MediaTypeImageLayerGzip + dmediaType = docker.V2S2MediaTypeLayer + logrus.Debugf("compressing layer %q with gzip", layerID) + case archive.Bzip2: + // Until the image specs define a media type for bzip2-compressed layers, even if we know + // how to decompress them, we can't try to compress layers with bzip2. + return nil, errors.New("media type for bzip2-compressed layers is not defined") + case archive.Xz: + // Until the image specs define a media type for xz-compressed layers, even if we know + // how to decompress them, we can't try to compress layers with xz. + return nil, errors.New("media type for xz-compressed layers is not defined") + default: + logrus.Debugf("compressing layer %q with unknown compressor(?)", layerID) + } + } + // Start reading the layer. + noCompression := archive.Uncompressed + diffOptions := &storage.DiffOptions{ + Compression: &noCompression, + } + rc, err := i.store.Diff("", layerID, diffOptions) + if err != nil { + return nil, errors.Wrapf(err, "error extracting layer %q", layerID) + } + defer rc.Close() + srcHasher := digest.Canonical.Digester() + reader := io.TeeReader(rc, srcHasher.Hash()) + // Set up to write the possibly-recompressed blob. + layerFile, err := os.OpenFile(filepath.Join(path, "layer"), os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + return nil, errors.Wrapf(err, "error opening file for layer %q", layerID) + } + destHasher := digest.Canonical.Digester() + counter := ioutils.NewWriteCounter(layerFile) + multiWriter := io.MultiWriter(counter, destHasher.Hash()) + // Compress the layer, if we're recompressing it. + writer, err := archive.CompressStream(multiWriter, i.compression) + if err != nil { + return nil, errors.Wrapf(err, "error compressing layer %q", layerID) + } + size, err := io.Copy(writer, reader) + if err != nil { + return nil, errors.Wrapf(err, "error storing layer %q to file", layerID) + } + writer.Close() + layerFile.Close() + if i.compression == archive.Uncompressed { + if size != counter.Count { + return nil, errors.Errorf("error storing layer %q to file: inconsistent layer size (copied %d, wrote %d)", layerID, size, counter.Count) + } + } else { + size = counter.Count + } + logrus.Debugf("layer %q size is %d bytes", layerID, size) + // Rename the layer so that we can more easily find it by digest later. + err = os.Rename(filepath.Join(path, "layer"), filepath.Join(path, destHasher.Digest().String())) + if err != nil { + return nil, errors.Wrapf(err, "error storing layer %q to file", layerID) + } + // Add a note in the manifest about the layer. The blobs are identified by their possibly- + // compressed blob digests. + olayerDescriptor := v1.Descriptor{ + MediaType: omediaType, + Digest: destHasher.Digest(), + Size: size, + } + omanifest.Layers = append(omanifest.Layers, olayerDescriptor) + dlayerDescriptor := docker.V2S2Descriptor{ + MediaType: dmediaType, + Digest: destHasher.Digest(), + Size: size, + } + dmanifest.Layers = append(dmanifest.Layers, dlayerDescriptor) + // Add a note about the diffID, which is always the layer's uncompressed digest. + oimage.RootFS.DiffIDs = append(oimage.RootFS.DiffIDs, srcHasher.Digest()) + dimage.RootFS.DiffIDs = append(dimage.RootFS.DiffIDs, srcHasher.Digest()) + } + + // Build history notes in the image configurations. + onews := v1.History{ + Created: &i.created, + CreatedBy: i.createdBy, + Author: oimage.Author, + EmptyLayer: false, + } + oimage.History = append(oimage.History, onews) + dnews := docker.V2S2History{ + Created: i.created, + CreatedBy: i.createdBy, + Author: dimage.Author, + EmptyLayer: false, + } + dimage.History = append(dimage.History, dnews) + + // Sanity check that we didn't just create a mismatch between non-empty layers in the + // history and the number of diffIDs. + expectedDiffIDs := expectedOCIDiffIDs(oimage) + if len(oimage.RootFS.DiffIDs) != expectedDiffIDs { + return nil, errors.Errorf("internal error: history lists %d non-empty layers, but we have %d layers on disk", expectedDiffIDs, len(oimage.RootFS.DiffIDs)) + } + expectedDiffIDs = expectedDockerDiffIDs(dimage) + if len(dimage.RootFS.DiffIDs) != expectedDiffIDs { + return nil, errors.Errorf("internal error: history lists %d non-empty layers, but we have %d layers on disk", expectedDiffIDs, len(dimage.RootFS.DiffIDs)) + } + + // Encode the image configuration blob. + oconfig, err := json.Marshal(&oimage) + if err != nil { + return nil, err + } + logrus.Debugf("OCIv1 config = %s", oconfig) + + // Add the configuration blob to the manifest. + omanifest.Config.Digest = digest.Canonical.FromBytes(oconfig) + omanifest.Config.Size = int64(len(oconfig)) + omanifest.Config.MediaType = v1.MediaTypeImageConfig + + // Encode the manifest. + omanifestbytes, err := json.Marshal(&omanifest) + if err != nil { + return nil, err + } + logrus.Debugf("OCIv1 manifest = %s", omanifestbytes) + + // Encode the image configuration blob. + dconfig, err := json.Marshal(&dimage) + if err != nil { + return nil, err + } + logrus.Debugf("Docker v2s2 config = %s", dconfig) + + // Add the configuration blob to the manifest. + dmanifest.Config.Digest = digest.Canonical.FromBytes(dconfig) + dmanifest.Config.Size = int64(len(dconfig)) + dmanifest.Config.MediaType = docker.V2S2MediaTypeImageConfig + + // Encode the manifest. + dmanifestbytes, err := json.Marshal(&dmanifest) + if err != nil { + return nil, err + } + logrus.Debugf("Docker v2s2 manifest = %s", dmanifestbytes) + + // Decide which manifest and configuration blobs we'll actually output. + var config []byte + var manifest []byte + switch manifestType { + case v1.MediaTypeImageManifest: + manifest = omanifestbytes + config = oconfig + case docker.V2S2MediaTypeManifest: + manifest = dmanifestbytes + config = dconfig + default: + panic("unreachable code: unsupported manifest type") + } + src = &containerImageSource{ + path: path, + ref: i, + store: i.store, + layerID: i.layerID, + names: i.names, + compression: i.compression, + config: config, + configDigest: digest.Canonical.FromBytes(config), + manifest: manifest, + manifestType: manifestType, + exporting: i.exporting, + } + return src, nil +} + +func (i *containerImageRef) NewImageDestination(ctx context.Context, sc *types.SystemContext) (types.ImageDestination, error) { + return nil, errors.Errorf("can't write to a container") +} + +func (i *containerImageRef) DockerReference() reference.Named { + return i.name +} + +func (i *containerImageRef) StringWithinTransport() string { + if len(i.names) > 0 { + return i.names[0] + } + return "" +} + +func (i *containerImageRef) DeleteImage(context.Context, *types.SystemContext) error { + // we were never here + return nil +} + +func (i *containerImageRef) PolicyConfigurationIdentity() string { + return "" +} + +func (i *containerImageRef) PolicyConfigurationNamespaces() []string { + return nil +} + +func (i *containerImageRef) Transport() types.ImageTransport { + return is.Transport +} + +func (i *containerImageSource) Close() error { + err := os.RemoveAll(i.path) + if err != nil { + logrus.Errorf("error removing %q: %v", i.path, err) + } + return err +} + +func (i *containerImageSource) Reference() types.ImageReference { + return i.ref +} + +func (i *containerImageSource) GetSignatures(ctx context.Context, instanceDigest *digest.Digest) ([][]byte, error) { + if instanceDigest != nil && *instanceDigest != digest.FromBytes(i.manifest) { + return nil, errors.Errorf("TODO") + } + return nil, nil +} + +func (i *containerImageSource) GetManifest(ctx context.Context, instanceDigest *digest.Digest) ([]byte, string, error) { + if instanceDigest != nil && *instanceDigest != digest.FromBytes(i.manifest) { + return nil, "", errors.Errorf("TODO") + } + return i.manifest, i.manifestType, nil +} + +func (i *containerImageSource) LayerInfosForCopy(ctx context.Context) ([]types.BlobInfo, error) { + return nil, nil +} + +func (i *containerImageSource) GetBlob(ctx context.Context, blob types.BlobInfo) (reader io.ReadCloser, size int64, err error) { + if blob.Digest == i.configDigest { + logrus.Debugf("start reading config") + reader := bytes.NewReader(i.config) + closer := func() error { + logrus.Debugf("finished reading config") + return nil + } + return ioutils.NewReadCloserWrapper(reader, closer), reader.Size(), nil + } + layerFile, err := os.OpenFile(filepath.Join(i.path, blob.Digest.String()), os.O_RDONLY, 0600) + if err != nil { + logrus.Debugf("error reading layer %q: %v", blob.Digest.String(), err) + return nil, -1, err + } + size = -1 + st, err := layerFile.Stat() + if err != nil { + logrus.Warnf("error reading size of layer %q: %v", blob.Digest.String(), err) + } else { + size = st.Size() + } + logrus.Debugf("reading layer %q", blob.Digest.String()) + closer := func() error { + layerFile.Close() + logrus.Debugf("finished reading layer %q", blob.Digest.String()) + return nil + } + return ioutils.NewReadCloserWrapper(layerFile, closer), size, nil +} + +func (b *Builder) makeImageRef(manifestType string, exporting bool, compress archive.Compression, historyTimestamp *time.Time) (types.ImageReference, error) { + var name reference.Named + container, err := b.store.Container(b.ContainerID) + if err != nil { + return nil, errors.Wrapf(err, "error locating container %q", b.ContainerID) + } + if len(container.Names) > 0 { + if parsed, err2 := reference.ParseNamed(container.Names[0]); err2 == nil { + name = parsed + } + } + if manifestType == "" { + manifestType = OCIv1ImageManifest + } + oconfig, err := json.Marshal(&b.OCIv1) + if err != nil { + return nil, errors.Wrapf(err, "error encoding OCI-format image configuration") + } + dconfig, err := json.Marshal(&b.Docker) + if err != nil { + return nil, errors.Wrapf(err, "error encoding docker-format image configuration") + } + created := time.Now().UTC() + if historyTimestamp != nil { + created = historyTimestamp.UTC() + } + ref := &containerImageRef{ + store: b.store, + compression: compress, + name: name, + names: container.Names, + layerID: container.LayerID, + oconfig: oconfig, + dconfig: dconfig, + created: created, + createdBy: b.CreatedBy(), + annotations: b.Annotations(), + preferredManifestType: manifestType, + exporting: exporting, + } + return ref, nil +} diff --git a/vendor/github.com/projectatomic/buildah/imagebuildah/build.go b/vendor/github.com/projectatomic/buildah/imagebuildah/build.go new file mode 100644 index 000000000..c477e0996 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/imagebuildah/build.go @@ -0,0 +1,775 @@ +package imagebuildah + +import ( + "context" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strconv" + "strings" + + is "github.com/containers/image/storage" + "github.com/containers/image/transports" + "github.com/containers/image/transports/alltransports" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/containers/storage/pkg/archive" + "github.com/containers/storage/pkg/stringid" + "github.com/docker/docker/builder/dockerfile/parser" + docker "github.com/fsouza/go-dockerclient" + "github.com/opencontainers/runtime-spec/specs-go" + "github.com/openshift/imagebuilder" + "github.com/pkg/errors" + "github.com/projectatomic/buildah" + "github.com/sirupsen/logrus" +) + +const ( + PullIfMissing = buildah.PullIfMissing + PullAlways = buildah.PullAlways + PullNever = buildah.PullNever + DefaultRuntime = buildah.DefaultRuntime + OCIv1ImageFormat = buildah.OCIv1ImageManifest + Dockerv2ImageFormat = buildah.Dockerv2ImageManifest + + Gzip = archive.Gzip + Bzip2 = archive.Bzip2 + Xz = archive.Xz + Uncompressed = archive.Uncompressed +) + +// Mount is a mountpoint for the build container. +type Mount specs.Mount + +// BuildOptions can be used to alter how an image is built. +type BuildOptions struct { + // ContextDirectory is the default source location for COPY and ADD + // commands. + ContextDirectory string + // PullPolicy controls whether or not we pull images. It should be one + // of PullIfMissing, PullAlways, or PullNever. + PullPolicy int + // Registry is a value which is prepended to the image's name, if it + // needs to be pulled and the image name alone can not be resolved to a + // reference to a source image. No separator is implicitly added. + Registry string + // Transport is a value which is prepended to the image's name, if it + // needs to be pulled and the image name alone, or the image name and + // the registry together, can not be resolved to a reference to a + // source image. No separator is implicitly added. + Transport string + // IgnoreUnrecognizedInstructions tells us to just log instructions we + // don't recognize, and try to keep going. + IgnoreUnrecognizedInstructions bool + // Quiet tells us whether or not to announce steps as we go through them. + Quiet bool + // Runtime is the name of the command to run for RUN instructions. It + // should accept the same arguments and flags that runc does. + Runtime string + // RuntimeArgs adds global arguments for the runtime. + RuntimeArgs []string + // TransientMounts is a list of mounts that won't be kept in the image. + TransientMounts []Mount + // Compression specifies the type of compression which is applied to + // layer blobs. The default is to not use compression, but + // archive.Gzip is recommended. + Compression archive.Compression + // Arguments which can be interpolated into Dockerfiles + Args map[string]string + // Name of the image to write to. + Output string + // Additional tags to add to the image that we write, if we know of a + // way to add them. + AdditionalTags []string + // Log is a callback that will print a progress message. If no value + // is supplied, the message will be sent to Err (or os.Stderr, if Err + // is nil) by default. + Log func(format string, args ...interface{}) + // Out is a place where non-error log messages are sent. + Out io.Writer + // Err is a place where error log messages should be sent. + Err io.Writer + // SignaturePolicyPath specifies an override location for the signature + // policy which should be used for verifying the new image as it is + // being written. Except in specific circumstances, no value should be + // specified, indicating that the shared, system-wide default policy + // should be used. + SignaturePolicyPath string + // ReportWriter is an io.Writer which will be used to report the + // progress of the (possible) pulling of the source image and the + // writing of the new image. + ReportWriter io.Writer + // OutputFormat is the format of the output image's manifest and + // configuration data. + // Accepted values are OCIv1ImageFormat and Dockerv2ImageFormat. + OutputFormat string + // SystemContext holds parameters used for authentication. + SystemContext *types.SystemContext + CommonBuildOpts *buildah.CommonBuildOptions + // DefaultMountsFilePath is the file path holding the mounts to be mounted in "host-path:container-path" format + DefaultMountsFilePath string +} + +// Executor is a buildah-based implementation of the imagebuilder.Executor +// interface. +type Executor struct { + index int + name string + named map[string]*Executor + store storage.Store + contextDir string + builder *buildah.Builder + pullPolicy int + registry string + transport string + ignoreUnrecognizedInstructions bool + quiet bool + runtime string + runtimeArgs []string + transientMounts []Mount + compression archive.Compression + output string + outputFormat string + additionalTags []string + log func(format string, args ...interface{}) + out io.Writer + err io.Writer + signaturePolicyPath string + systemContext *types.SystemContext + mountPoint string + preserved int + volumes imagebuilder.VolumeSet + volumeCache map[string]string + volumeCacheInfo map[string]os.FileInfo + reportWriter io.Writer + commonBuildOptions *buildah.CommonBuildOptions + defaultMountsFilePath string +} + +// withName creates a new child executor that will be used whenever a COPY statement uses --from=NAME. +func (b *Executor) withName(name string, index int) *Executor { + if b.named == nil { + b.named = make(map[string]*Executor) + } + copied := *b + copied.index = index + copied.name = name + child := &copied + b.named[name] = child + if idx := strconv.Itoa(index); idx != name { + b.named[idx] = child + } + return child +} + +// Preserve informs the executor that from this point on, it needs to ensure +// that only COPY and ADD instructions can modify the contents of this +// directory or anything below it. +// The Executor handles this by caching the contents of directories which have +// been marked this way before executing a RUN instruction, invalidating that +// cache when an ADD or COPY instruction sets any location under the directory +// as the destination, and using the cache to reset the contents of the +// directory tree after processing each RUN instruction. +// It would be simpler if we could just mark the directory as a read-only bind +// mount of itself during Run(), but the directory is expected to be remain +// writeable, even if any changes within it are ultimately discarded. +func (b *Executor) Preserve(path string) error { + logrus.Debugf("PRESERVE %q", path) + if b.volumes.Covers(path) { + // This path is already a subdirectory of a volume path that + // we're already preserving, so there's nothing new to be done + // except ensure that it exists. + archivedPath := filepath.Join(b.mountPoint, path) + if err := os.MkdirAll(archivedPath, 0755); err != nil { + return errors.Wrapf(err, "error ensuring volume path %q exists", archivedPath) + } + if err := b.volumeCacheInvalidate(path); err != nil { + return errors.Wrapf(err, "error ensuring volume path %q is preserved", archivedPath) + } + return nil + } + // Figure out where the cache for this volume would be stored. + b.preserved++ + cacheDir, err := b.store.ContainerDirectory(b.builder.ContainerID) + if err != nil { + return errors.Errorf("unable to locate temporary directory for container") + } + cacheFile := filepath.Join(cacheDir, fmt.Sprintf("volume%d.tar", b.preserved)) + // Save info about the top level of the location that we'll be archiving. + archivedPath := filepath.Join(b.mountPoint, path) + + // Try and resolve the symlink (if one exists) + // Set archivedPath and path based on whether a symlink is found or not + if symLink, err := resolveSymLink(b.mountPoint, path); err == nil { + archivedPath = filepath.Join(b.mountPoint, symLink) + path = symLink + } else { + return errors.Wrapf(err, "error reading symbolic link to %q", path) + } + + st, err := os.Stat(archivedPath) + if os.IsNotExist(err) { + if err = os.MkdirAll(archivedPath, 0755); err != nil { + return errors.Wrapf(err, "error ensuring volume path %q exists", archivedPath) + } + st, err = os.Stat(archivedPath) + } + if err != nil { + logrus.Debugf("error reading info about %q: %v", archivedPath, err) + return errors.Wrapf(err, "error reading info about volume path %q", archivedPath) + } + b.volumeCacheInfo[path] = st + if !b.volumes.Add(path) { + // This path is not a subdirectory of a volume path that we're + // already preserving, so adding it to the list should work. + return errors.Errorf("error adding %q to the volume cache", path) + } + b.volumeCache[path] = cacheFile + // Now prune cache files for volumes that are now supplanted by this one. + removed := []string{} + for cachedPath := range b.volumeCache { + // Walk our list of cached volumes, and check that they're + // still in the list of locations that we need to cache. + found := false + for _, volume := range b.volumes { + if volume == cachedPath { + // We need to keep this volume's cache. + found = true + break + } + } + if !found { + // We don't need to keep this volume's cache. Make a + // note to remove it. + removed = append(removed, cachedPath) + } + } + // Actually remove the caches that we decided to remove. + for _, cachedPath := range removed { + archivedPath := filepath.Join(b.mountPoint, cachedPath) + logrus.Debugf("no longer need cache of %q in %q", archivedPath, b.volumeCache[cachedPath]) + if err := os.Remove(b.volumeCache[cachedPath]); err != nil { + return errors.Wrapf(err, "error removing %q", b.volumeCache[cachedPath]) + } + delete(b.volumeCache, cachedPath) + } + return nil +} + +// Remove any volume cache item which will need to be re-saved because we're +// writing to part of it. +func (b *Executor) volumeCacheInvalidate(path string) error { + invalidated := []string{} + for cachedPath := range b.volumeCache { + if strings.HasPrefix(path, cachedPath+string(os.PathSeparator)) { + invalidated = append(invalidated, cachedPath) + } + } + for _, cachedPath := range invalidated { + if err := os.Remove(b.volumeCache[cachedPath]); err != nil { + return errors.Wrapf(err, "error removing volume cache %q", b.volumeCache[cachedPath]) + } + archivedPath := filepath.Join(b.mountPoint, cachedPath) + logrus.Debugf("invalidated volume cache for %q from %q", archivedPath, b.volumeCache[cachedPath]) + delete(b.volumeCache, cachedPath) + } + return nil +} + +// Save the contents of each of the executor's list of volumes for which we +// don't already have a cache file. +func (b *Executor) volumeCacheSave() error { + for cachedPath, cacheFile := range b.volumeCache { + archivedPath := filepath.Join(b.mountPoint, cachedPath) + _, err := os.Stat(cacheFile) + if err == nil { + logrus.Debugf("contents of volume %q are already cached in %q", archivedPath, cacheFile) + continue + } + if !os.IsNotExist(err) { + return errors.Wrapf(err, "error checking for cache of %q in %q", archivedPath, cacheFile) + } + if err := os.MkdirAll(archivedPath, 0755); err != nil { + return errors.Wrapf(err, "error ensuring volume path %q exists", archivedPath) + } + logrus.Debugf("caching contents of volume %q in %q", archivedPath, cacheFile) + cache, err := os.Create(cacheFile) + if err != nil { + return errors.Wrapf(err, "error creating archive at %q", cacheFile) + } + defer cache.Close() + rc, err := archive.Tar(archivedPath, archive.Uncompressed) + if err != nil { + return errors.Wrapf(err, "error archiving %q", archivedPath) + } + defer rc.Close() + _, err = io.Copy(cache, rc) + if err != nil { + return errors.Wrapf(err, "error archiving %q to %q", archivedPath, cacheFile) + } + } + return nil +} + +// Restore the contents of each of the executor's list of volumes. +func (b *Executor) volumeCacheRestore() error { + for cachedPath, cacheFile := range b.volumeCache { + archivedPath := filepath.Join(b.mountPoint, cachedPath) + logrus.Debugf("restoring contents of volume %q from %q", archivedPath, cacheFile) + cache, err := os.Open(cacheFile) + if err != nil { + return errors.Wrapf(err, "error opening archive at %q", cacheFile) + } + defer cache.Close() + if err := os.RemoveAll(archivedPath); err != nil { + return errors.Wrapf(err, "error clearing volume path %q", archivedPath) + } + if err := os.MkdirAll(archivedPath, 0755); err != nil { + return errors.Wrapf(err, "error recreating volume path %q", archivedPath) + } + err = archive.Untar(cache, archivedPath, nil) + if err != nil { + return errors.Wrapf(err, "error extracting archive at %q", archivedPath) + } + if st, ok := b.volumeCacheInfo[cachedPath]; ok { + if err := os.Chmod(archivedPath, st.Mode()); err != nil { + return errors.Wrapf(err, "error restoring permissions on %q", archivedPath) + } + if err := os.Chown(archivedPath, 0, 0); err != nil { + return errors.Wrapf(err, "error setting ownership on %q", archivedPath) + } + if err := os.Chtimes(archivedPath, st.ModTime(), st.ModTime()); err != nil { + return errors.Wrapf(err, "error restoring datestamps on %q", archivedPath) + } + } + } + return nil +} + +// Copy copies data into the working tree. The "Download" field is how +// imagebuilder tells us the instruction was "ADD" and not "COPY". +func (b *Executor) Copy(excludes []string, copies ...imagebuilder.Copy) error { + for _, copy := range copies { + logrus.Debugf("COPY %#v, %#v", excludes, copy) + if err := b.volumeCacheInvalidate(copy.Dest); err != nil { + return err + } + sources := []string{} + for _, src := range copy.Src { + if strings.HasPrefix(src, "http://") || strings.HasPrefix(src, "https://") { + sources = append(sources, src) + } else if len(copy.From) > 0 { + if other, ok := b.named[copy.From]; ok && other.index < b.index { + sources = append(sources, filepath.Join(other.mountPoint, src)) + } else { + return errors.Errorf("the stage %q has not been built", copy.From) + } + } else { + sources = append(sources, filepath.Join(b.contextDir, src)) + } + } + if err := b.builder.Add(copy.Dest, copy.Download, buildah.AddAndCopyOptions{}, sources...); err != nil { + return err + } + } + return nil +} + +func convertMounts(mounts []Mount) []specs.Mount { + specmounts := []specs.Mount{} + for _, m := range mounts { + s := specs.Mount{ + Destination: m.Destination, + Type: m.Type, + Source: m.Source, + Options: m.Options, + } + specmounts = append(specmounts, s) + } + return specmounts +} + +// Run executes a RUN instruction using the working container as a root +// directory. +func (b *Executor) Run(run imagebuilder.Run, config docker.Config) error { + logrus.Debugf("RUN %#v, %#v", run, config) + if b.builder == nil { + return errors.Errorf("no build container available") + } + options := buildah.RunOptions{ + Hostname: config.Hostname, + Runtime: b.runtime, + Args: b.runtimeArgs, + Mounts: convertMounts(b.transientMounts), + Env: config.Env, + User: config.User, + WorkingDir: config.WorkingDir, + Entrypoint: config.Entrypoint, + Cmd: config.Cmd, + NetworkDisabled: config.NetworkDisabled, + Quiet: b.quiet, + } + + args := run.Args + if run.Shell { + args = append([]string{"/bin/sh", "-c"}, args...) + } + if err := b.volumeCacheSave(); err != nil { + return err + } + err := b.builder.Run(args, options) + if err2 := b.volumeCacheRestore(); err2 != nil { + if err == nil { + return err2 + } + } + return err +} + +// UnrecognizedInstruction is called when we encounter an instruction that the +// imagebuilder parser didn't understand. +func (b *Executor) UnrecognizedInstruction(step *imagebuilder.Step) error { + errStr := fmt.Sprintf("Build error: Unknown instruction: %q ", step.Command) + err := fmt.Sprintf(errStr+"%#v", step) + if b.ignoreUnrecognizedInstructions { + logrus.Debugf(err) + return nil + } + + switch logrus.GetLevel() { + case logrus.ErrorLevel: + logrus.Errorf(errStr) + case logrus.DebugLevel: + logrus.Debugf(err) + default: + logrus.Errorf("+(UNHANDLED LOGLEVEL) %#v", step) + } + + return errors.Errorf(err) +} + +// NewExecutor creates a new instance of the imagebuilder.Executor interface. +func NewExecutor(store storage.Store, options BuildOptions) (*Executor, error) { + exec := Executor{ + store: store, + contextDir: options.ContextDirectory, + pullPolicy: options.PullPolicy, + registry: options.Registry, + transport: options.Transport, + ignoreUnrecognizedInstructions: options.IgnoreUnrecognizedInstructions, + quiet: options.Quiet, + runtime: options.Runtime, + runtimeArgs: options.RuntimeArgs, + transientMounts: options.TransientMounts, + compression: options.Compression, + output: options.Output, + outputFormat: options.OutputFormat, + additionalTags: options.AdditionalTags, + signaturePolicyPath: options.SignaturePolicyPath, + systemContext: options.SystemContext, + volumeCache: make(map[string]string), + volumeCacheInfo: make(map[string]os.FileInfo), + log: options.Log, + out: options.Out, + err: options.Err, + reportWriter: options.ReportWriter, + commonBuildOptions: options.CommonBuildOpts, + defaultMountsFilePath: options.DefaultMountsFilePath, + } + if exec.err == nil { + exec.err = os.Stderr + } + if exec.out == nil { + exec.out = os.Stdout + } + if exec.log == nil { + stepCounter := 0 + exec.log = func(format string, args ...interface{}) { + stepCounter++ + prefix := fmt.Sprintf("STEP %d: ", stepCounter) + suffix := "\n" + fmt.Fprintf(exec.err, prefix+format+suffix, args...) + } + } + return &exec, nil +} + +// Prepare creates a working container based on specified image, or if one +// isn't specified, the first FROM instruction we can find in the parsed tree. +func (b *Executor) Prepare(ctx context.Context, ib *imagebuilder.Builder, node *parser.Node, from string) error { + if from == "" { + base, err := ib.From(node) + if err != nil { + logrus.Debugf("Prepare(node.Children=%#v)", node.Children) + return errors.Wrapf(err, "error determining starting point for build") + } + from = base + } + logrus.Debugf("FROM %#v", from) + if !b.quiet { + b.log("FROM %s", from) + } + builderOptions := buildah.BuilderOptions{ + FromImage: from, + PullPolicy: b.pullPolicy, + Registry: b.registry, + Transport: b.transport, + SignaturePolicyPath: b.signaturePolicyPath, + ReportWriter: b.reportWriter, + SystemContext: b.systemContext, + CommonBuildOpts: b.commonBuildOptions, + DefaultMountsFilePath: b.defaultMountsFilePath, + } + builder, err := buildah.NewBuilder(ctx, b.store, builderOptions) + if err != nil { + return errors.Wrapf(err, "error creating build container") + } + volumes := map[string]struct{}{} + for _, v := range builder.Volumes() { + volumes[v] = struct{}{} + } + dConfig := docker.Config{ + Hostname: builder.Hostname(), + Domainname: builder.Domainname(), + User: builder.User(), + Env: builder.Env(), + Cmd: builder.Cmd(), + Image: from, + Volumes: volumes, + WorkingDir: builder.WorkDir(), + Entrypoint: builder.Entrypoint(), + Labels: builder.Labels(), + Shell: builder.Shell(), + StopSignal: builder.StopSignal(), + } + var rootfs *docker.RootFS + if builder.Docker.RootFS != nil { + rootfs = &docker.RootFS{ + Type: builder.Docker.RootFS.Type, + } + for _, id := range builder.Docker.RootFS.DiffIDs { + rootfs.Layers = append(rootfs.Layers, id.String()) + } + } + dImage := docker.Image{ + Parent: builder.FromImage, + ContainerConfig: dConfig, + Container: builder.Container, + Author: builder.Maintainer(), + Architecture: builder.Architecture(), + RootFS: rootfs, + } + dImage.Config = &dImage.ContainerConfig + err = ib.FromImage(&dImage, node) + if err != nil { + if err2 := builder.Delete(); err2 != nil { + logrus.Debugf("error deleting container which we failed to update: %v", err2) + } + return errors.Wrapf(err, "error updating build context") + } + mountPoint, err := builder.Mount(builder.MountLabel) + if err != nil { + if err2 := builder.Delete(); err2 != nil { + logrus.Debugf("error deleting container which we failed to mount: %v", err2) + } + return errors.Wrapf(err, "error mounting new container") + } + b.mountPoint = mountPoint + b.builder = builder + return nil +} + +// Delete deletes the working container, if we have one. The Executor object +// should not be used to build another image, as the name of the output image +// isn't resettable. +func (b *Executor) Delete() (err error) { + if b.builder != nil { + err = b.builder.Delete() + b.builder = nil + } + return err +} + +// Execute runs each of the steps in the parsed tree, in turn. +func (b *Executor) Execute(ib *imagebuilder.Builder, node *parser.Node) error { + for i, node := range node.Children { + step := ib.Step() + if err := step.Resolve(node); err != nil { + return errors.Wrapf(err, "error resolving step %+v", *node) + } + logrus.Debugf("Parsed Step: %+v", *step) + if !b.quiet { + b.log("%s", step.Original) + } + requiresStart := false + if i < len(node.Children)-1 { + requiresStart = ib.RequiresStart(&parser.Node{Children: node.Children[i+1:]}) + } + err := ib.Run(step, b, requiresStart) + if err != nil { + return errors.Wrapf(err, "error building at step %+v", *step) + } + } + return nil +} + +// Commit writes the container's contents to an image, using a passed-in tag as +// the name if there is one, generating a unique ID-based one otherwise. +func (b *Executor) Commit(ctx context.Context, ib *imagebuilder.Builder) (err error) { + var imageRef types.ImageReference + if b.output != "" { + imageRef, err = alltransports.ParseImageName(b.output) + if err != nil { + imageRef2, err2 := is.Transport.ParseStoreReference(b.store, b.output) + if err2 == nil { + imageRef = imageRef2 + err = nil + } else { + err = err2 + } + } + } else { + imageRef, err = is.Transport.ParseStoreReference(b.store, "@"+stringid.GenerateRandomID()) + } + if err != nil { + return errors.Wrapf(err, "error parsing reference for image to be written") + } + if ib.Author != "" { + b.builder.SetMaintainer(ib.Author) + } + config := ib.Config() + b.builder.SetHostname(config.Hostname) + b.builder.SetDomainname(config.Domainname) + b.builder.SetUser(config.User) + b.builder.ClearPorts() + for p := range config.ExposedPorts { + b.builder.SetPort(string(p)) + } + b.builder.ClearEnv() + for _, envSpec := range config.Env { + spec := strings.SplitN(envSpec, "=", 2) + b.builder.SetEnv(spec[0], spec[1]) + } + b.builder.SetCmd(config.Cmd) + b.builder.ClearVolumes() + for v := range config.Volumes { + b.builder.AddVolume(v) + } + b.builder.SetWorkDir(config.WorkingDir) + b.builder.SetEntrypoint(config.Entrypoint) + b.builder.SetShell(config.Shell) + b.builder.SetStopSignal(config.StopSignal) + b.builder.ClearLabels() + for k, v := range config.Labels { + b.builder.SetLabel(k, v) + } + if imageRef != nil { + logName := transports.ImageName(imageRef) + logrus.Debugf("COMMIT %q", logName) + if !b.quiet { + b.log("COMMIT %s", logName) + } + } else { + logrus.Debugf("COMMIT") + if !b.quiet { + b.log("COMMIT") + } + } + options := buildah.CommitOptions{ + Compression: b.compression, + SignaturePolicyPath: b.signaturePolicyPath, + AdditionalTags: b.additionalTags, + ReportWriter: b.reportWriter, + PreferredManifestType: b.outputFormat, + } + return b.builder.Commit(ctx, imageRef, options) +} + +// Build takes care of the details of running Prepare/Execute/Commit/Delete +// over each of the one or more parsed Dockerfiles and stages. +func (b *Executor) Build(ctx context.Context, stages imagebuilder.Stages) error { + if len(stages) == 0 { + errors.New("error building: no stages to build") + } + var stageExecutor *Executor + for _, stage := range stages { + stageExecutor = b.withName(stage.Name, stage.Position) + if err := stageExecutor.Prepare(ctx, stage.Builder, stage.Node, ""); err != nil { + return err + } + defer stageExecutor.Delete() + if err := stageExecutor.Execute(stage.Builder, stage.Node); err != nil { + return err + } + } + return stageExecutor.Commit(ctx, stages[len(stages)-1].Builder) +} + +// BuildDockerfiles parses a set of one or more Dockerfiles (which may be +// URLs), creates a new Executor, and then runs Prepare/Execute/Commit/Delete +// over the entire set of instructions. +func BuildDockerfiles(ctx context.Context, store storage.Store, options BuildOptions, paths ...string) error { + if len(paths) == 0 { + return errors.Errorf("error building: no dockerfiles specified") + } + var dockerfiles []io.ReadCloser + defer func(dockerfiles ...io.ReadCloser) { + for _, d := range dockerfiles { + d.Close() + } + }(dockerfiles...) + for _, dfile := range paths { + if strings.HasPrefix(dfile, "http://") || strings.HasPrefix(dfile, "https://") { + logrus.Debugf("reading remote Dockerfile %q", dfile) + resp, err := http.Get(dfile) + if err != nil { + return errors.Wrapf(err, "error getting %q", dfile) + } + if resp.ContentLength == 0 { + resp.Body.Close() + return errors.Errorf("no contents in %q", dfile) + } + dockerfiles = append(dockerfiles, resp.Body) + } else { + if !filepath.IsAbs(dfile) { + logrus.Debugf("resolving local Dockerfile %q", dfile) + dfile = filepath.Join(options.ContextDirectory, dfile) + } + logrus.Debugf("reading local Dockerfile %q", dfile) + contents, err := os.Open(dfile) + if err != nil { + return errors.Wrapf(err, "error reading %q", dfile) + } + dinfo, err := contents.Stat() + if err != nil { + contents.Close() + return errors.Wrapf(err, "error reading info about %q", dfile) + } + if dinfo.Size() == 0 { + contents.Close() + return errors.Wrapf(err, "no contents in %q", dfile) + } + dockerfiles = append(dockerfiles, contents) + } + } + mainNode, err := imagebuilder.ParseDockerfile(dockerfiles[0]) + if err != nil { + return errors.Wrapf(err, "error parsing main Dockerfile") + } + for _, d := range dockerfiles[1:] { + additionalNode, err := imagebuilder.ParseDockerfile(d) + if err != nil { + return errors.Wrapf(err, "error parsing additional Dockerfile") + } + mainNode.Children = append(mainNode.Children, additionalNode.Children...) + } + exec, err := NewExecutor(store, options) + if err != nil { + return errors.Wrapf(err, "error creating build executor") + } + b := imagebuilder.NewBuilder(options.Args) + stages := imagebuilder.NewStages(mainNode, b) + return exec.Build(ctx, stages) +} diff --git a/vendor/github.com/projectatomic/buildah/imagebuildah/chroot_symlink.go b/vendor/github.com/projectatomic/buildah/imagebuildah/chroot_symlink.go new file mode 100644 index 000000000..b2452b61c --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/imagebuildah/chroot_symlink.go @@ -0,0 +1,145 @@ +package imagebuildah + +import ( + "flag" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/containers/storage/pkg/reexec" + "github.com/pkg/errors" + "golang.org/x/sys/unix" +) + +const ( + symlinkChrootedCommand = "chrootsymlinks-resolve" + maxSymlinksResolved = 40 +) + +func init() { + reexec.Register(symlinkChrootedCommand, resolveChrootedSymlinks) +} + +func resolveChrootedSymlinks() { + status := 0 + flag.Parse() + if len(flag.Args()) < 1 { + os.Exit(1) + } + // Our first parameter is the directory to chroot into. + if err := unix.Chdir(flag.Arg(0)); err != nil { + fmt.Fprintf(os.Stderr, "chdir(): %v\n", err) + os.Exit(1) + } + if err := unix.Chroot(flag.Arg(0)); err != nil { + fmt.Fprintf(os.Stderr, "chroot(): %v\n", err) + os.Exit(1) + } + + // Our second paramter is the path name to evaluate for symbolic links + symLink, err := getSymbolicLink(flag.Arg(0), flag.Arg(1)) + if err != nil { + fmt.Fprintf(os.Stderr, "error getting symbolic links: %v\n", err) + os.Exit(1) + } + if _, err := os.Stdout.WriteString(symLink); err != nil { + fmt.Fprintf(os.Stderr, "error writing string to stdout: %v\n", err) + os.Exit(1) + } + os.Exit(status) +} + +func resolveSymLink(rootdir, filename string) (string, error) { + // The child process expects a chroot and one path that + // will be consulted relative to the chroot directory and evaluated + // for any symbolic links present. + cmd := reexec.Command(symlinkChrootedCommand, rootdir, filename) + output, err := cmd.CombinedOutput() + if err != nil { + return "", errors.Wrapf(err, string(output)) + } + + // Hand back the resolved symlink, will be "" if a symlink is not found + return string(output), nil +} + +// getSymbolic link goes through each part of the path and continues resolving symlinks as they appear. +// Returns what the whole target path for what "path" resolves to. +func getSymbolicLink(rootdir, path string) (string, error) { + var ( + symPath string + symLinksResolved int + ) + + // Splitting path as we need to resolve each parth of the path at a time + splitPath := strings.Split(path, "/") + if splitPath[0] == "" { + splitPath = splitPath[1:] + symPath = "/" + } + + for _, p := range splitPath { + // If we have resolved 40 symlinks, that means something is terribly wrong + // will return an error and exit + if symLinksResolved >= maxSymlinksResolved { + return "", errors.Errorf("have resolved %q symlinks, something is terribly wrong!", maxSymlinksResolved) + } + + symPath = filepath.Join(symPath, p) + isSymlink, resolvedPath, err := hasSymlink(symPath) + if err != nil { + return "", errors.Wrapf(err, "error checking symlink for %q", symPath) + } + // if isSymlink is true, check if resolvedPath is potentially another symlink + // keep doing this till resolvedPath is not a symlink and isSymlink is false + for isSymlink == true { + // Need to keep track of number of symlinks resolved + // Will also return an error if the symlink points to itself as that will exceed maxSymlinksResolved + if symLinksResolved >= maxSymlinksResolved { + return "", errors.Errorf("have resolved %q symlinks, something is terribly wrong!", maxSymlinksResolved) + } + isSymlink, resolvedPath, err = hasSymlink(resolvedPath) + if err != nil { + return "", errors.Wrapf(err, "error checking symlink for %q", resolvedPath) + } + symLinksResolved++ + } + // Assign resolvedPath to symPath. The next part of the loop will append the next part of the original path + // and continue resolving + symPath = resolvedPath + symLinksResolved++ + } + return symPath, nil +} + +// hasSymlink returns true and the target if path is symlink +// otherwise it returns false and path +func hasSymlink(path string) (bool, string, error) { + info, err := os.Lstat(path) + if os.IsNotExist(err) { + if err = os.MkdirAll(path, 0755); err != nil { + return false, "", errors.Wrapf(err, "error ensuring volume path %q exists", path) + } + info, err = os.Lstat(path) + if err != nil { + return false, "", errors.Wrapf(err, "error running lstat on %q", path) + } + } + // Return false and path as path is not a symlink + if info.Mode()&os.ModeSymlink != os.ModeSymlink { + return false, path, nil + } + + // Read the symlink to get what it points to + targetDir, err := os.Readlink(path) + if err != nil { + return false, "", errors.Wrapf(err, "error reading link %q", path) + } + // if the symlink points to a relative path, prepend the path till now to the resolved path + if !filepath.IsAbs(targetDir) { + targetDir = filepath.Join(path, targetDir) + } + // run filepath.Clean to remove the ".." from relative paths + return true, filepath.Clean(targetDir), nil +} diff --git a/vendor/github.com/projectatomic/buildah/imagebuildah/util.go b/vendor/github.com/projectatomic/buildah/imagebuildah/util.go new file mode 100644 index 000000000..805cfce44 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/imagebuildah/util.go @@ -0,0 +1,96 @@ +package imagebuildah + +import ( + "fmt" + "io/ioutil" + "net/http" + "os" + "os/exec" + "path" + "strings" + + "github.com/containers/storage/pkg/chrootarchive" + "github.com/pkg/errors" + "github.com/projectatomic/buildah" + "github.com/sirupsen/logrus" +) + +func cloneToDirectory(url, dir string) error { + if !strings.HasPrefix(url, "git://") { + url = "git://" + url + } + logrus.Debugf("cloning %q to %q", url, dir) + cmd := exec.Command("git", "clone", url, dir) + return cmd.Run() +} + +func downloadToDirectory(url, dir string) error { + logrus.Debugf("extracting %q to %q", url, dir) + resp, err := http.Get(url) + if err != nil { + return errors.Wrapf(err, "error getting %q", url) + } + defer resp.Body.Close() + if resp.ContentLength == 0 { + return errors.Errorf("no contents in %q", url) + } + return chrootarchive.Untar(resp.Body, dir, nil) +} + +// TempDirForURL checks if the passed-in string looks like a URL. If it is, +// TempDirForURL creates a temporary directory, arranges for its contents to be +// the contents of that URL, and returns the temporary directory's path, along +// with the name of a subdirectory which should be used as the build context +// (which may be empty or "."). Removal of the temporary directory is the +// responsibility of the caller. If the string doesn't look like a URL, +// TempDirForURL returns empty strings and a nil error code. +func TempDirForURL(dir, prefix, url string) (name string, subdir string, err error) { + if !strings.HasPrefix(url, "http://") && + !strings.HasPrefix(url, "https://") && + !strings.HasPrefix(url, "git://") && + !strings.HasPrefix(url, "github.com/") { + return "", "", nil + } + name, err = ioutil.TempDir(dir, prefix) + if err != nil { + return "", "", errors.Wrapf(err, "error creating temporary directory for %q", url) + } + if strings.HasPrefix(url, "git://") { + err = cloneToDirectory(url, name) + if err != nil { + if err2 := os.Remove(name); err2 != nil { + logrus.Debugf("error removing temporary directory %q: %v", name, err2) + } + return "", "", err + } + return name, "", nil + } + if strings.HasPrefix(url, "github.com/") { + ghurl := url + url = fmt.Sprintf("https://%s/archive/master.tar.gz", ghurl) + logrus.Debugf("resolving url %q to %q", ghurl, url) + subdir = path.Base(ghurl) + "-master" + } + if strings.HasPrefix(url, "http://") || strings.HasPrefix(url, "https://") { + err = downloadToDirectory(url, name) + if err != nil { + if err2 := os.Remove(name); err2 != nil { + logrus.Debugf("error removing temporary directory %q: %v", name, err2) + } + return "", subdir, err + } + return name, subdir, nil + } + logrus.Debugf("don't know how to retrieve %q", url) + if err2 := os.Remove(name); err2 != nil { + logrus.Debugf("error removing temporary directory %q: %v", name, err2) + } + return "", "", errors.Errorf("unreachable code reached") +} + +// InitReexec is a wrapper for buildah.InitReexec(). It should be called at +// the start of main(), and if it returns true, main() should return +// immediately. +func InitReexec() bool { + return buildah.InitReexec() +} diff --git a/vendor/github.com/projectatomic/buildah/import.go b/vendor/github.com/projectatomic/buildah/import.go new file mode 100644 index 000000000..b98219107 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/import.go @@ -0,0 +1,123 @@ +package buildah + +import ( + "context" + + is "github.com/containers/image/storage" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/opencontainers/go-digest" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/docker" + "github.com/projectatomic/buildah/util" +) + +func importBuilderDataFromImage(ctx context.Context, store storage.Store, systemContext *types.SystemContext, imageID, containerName, containerID string) (*Builder, error) { + manifest := []byte{} + config := []byte{} + imageName := "" + + if imageID != "" { + ref, err := is.Transport.ParseStoreReference(store, imageID) + if err != nil { + return nil, errors.Wrapf(err, "no such image %q", imageID) + } + src, err2 := ref.NewImage(ctx, systemContext) + if err2 != nil { + return nil, errors.Wrapf(err2, "error instantiating image") + } + defer src.Close() + config, err = src.ConfigBlob(ctx) + if err != nil { + return nil, errors.Wrapf(err, "error reading image configuration") + } + manifest, _, err = src.Manifest(ctx) + if err != nil { + return nil, errors.Wrapf(err, "error reading image manifest") + } + if img, err3 := store.Image(imageID); err3 == nil { + if len(img.Names) > 0 { + imageName = img.Names[0] + } + } + } + + builder := &Builder{ + store: store, + Type: containerType, + FromImage: imageName, + FromImageID: imageID, + Config: config, + Manifest: manifest, + Container: containerName, + ContainerID: containerID, + ImageAnnotations: map[string]string{}, + ImageCreatedBy: "", + } + + builder.initConfig() + + return builder, nil +} + +func importBuilder(ctx context.Context, store storage.Store, options ImportOptions) (*Builder, error) { + if options.Container == "" { + return nil, errors.Errorf("container name must be specified") + } + + c, err := store.Container(options.Container) + if err != nil { + return nil, err + } + + systemContext := getSystemContext(&types.SystemContext{}, options.SignaturePolicyPath) + + builder, err := importBuilderDataFromImage(ctx, store, systemContext, c.ImageID, options.Container, c.ID) + if err != nil { + return nil, err + } + + if builder.FromImageID != "" { + if d, err2 := digest.Parse(builder.FromImageID); err2 == nil { + builder.Docker.Parent = docker.ID(d) + } else { + builder.Docker.Parent = docker.ID(digest.NewDigestFromHex(digest.Canonical.String(), builder.FromImageID)) + } + } + if builder.FromImage != "" { + builder.Docker.ContainerConfig.Image = builder.FromImage + } + + err = builder.Save() + if err != nil { + return nil, errors.Wrapf(err, "error saving builder state") + } + + return builder, nil +} + +func importBuilderFromImage(ctx context.Context, store storage.Store, options ImportFromImageOptions) (*Builder, error) { + var img *storage.Image + var err error + + if options.Image == "" { + return nil, errors.Errorf("image name must be specified") + } + + systemContext := getSystemContext(options.SystemContext, options.SignaturePolicyPath) + + for _, image := range util.ResolveName(options.Image, "", systemContext, store) { + img, err = util.FindImage(store, image) + if err != nil { + continue + } + + builder, err2 := importBuilderDataFromImage(ctx, store, systemContext, img.ID, "", "") + if err2 != nil { + return nil, errors.Wrapf(err2, "error importing build settings from image %q", options.Image) + } + + return builder, nil + } + return nil, errors.Wrapf(err, "error locating image %q for importing settings", options.Image) +} diff --git a/vendor/github.com/projectatomic/buildah/mount.go b/vendor/github.com/projectatomic/buildah/mount.go new file mode 100644 index 000000000..4f1ae3c6e --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/mount.go @@ -0,0 +1,17 @@ +package buildah + +// Mount mounts a container's root filesystem in a location which can be +// accessed from the host, and returns the location. +func (b *Builder) Mount(label string) (string, error) { + mountpoint, err := b.store.Mount(b.ContainerID, label) + if err != nil { + return "", err + } + b.MountPoint = mountpoint + + err = b.Save() + if err != nil { + return "", err + } + return mountpoint, nil +} diff --git a/vendor/github.com/projectatomic/buildah/new.go b/vendor/github.com/projectatomic/buildah/new.go new file mode 100644 index 000000000..82de524c0 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/new.go @@ -0,0 +1,313 @@ +package buildah + +import ( + "context" + "fmt" + "os" + "strings" + + is "github.com/containers/image/storage" + "github.com/containers/image/transports" + "github.com/containers/image/transports/alltransports" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/opencontainers/selinux/go-selinux" + "github.com/opencontainers/selinux/go-selinux/label" + "github.com/openshift/imagebuilder" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/util" + "github.com/sirupsen/logrus" +) + +const ( + // BaseImageFakeName is the "name" of a source image which we interpret + // as "no image". + BaseImageFakeName = imagebuilder.NoBaseImageSpecifier + + // DefaultTransport is a prefix that we apply to an image name if we + // can't find one in the local Store, in order to generate a source + // reference for the image that we can then copy to the local Store. + DefaultTransport = "docker://" + + // minimumTruncatedIDLength is the minimum length of an identifier that + // we'll accept as possibly being a truncated image ID. + minimumTruncatedIDLength = 3 +) + +func reserveSELinuxLabels(store storage.Store, id string) error { + if selinux.GetEnabled() { + containers, err := store.Containers() + if err != nil { + return err + } + + for _, c := range containers { + if id == c.ID { + continue + } else { + b, err := OpenBuilder(store, c.ID) + if err != nil { + if os.IsNotExist(err) { + // Ignore not exist errors since containers probably created by other tool + // TODO, we need to read other containers json data to reserve their SELinux labels + continue + } + return err + } + // Prevent containers from using same MCS Label + if err := label.ReserveLabel(b.ProcessLabel); err != nil { + return err + } + } + } + } + return nil +} + +func pullAndFindImage(ctx context.Context, store storage.Store, imageName string, options BuilderOptions, sc *types.SystemContext) (*storage.Image, types.ImageReference, error) { + ref, err := pullImage(ctx, store, imageName, options, sc) + if err != nil { + logrus.Debugf("error pulling image %q: %v", imageName, err) + return nil, nil, err + } + img, err := is.Transport.GetStoreImage(store, ref) + if err != nil { + logrus.Debugf("error reading pulled image %q: %v", imageName, err) + return nil, nil, err + } + return img, ref, nil +} + +func getImageName(name string, img *storage.Image) string { + imageName := name + if len(img.Names) > 0 { + imageName = img.Names[0] + // When the image used by the container is a tagged image + // the container name might be set to the original image instead of + // the image given in the "form" command line. + // This loop is supposed to fix this. + for _, n := range img.Names { + if strings.Contains(n, name) { + imageName = n + break + } + } + } + return imageName +} + +func imageNamePrefix(imageName string) string { + prefix := imageName + s := strings.Split(imageName, "/") + if len(s) > 0 { + prefix = s[len(s)-1] + } + s = strings.Split(prefix, ":") + if len(s) > 0 { + prefix = s[0] + } + s = strings.Split(prefix, "@") + if len(s) > 0 { + prefix = s[0] + } + return prefix +} + +func imageManifestAndConfig(ctx context.Context, ref types.ImageReference, systemContext *types.SystemContext) (manifest, config []byte, err error) { + if ref != nil { + src, err := ref.NewImage(ctx, systemContext) + if err != nil { + return nil, nil, errors.Wrapf(err, "error instantiating image for %q", transports.ImageName(ref)) + } + defer src.Close() + config, err := src.ConfigBlob(ctx) + if err != nil { + return nil, nil, errors.Wrapf(err, "error reading image configuration for %q", transports.ImageName(ref)) + } + manifest, _, err := src.Manifest(ctx) + if err != nil { + return nil, nil, errors.Wrapf(err, "error reading image manifest for %q", transports.ImageName(ref)) + } + return manifest, config, nil + } + return nil, nil, nil +} + +func newBuilder(ctx context.Context, store storage.Store, options BuilderOptions) (*Builder, error) { + var ref types.ImageReference + var img *storage.Image + var err error + var manifest []byte + var config []byte + + if options.FromImage == BaseImageFakeName { + options.FromImage = "" + } + if options.Transport == "" { + options.Transport = DefaultTransport + } + + systemContext := getSystemContext(options.SystemContext, options.SignaturePolicyPath) + + for _, image := range util.ResolveName(options.FromImage, options.Registry, systemContext, store) { + if len(image) >= minimumTruncatedIDLength { + if img, err = store.Image(image); err == nil && img != nil && strings.HasPrefix(img.ID, image) { + if ref, err = is.Transport.ParseStoreReference(store, img.ID); err != nil { + return nil, errors.Wrapf(err, "error parsing reference to image %q", img.ID) + } + break + } + } + + if options.PullPolicy == PullAlways { + pulledImg, pulledReference, err2 := pullAndFindImage(ctx, store, image, options, systemContext) + if err2 != nil { + logrus.Debugf("error pulling and reading image %q: %v", image, err2) + err = err2 + continue + } + ref = pulledReference + img = pulledImg + break + } + + srcRef, err2 := alltransports.ParseImageName(image) + if err2 != nil { + if options.Transport == "" { + logrus.Debugf("error parsing image name %q: %v", image, err2) + err = err2 + continue + } + transport := options.Transport + if transport != DefaultTransport { + transport = transport + ":" + } + srcRef2, err3 := alltransports.ParseImageName(transport + image) + if err3 != nil { + logrus.Debugf("error parsing image name %q: %v", image, err2) + err = err3 + continue + } + srcRef = srcRef2 + } + + destImage, err2 := localImageNameForReference(ctx, store, srcRef, options.FromImage) + if err2 != nil { + return nil, errors.Wrapf(err2, "error computing local image name for %q", transports.ImageName(srcRef)) + } + if destImage == "" { + return nil, errors.Errorf("error computing local image name for %q", transports.ImageName(srcRef)) + } + + ref, err = is.Transport.ParseStoreReference(store, destImage) + if err != nil { + return nil, errors.Wrapf(err, "error parsing reference to image %q", destImage) + } + img, err = is.Transport.GetStoreImage(store, ref) + if err != nil { + if errors.Cause(err) == storage.ErrImageUnknown && options.PullPolicy != PullIfMissing { + logrus.Debugf("no such image %q: %v", transports.ImageName(ref), err) + continue + } + pulledImg, pulledReference, err2 := pullAndFindImage(ctx, store, image, options, systemContext) + if err2 != nil { + logrus.Debugf("error pulling and reading image %q: %v", image, err2) + err = err2 + continue + } + ref = pulledReference + img = pulledImg + } + break + } + + if options.FromImage != "" && (ref == nil || img == nil) { + // If options.FromImage is set but we ended up + // with nil in ref or in img then there was an error that + // we should return. + return nil, util.GetFailureCause(err, errors.Wrapf(storage.ErrImageUnknown, "no such image %q in registry", options.FromImage)) + } + image := options.FromImage + imageID := "" + if img != nil { + image = getImageName(imageNamePrefix(image), img) + imageID = img.ID + } + if manifest, config, err = imageManifestAndConfig(ctx, ref, systemContext); err != nil { + return nil, errors.Wrapf(err, "error reading data from image %q", transports.ImageName(ref)) + } + + name := "working-container" + if options.Container != "" { + name = options.Container + } else { + var err2 error + if image != "" { + name = imageNamePrefix(image) + "-" + name + } + suffix := 1 + tmpName := name + for errors.Cause(err2) != storage.ErrContainerUnknown { + _, err2 = store.Container(tmpName) + if err2 == nil { + suffix++ + tmpName = fmt.Sprintf("%s-%d", name, suffix) + } + } + name = tmpName + } + + coptions := storage.ContainerOptions{} + container, err := store.CreateContainer("", []string{name}, imageID, "", "", &coptions) + if err != nil { + return nil, errors.Wrapf(err, "error creating container") + } + + defer func() { + if err != nil { + if err2 := store.DeleteContainer(container.ID); err != nil { + logrus.Errorf("error deleting container %q: %v", container.ID, err2) + } + } + }() + + if err = reserveSELinuxLabels(store, container.ID); err != nil { + return nil, err + } + processLabel, mountLabel, err := label.InitLabels(options.CommonBuildOpts.LabelOpts) + if err != nil { + return nil, err + } + + builder := &Builder{ + store: store, + Type: containerType, + FromImage: image, + FromImageID: imageID, + Config: config, + Manifest: manifest, + Container: name, + ContainerID: container.ID, + ImageAnnotations: map[string]string{}, + ImageCreatedBy: "", + ProcessLabel: processLabel, + MountLabel: mountLabel, + DefaultMountsFilePath: options.DefaultMountsFilePath, + CommonBuildOpts: options.CommonBuildOpts, + } + + if options.Mount { + _, err = builder.Mount(mountLabel) + if err != nil { + return nil, errors.Wrapf(err, "error mounting build container") + } + } + + builder.initConfig() + err = builder.Save() + if err != nil { + return nil, errors.Wrapf(err, "error saving builder state") + } + + return builder, nil +} diff --git a/vendor/github.com/projectatomic/buildah/pkg/cli/common.go b/vendor/github.com/projectatomic/buildah/pkg/cli/common.go new file mode 100644 index 000000000..bead9e6be --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/pkg/cli/common.go @@ -0,0 +1,130 @@ +package cli + +// the cli package contains urfave/cli related structs that help make up +// the command line for buildah commands. it resides here so other projects +// that vendor in this code can use them too. + +import ( + "github.com/projectatomic/buildah/imagebuildah" + "github.com/urfave/cli" +) + +var ( + BudFlags = []cli.Flag{ + cli.StringFlag{ + Name: "authfile", + Usage: "path of the authentication file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json", + }, + cli.StringSliceFlag{ + Name: "build-arg", + Usage: "`argument=value` to supply to the builder", + }, + cli.StringFlag{ + Name: "cert-dir", + Value: "", + Usage: "use certificates at the specified path to access the registry", + }, + cli.StringFlag{ + Name: "creds", + Value: "", + Usage: "use `[username[:password]]` for accessing the registry", + }, + cli.StringSliceFlag{ + Name: "file, f", + Usage: "`pathname or URL` of a Dockerfile", + }, + cli.StringFlag{ + Name: "format", + Usage: "`format` of the built image's manifest and metadata", + }, + cli.BoolTFlag{ + Name: "pull", + Usage: "pull the image if not present", + }, + cli.BoolFlag{ + Name: "pull-always", + Usage: "pull the image, even if a version is present", + }, + cli.BoolFlag{ + Name: "quiet, q", + Usage: "refrain from announcing build instructions and image read/write progress", + }, + cli.StringFlag{ + Name: "runtime", + Usage: "`path` to an alternate runtime", + Value: imagebuildah.DefaultRuntime, + }, + cli.StringSliceFlag{ + Name: "runtime-flag", + Usage: "add global flags for the container runtime", + }, + cli.StringFlag{ + Name: "signature-policy", + Usage: "`pathname` of signature policy file (not usually used)", + }, + cli.StringSliceFlag{ + Name: "tag, t", + Usage: "`tag` to apply to the built image", + }, + cli.BoolTFlag{ + Name: "tls-verify", + Usage: "require HTTPS and verify certificates when accessing the registry", + }, + } + + FromAndBudFlags = []cli.Flag{ + cli.StringSliceFlag{ + Name: "add-host", + Usage: "add a custom host-to-IP mapping (host:ip) (default [])", + }, + cli.StringFlag{ + Name: "cgroup-parent", + Usage: "optional parent cgroup for the container", + }, + cli.Uint64Flag{ + Name: "cpu-period", + Usage: "limit the CPU CFS (Completely Fair Scheduler) period", + }, + cli.Int64Flag{ + Name: "cpu-quota", + Usage: "limit the CPU CFS (Completely Fair Scheduler) quota", + }, + cli.Uint64Flag{ + Name: "cpu-shares", + Usage: "CPU shares (relative weight)", + }, + cli.StringFlag{ + Name: "cpuset-cpus", + Usage: "CPUs in which to allow execution (0-3, 0,1)", + }, + cli.StringFlag{ + Name: "cpuset-mems", + Usage: "memory nodes (MEMs) in which to allow execution (0-3, 0,1). Only effective on NUMA systems.", + }, + cli.StringFlag{ + Name: "memory, m", + Usage: "memory limit (format: <number>[<unit>], where unit = b, k, m or g)", + }, + cli.StringFlag{ + Name: "memory-swap", + Usage: "swap limit equal to memory plus swap: '-1' to enable unlimited swap", + }, + cli.StringSliceFlag{ + Name: "security-opt", + Usage: "security Options (default [])", + }, + cli.StringFlag{ + Name: "shm-size", + Usage: "size of `/dev/shm`. The format is `<number><unit>`.", + Value: "65536k", + }, + cli.StringSliceFlag{ + Name: "ulimit", + Usage: "ulimit options (default [])", + }, + cli.StringSliceFlag{ + Name: "volume, v", + Usage: "bind mount a volume into the container (default [])", + }, + } +) diff --git a/vendor/github.com/projectatomic/buildah/pkg/parse/parse.go b/vendor/github.com/projectatomic/buildah/pkg/parse/parse.go new file mode 100644 index 000000000..f2159d930 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/pkg/parse/parse.go @@ -0,0 +1,323 @@ +package parse + +// this package should contain functions that parse and validate +// user input and is shared either amongst buildah subcommands or +// would be useful to projects vendoring buildah + +import ( + "fmt" + "net" + "os" + "reflect" + "regexp" + "strings" + + "github.com/containers/image/types" + "github.com/docker/go-units" + "github.com/pkg/errors" + "github.com/projectatomic/buildah" + "github.com/urfave/cli" + "golang.org/x/crypto/ssh/terminal" +) + +const ( + // SeccompDefaultPath defines the default seccomp path + SeccompDefaultPath = "/usr/share/containers/seccomp.json" + // SeccompOverridePath if this exists it overrides the default seccomp path + SeccompOverridePath = "/etc/crio/seccomp.json" +) + +// ParseCommonBuildOptions parses the build options from the bud cli +func ParseCommonBuildOptions(c *cli.Context) (*buildah.CommonBuildOptions, error) { + var ( + memoryLimit int64 + memorySwap int64 + err error + ) + if c.String("memory") != "" { + memoryLimit, err = units.RAMInBytes(c.String("memory")) + if err != nil { + return nil, errors.Wrapf(err, "invalid value for memory") + } + } + if c.String("memory-swap") != "" { + memorySwap, err = units.RAMInBytes(c.String("memory-swap")) + if err != nil { + return nil, errors.Wrapf(err, "invalid value for memory-swap") + } + } + if len(c.StringSlice("add-host")) > 0 { + for _, host := range c.StringSlice("add-host") { + if err := validateExtraHost(host); err != nil { + return nil, errors.Wrapf(err, "invalid value for add-host") + } + } + } + if _, err := units.FromHumanSize(c.String("shm-size")); err != nil { + return nil, errors.Wrapf(err, "invalid --shm-size") + } + if err := parseVolumes(c.StringSlice("volume")); err != nil { + return nil, err + } + + commonOpts := &buildah.CommonBuildOptions{ + AddHost: c.StringSlice("add-host"), + CgroupParent: c.String("cgroup-parent"), + CPUPeriod: c.Uint64("cpu-period"), + CPUQuota: c.Int64("cpu-quota"), + CPUSetCPUs: c.String("cpuset-cpus"), + CPUSetMems: c.String("cpuset-mems"), + CPUShares: c.Uint64("cpu-shares"), + Memory: memoryLimit, + MemorySwap: memorySwap, + ShmSize: c.String("shm-size"), + Ulimit: c.StringSlice("ulimit"), + Volumes: c.StringSlice("volume"), + } + if err := parseSecurityOpts(c.StringSlice("security-opt"), commonOpts); err != nil { + return nil, err + } + return commonOpts, nil +} + +func parseSecurityOpts(securityOpts []string, commonOpts *buildah.CommonBuildOptions) error { + for _, opt := range securityOpts { + if opt == "no-new-privileges" { + return errors.Errorf("no-new-privileges is not supported") + } + con := strings.SplitN(opt, "=", 2) + if len(con) != 2 { + return errors.Errorf("Invalid --security-opt 1: %q", opt) + } + + switch con[0] { + case "label": + commonOpts.LabelOpts = append(commonOpts.LabelOpts, con[1]) + case "apparmor": + commonOpts.ApparmorProfile = con[1] + case "seccomp": + commonOpts.SeccompProfilePath = con[1] + default: + return errors.Errorf("Invalid --security-opt 2: %q", opt) + } + + } + + if commonOpts.SeccompProfilePath == "" { + if _, err := os.Stat(SeccompOverridePath); err == nil { + commonOpts.SeccompProfilePath = SeccompOverridePath + } else { + if !os.IsNotExist(err) { + return errors.Wrapf(err, "can't check if %q exists", SeccompOverridePath) + } + if _, err := os.Stat(SeccompDefaultPath); err != nil { + if !os.IsNotExist(err) { + return errors.Wrapf(err, "can't check if %q exists", SeccompDefaultPath) + } + } else { + commonOpts.SeccompProfilePath = SeccompDefaultPath + } + } + } + return nil +} + +func parseVolumes(volumes []string) error { + if len(volumes) == 0 { + return nil + } + for _, volume := range volumes { + arr := strings.SplitN(volume, ":", 3) + if len(arr) < 2 { + return errors.Errorf("incorrect volume format %q, should be host-dir:ctr-dir[:option]", volume) + } + if err := validateVolumeHostDir(arr[0]); err != nil { + return err + } + if err := validateVolumeCtrDir(arr[1]); err != nil { + return err + } + if len(arr) > 2 { + if err := validateVolumeOpts(arr[2]); err != nil { + return err + } + } + } + return nil +} + +func validateVolumeHostDir(hostDir string) error { + if _, err := os.Stat(hostDir); err != nil { + return errors.Wrapf(err, "error checking path %q", hostDir) + } + return nil +} + +func validateVolumeCtrDir(ctrDir string) error { + if ctrDir[0] != '/' { + return errors.Errorf("invalid container directory path %q", ctrDir) + } + return nil +} + +func validateVolumeOpts(option string) error { + var foundRootPropagation, foundRWRO, foundLabelChange int + options := strings.Split(option, ",") + for _, opt := range options { + switch opt { + case "rw", "ro": + if foundRWRO > 1 { + return errors.Errorf("invalid options %q, can only specify 1 'rw' or 'ro' option", option) + } + foundRWRO++ + case "z", "Z": + if foundLabelChange > 1 { + return errors.Errorf("invalid options %q, can only specify 1 'z' or 'Z' option", option) + } + foundLabelChange++ + case "private", "rprivate", "shared", "rshared", "slave", "rslave": + if foundRootPropagation > 1 { + return errors.Errorf("invalid options %q, can only specify 1 '[r]shared', '[r]private' or '[r]slave' option", option) + } + foundRootPropagation++ + default: + return errors.Errorf("invalid option type %q", option) + } + } + return nil +} + +// validateExtraHost validates that the specified string is a valid extrahost and returns it. +// ExtraHost is in the form of name:ip where the ip has to be a valid ip (ipv4 or ipv6). +// for add-host flag +func validateExtraHost(val string) error { + // allow for IPv6 addresses in extra hosts by only splitting on first ":" + arr := strings.SplitN(val, ":", 2) + if len(arr) != 2 || len(arr[0]) == 0 { + return fmt.Errorf("bad format for add-host: %q", val) + } + if _, err := validateIPAddress(arr[1]); err != nil { + return fmt.Errorf("invalid IP address in add-host: %q", arr[1]) + } + return nil +} + +// validateIPAddress validates an Ip address. +// for dns, ip, and ip6 flags also +func validateIPAddress(val string) (string, error) { + var ip = net.ParseIP(strings.TrimSpace(val)) + if ip != nil { + return ip.String(), nil + } + return "", fmt.Errorf("%s is not an ip address", val) +} + +// ValidateFlags searches for StringFlags or StringSlice flags that never had +// a value set. This commonly occurs when the CLI mistakenly takes the next +// option and uses it as a value. +func ValidateFlags(c *cli.Context, flags []cli.Flag) error { + re, err := regexp.Compile("^-.+") + if err != nil { + return errors.Wrap(err, "compiling regex failed") + } + + // The --cmd flag can have a following command i.e. --cmd="--help". + // Let's skip this check just for the --cmd flag. + for _, flag := range flags { + switch reflect.TypeOf(flag).String() { + case "cli.StringSliceFlag": + { + f := flag.(cli.StringSliceFlag) + name := strings.Split(f.Name, ",") + if f.Name == "cmd" { + continue + } + val := c.StringSlice(name[0]) + for _, v := range val { + if ok := re.MatchString(v); ok { + return errors.Errorf("option --%s requires a value", name[0]) + } + } + } + case "cli.StringFlag": + { + f := flag.(cli.StringFlag) + name := strings.Split(f.Name, ",") + if f.Name == "cmd" { + continue + } + val := c.String(name[0]) + if ok := re.MatchString(val); ok { + return errors.Errorf("option --%s requires a value", name[0]) + } + } + } + } + return nil +} + +// SystemContextFromOptions returns a SystemContext populated with values +// per the input parameters provided by the caller for the use in authentication. +func SystemContextFromOptions(c *cli.Context) (*types.SystemContext, error) { + ctx := &types.SystemContext{ + DockerCertPath: c.String("cert-dir"), + } + if c.IsSet("tls-verify") { + ctx.DockerInsecureSkipTLSVerify = !c.BoolT("tls-verify") + } + if c.IsSet("creds") { + var err error + ctx.DockerAuthConfig, err = getDockerAuth(c.String("creds")) + if err != nil { + return nil, err + } + } + if c.IsSet("signature-policy") { + ctx.SignaturePolicyPath = c.String("signature-policy") + } + if c.IsSet("authfile") { + ctx.AuthFilePath = c.String("authfile") + } + if c.GlobalIsSet("registries-conf") { + ctx.SystemRegistriesConfPath = c.GlobalString("registries-conf") + } + if c.GlobalIsSet("registries-conf-dir") { + ctx.RegistriesDirPath = c.GlobalString("registries-conf-dir") + } + return ctx, nil +} + +func parseCreds(creds string) (string, string) { + if creds == "" { + return "", "" + } + up := strings.SplitN(creds, ":", 2) + if len(up) == 1 { + return up[0], "" + } + if up[0] == "" { + return "", up[1] + } + return up[0], up[1] +} + +func getDockerAuth(creds string) (*types.DockerAuthConfig, error) { + username, password := parseCreds(creds) + if username == "" { + fmt.Print("Username: ") + fmt.Scanln(&username) + } + if password == "" { + fmt.Print("Password: ") + termPassword, err := terminal.ReadPassword(0) + if err != nil { + return nil, errors.Wrapf(err, "could not read password from terminal") + } + password = string(termPassword) + } + + return &types.DockerAuthConfig{ + Username: username, + Password: password, + }, nil +} diff --git a/vendor/github.com/projectatomic/buildah/pull.go b/vendor/github.com/projectatomic/buildah/pull.go new file mode 100644 index 000000000..9b8578651 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/pull.go @@ -0,0 +1,186 @@ +package buildah + +import ( + "context" + "strings" + + cp "github.com/containers/image/copy" + "github.com/containers/image/docker/reference" + tarfile "github.com/containers/image/docker/tarfile" + ociarchive "github.com/containers/image/oci/archive" + "github.com/containers/image/signature" + is "github.com/containers/image/storage" + "github.com/containers/image/transports" + "github.com/containers/image/transports/alltransports" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/pkg/errors" + "github.com/projectatomic/buildah/util" + "github.com/sirupsen/logrus" +) + +func localImageNameForReference(ctx context.Context, store storage.Store, srcRef types.ImageReference, spec string) (string, error) { + if srcRef == nil { + return "", errors.Errorf("reference to image is empty") + } + split := strings.SplitN(spec, ":", 2) + file := split[len(split)-1] + var name string + switch srcRef.Transport().Name() { + case util.DockerArchive: + tarSource, err := tarfile.NewSourceFromFile(file) + if err != nil { + return "", err + } + manifest, err := tarSource.LoadTarManifest() + if err != nil { + return "", errors.Errorf("error retrieving manifest.json: %v", err) + } + // to pull the first image stored in the tar file + if len(manifest) == 0 { + // use the hex of the digest if no manifest is found + name, err = getImageDigest(ctx, srcRef, nil) + if err != nil { + return "", err + } + } else { + if len(manifest[0].RepoTags) > 0 { + name = manifest[0].RepoTags[0] + } else { + // If the input image has no repotags, we need to feed it a dest anyways + name, err = getImageDigest(ctx, srcRef, nil) + if err != nil { + return "", err + } + } + } + case util.OCIArchive: + // retrieve the manifest from index.json to access the image name + manifest, err := ociarchive.LoadManifestDescriptor(srcRef) + if err != nil { + return "", errors.Wrapf(err, "error loading manifest for %q", srcRef) + } + if manifest.Annotations == nil || manifest.Annotations["org.opencontainers.image.ref.name"] == "" { + return "", errors.Errorf("error, archive doesn't have a name annotation. Cannot store image with no name") + } + name = manifest.Annotations["org.opencontainers.image.ref.name"] + case util.DirTransport: + // supports pull from a directory + name = split[1] + // remove leading "/" + if name[:1] == "/" { + name = name[1:] + } + default: + ref := srcRef.DockerReference() + if ref == nil { + name = srcRef.StringWithinTransport() + _, err := is.Transport.ParseStoreReference(store, name) + if err == nil { + return name, nil + } + if strings.LastIndex(name, "/") != -1 { + name = name[strings.LastIndex(name, "/")+1:] + _, err = is.Transport.ParseStoreReference(store, name) + if err == nil { + return name, nil + } + } + return "", errors.Errorf("reference to image %q is not a named reference", transports.ImageName(srcRef)) + } + + if named, ok := ref.(reference.Named); ok { + name = named.Name() + if namedTagged, ok := ref.(reference.NamedTagged); ok { + name = name + ":" + namedTagged.Tag() + } + if canonical, ok := ref.(reference.Canonical); ok { + name = name + "@" + canonical.Digest().String() + } + } + } + + if _, err := is.Transport.ParseStoreReference(store, name); err != nil { + return "", errors.Wrapf(err, "error parsing computed local image name %q", name) + } + return name, nil +} + +func pullImage(ctx context.Context, store storage.Store, imageName string, options BuilderOptions, sc *types.SystemContext) (types.ImageReference, error) { + spec := imageName + srcRef, err := alltransports.ParseImageName(spec) + if err != nil { + if options.Transport == "" { + return nil, errors.Wrapf(err, "error parsing image name %q", spec) + } + transport := options.Transport + if transport != DefaultTransport { + transport = transport + ":" + } + spec = transport + spec + srcRef2, err2 := alltransports.ParseImageName(spec) + if err2 != nil { + return nil, errors.Wrapf(err2, "error parsing image name %q", spec) + } + srcRef = srcRef2 + } + + destName, err := localImageNameForReference(ctx, store, srcRef, spec) + if err != nil { + return nil, errors.Wrapf(err, "error computing local image name for %q", transports.ImageName(srcRef)) + } + if destName == "" { + return nil, errors.Errorf("error computing local image name for %q", transports.ImageName(srcRef)) + } + + destRef, err := is.Transport.ParseStoreReference(store, destName) + if err != nil { + return nil, errors.Wrapf(err, "error parsing image name %q", destName) + } + + img, err := srcRef.NewImageSource(ctx, sc) + if err != nil { + return nil, errors.Wrapf(err, "error initializing %q as an image source", spec) + } + img.Close() + + policy, err := signature.DefaultPolicy(sc) + if err != nil { + return nil, errors.Wrapf(err, "error obtaining default signature policy") + } + + policyContext, err := signature.NewPolicyContext(policy) + if err != nil { + return nil, errors.Wrapf(err, "error creating new signature policy context") + } + + defer func() { + if err2 := policyContext.Destroy(); err2 != nil { + logrus.Debugf("error destroying signature policy context: %v", err2) + } + }() + + logrus.Debugf("copying %q to %q", spec, destName) + + err = cp.Image(ctx, policyContext, destRef, srcRef, getCopyOptions(options.ReportWriter, options.SystemContext, nil, "")) + if err == nil { + return destRef, nil + } + return nil, err +} + +// getImageDigest creates an image object and uses the hex value of the digest as the image ID +// for parsing the store reference +func getImageDigest(ctx context.Context, src types.ImageReference, sc *types.SystemContext) (string, error) { + newImg, err := src.NewImage(ctx, sc) + if err != nil { + return "", err + } + defer newImg.Close() + + digest := newImg.ConfigInfo().Digest + if err = digest.Validate(); err != nil { + return "", errors.Wrapf(err, "error getting config info") + } + return "@" + digest.Hex(), nil +} diff --git a/vendor/github.com/projectatomic/buildah/run.go b/vendor/github.com/projectatomic/buildah/run.go new file mode 100644 index 000000000..12312f6a4 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/run.go @@ -0,0 +1,479 @@ +package buildah + +import ( + "bufio" + "encoding/json" + "fmt" + "io" + "io/ioutil" + "os" + "os/exec" + "path/filepath" + "strings" + + "github.com/containers/storage/pkg/ioutils" + "github.com/docker/docker/profiles/seccomp" + units "github.com/docker/go-units" + digest "github.com/opencontainers/go-digest" + "github.com/opencontainers/runtime-spec/specs-go" + "github.com/opencontainers/runtime-tools/generate" + "github.com/opencontainers/selinux/go-selinux/label" + "github.com/pkg/errors" + "github.com/sirupsen/logrus" + "golang.org/x/crypto/ssh/terminal" +) + +const ( + // DefaultWorkingDir is used if none was specified. + DefaultWorkingDir = "/" + // DefaultRuntime is the default command to use to run the container. + DefaultRuntime = "runc" +) + +const ( + // DefaultTerminal indicates that this Run invocation should be + // connected to a pseudoterminal if we're connected to a terminal. + DefaultTerminal = iota + // WithoutTerminal indicates that this Run invocation should NOT be + // connected to a pseudoterminal. + WithoutTerminal + // WithTerminal indicates that this Run invocation should be connected + // to a pseudoterminal. + WithTerminal +) + +// RunOptions can be used to alter how a command is run in the container. +type RunOptions struct { + // Hostname is the hostname we set for the running container. + Hostname string + // Runtime is the name of the command to run. It should accept the same arguments that runc does. + Runtime string + // Args adds global arguments for the runtime. + Args []string + // Mounts are additional mount points which we want to provide. + Mounts []specs.Mount + // Env is additional environment variables to set. + Env []string + // User is the user as whom to run the command. + User string + // WorkingDir is an override for the working directory. + WorkingDir string + // Shell is default shell to run in a container. + Shell string + // Cmd is an override for the configured default command. + Cmd []string + // Entrypoint is an override for the configured entry point. + Entrypoint []string + // NetworkDisabled puts the container into its own network namespace. + NetworkDisabled bool + // Terminal provides a way to specify whether or not the command should + // be run with a pseudoterminal. By default (DefaultTerminal), a + // terminal is used if os.Stdout is connected to a terminal, but that + // decision can be overridden by specifying either WithTerminal or + // WithoutTerminal. + Terminal int + // Quiet tells the run to turn off output to stdout. + Quiet bool +} + +func addRlimits(ulimit []string, g *generate.Generator) error { + var ( + ul *units.Ulimit + err error + ) + + for _, u := range ulimit { + if ul, err = units.ParseUlimit(u); err != nil { + return errors.Wrapf(err, "ulimit option %q requires name=SOFT:HARD, failed to be parsed", u) + } + + g.AddProcessRlimits("RLIMIT_"+strings.ToUpper(ul.Name), uint64(ul.Hard), uint64(ul.Soft)) + } + return nil +} + +func addHosts(hosts []string, w io.Writer) error { + buf := bufio.NewWriter(w) + for _, host := range hosts { + fmt.Fprintln(buf, host) + } + return buf.Flush() +} + +func addHostsToFile(hosts []string, filename string) error { + if len(hosts) == 0 { + return nil + } + file, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY, os.ModeAppend) + if err != nil { + return err + } + defer file.Close() + return addHosts(hosts, file) +} + +func addCommonOptsToSpec(commonOpts *CommonBuildOptions, g *generate.Generator) error { + // RESOURCES - CPU + if commonOpts.CPUPeriod != 0 { + g.SetLinuxResourcesCPUPeriod(commonOpts.CPUPeriod) + } + if commonOpts.CPUQuota != 0 { + g.SetLinuxResourcesCPUQuota(commonOpts.CPUQuota) + } + if commonOpts.CPUShares != 0 { + g.SetLinuxResourcesCPUShares(commonOpts.CPUShares) + } + if commonOpts.CPUSetCPUs != "" { + g.SetLinuxResourcesCPUCpus(commonOpts.CPUSetCPUs) + } + if commonOpts.CPUSetMems != "" { + g.SetLinuxResourcesCPUMems(commonOpts.CPUSetMems) + } + + // RESOURCES - MEMORY + if commonOpts.Memory != 0 { + g.SetLinuxResourcesMemoryLimit(commonOpts.Memory) + } + if commonOpts.MemorySwap != 0 { + g.SetLinuxResourcesMemorySwap(commonOpts.MemorySwap) + } + + if commonOpts.CgroupParent != "" { + g.SetLinuxCgroupsPath(commonOpts.CgroupParent) + } + + if err := addRlimits(commonOpts.Ulimit, g); err != nil { + return err + } + if err := addHostsToFile(commonOpts.AddHost, "/etc/hosts"); err != nil { + return err + } + + logrus.Debugln("Resources:", commonOpts) + return nil +} + +func (b *Builder) setupMounts(mountPoint string, spec *specs.Spec, optionMounts []specs.Mount, bindFiles, builtinVolumes, volumeMounts []string, shmSize string) error { + // The passed-in mounts matter the most to us. + mounts := make([]specs.Mount, len(optionMounts)) + copy(mounts, optionMounts) + haveMount := func(destination string) bool { + for _, mount := range mounts { + if mount.Destination == destination { + // Already have something to mount there. + return true + } + } + return false + } + // Add mounts from the generated list, unless they conflict. + for _, specMount := range spec.Mounts { + if specMount.Destination == "/dev/shm" { + specMount.Options = []string{"nosuid", "noexec", "nodev", "mode=1777", "size=" + shmSize} + } + if haveMount(specMount.Destination) { + // Already have something to mount there, so skip this one. + continue + } + mounts = append(mounts, specMount) + } + // Add bind mounts for important files, unless they conflict. + for _, boundFile := range bindFiles { + if haveMount(boundFile) { + // Already have something to mount there, so skip this one. + continue + } + mounts = append(mounts, specs.Mount{ + Source: boundFile, + Destination: boundFile, + Type: "bind", + Options: []string{"rbind", "ro"}, + }) + } + + cdir, err := b.store.ContainerDirectory(b.ContainerID) + if err != nil { + return errors.Wrapf(err, "error determining work directory for container %q", b.ContainerID) + } + + // Add secrets mounts + mountsFiles := []string{OverrideMountsFile, b.DefaultMountsFilePath} + for _, file := range mountsFiles { + secretMounts, err := secretMounts(file, b.MountLabel, cdir) + if err != nil { + logrus.Warn("error mounting secrets, skipping...") + continue + } + for _, mount := range secretMounts { + if haveMount(mount.Destination) { + continue + } + mounts = append(mounts, mount) + } + } + // Add temporary copies of the contents of volume locations at the + // volume locations, unless we already have something there. + for _, volume := range builtinVolumes { + if haveMount(volume) { + // Already mounting something there, no need to bother. + continue + } + subdir := digest.Canonical.FromString(volume).Hex() + volumePath := filepath.Join(cdir, "buildah-volumes", subdir) + // If we need to, initialize the volume path's initial contents. + if _, err = os.Stat(volumePath); os.IsNotExist(err) { + if err = os.MkdirAll(volumePath, 0755); err != nil { + return errors.Wrapf(err, "error creating directory %q for volume %q in container %q", volumePath, volume, b.ContainerID) + } + if err = label.Relabel(volumePath, b.MountLabel, false); err != nil { + return errors.Wrapf(err, "error relabeling directory %q for volume %q in container %q", volumePath, volume, b.ContainerID) + } + srcPath := filepath.Join(mountPoint, volume) + if err = copyWithTar(srcPath, volumePath); err != nil && !os.IsNotExist(err) { + return errors.Wrapf(err, "error populating directory %q for volume %q in container %q using contents of %q", volumePath, volume, b.ContainerID, srcPath) + } + + } + // Add the bind mount. + mounts = append(mounts, specs.Mount{ + Source: volumePath, + Destination: volume, + Type: "bind", + Options: []string{"bind"}, + }) + } + // Bind mount volumes given by the user at execution + var options []string + for _, i := range volumeMounts { + spliti := strings.Split(i, ":") + if len(spliti) > 2 { + options = strings.Split(spliti[2], ",") + } + if haveMount(spliti[1]) { + continue + } + options = append(options, "rbind") + var foundrw, foundro, foundz, foundZ bool + var rootProp string + for _, opt := range options { + switch opt { + case "rw": + foundrw = true + case "ro": + foundro = true + case "z": + foundz = true + case "Z": + foundZ = true + case "private", "rprivate", "slave", "rslave", "shared", "rshared": + rootProp = opt + } + } + if !foundrw && !foundro { + options = append(options, "rw") + } + if foundz { + if err := label.Relabel(spliti[0], spec.Linux.MountLabel, true); err != nil { + return errors.Wrapf(err, "relabel failed %q", spliti[0]) + } + } + if foundZ { + if err := label.Relabel(spliti[0], spec.Linux.MountLabel, false); err != nil { + return errors.Wrapf(err, "relabel failed %q", spliti[0]) + } + } + if rootProp == "" { + options = append(options, "private") + } + + mounts = append(mounts, specs.Mount{ + Destination: spliti[1], + Type: "bind", + Source: spliti[0], + Options: options, + }) + } + // Set the list in the spec. + spec.Mounts = mounts + return nil +} + +// Run runs the specified command in the container's root filesystem. +func (b *Builder) Run(command []string, options RunOptions) error { + var user specs.User + path, err := ioutil.TempDir(os.TempDir(), Package) + if err != nil { + return err + } + logrus.Debugf("using %q to hold bundle data", path) + defer func() { + if err2 := os.RemoveAll(path); err2 != nil { + logrus.Errorf("error removing %q: %v", path, err2) + } + }() + g := generate.New() + + for _, envSpec := range append(b.Env(), options.Env...) { + env := strings.SplitN(envSpec, "=", 2) + if len(env) > 1 { + g.AddProcessEnv(env[0], env[1]) + } + } + + if b.CommonBuildOpts == nil { + return errors.Errorf("Invalid format on container you must recreate the container") + } + + if err := addCommonOptsToSpec(b.CommonBuildOpts, &g); err != nil { + return err + } + + if len(command) > 0 { + g.SetProcessArgs(command) + } else { + cmd := b.Cmd() + if len(options.Cmd) > 0 { + cmd = options.Cmd + } + entrypoint := b.Entrypoint() + if len(options.Entrypoint) > 0 { + entrypoint = options.Entrypoint + } + g.SetProcessArgs(append(entrypoint, cmd...)) + } + if options.WorkingDir != "" { + g.SetProcessCwd(options.WorkingDir) + } else if b.WorkDir() != "" { + g.SetProcessCwd(b.WorkDir()) + } + if options.Hostname != "" { + g.SetHostname(options.Hostname) + } else if b.Hostname() != "" { + g.SetHostname(b.Hostname()) + } + g.SetProcessSelinuxLabel(b.ProcessLabel) + g.SetLinuxMountLabel(b.MountLabel) + mountPoint, err := b.Mount(b.MountLabel) + if err != nil { + return err + } + defer func() { + if err2 := b.Unmount(); err2 != nil { + logrus.Errorf("error unmounting container: %v", err2) + } + }() + for _, mp := range []string{ + "/proc/kcore", + "/proc/latency_stats", + "/proc/timer_list", + "/proc/timer_stats", + "/proc/sched_debug", + "/proc/scsi", + "/sys/firmware", + } { + g.AddLinuxMaskedPaths(mp) + } + + for _, rp := range []string{ + "/proc/asound", + "/proc/bus", + "/proc/fs", + "/proc/irq", + "/proc/sys", + "/proc/sysrq-trigger", + } { + g.AddLinuxReadonlyPaths(rp) + } + g.SetRootPath(mountPoint) + switch options.Terminal { + case DefaultTerminal: + g.SetProcessTerminal(terminal.IsTerminal(int(os.Stdout.Fd()))) + case WithTerminal: + g.SetProcessTerminal(true) + case WithoutTerminal: + g.SetProcessTerminal(false) + } + if !options.NetworkDisabled { + if err = g.RemoveLinuxNamespace("network"); err != nil { + return errors.Wrapf(err, "error removing network namespace for run") + } + } + user, err = b.user(mountPoint, options.User) + if err != nil { + return err + } + g.SetProcessUID(user.UID) + g.SetProcessGID(user.GID) + spec := g.Spec() + if spec.Process.Cwd == "" { + spec.Process.Cwd = DefaultWorkingDir + } + if err = os.MkdirAll(filepath.Join(mountPoint, spec.Process.Cwd), 0755); err != nil { + return errors.Wrapf(err, "error ensuring working directory %q exists", spec.Process.Cwd) + } + + //Security Opts + g.SetProcessApparmorProfile(b.CommonBuildOpts.ApparmorProfile) + + // HANDLE SECCOMP + if b.CommonBuildOpts.SeccompProfilePath != "unconfined" { + if b.CommonBuildOpts.SeccompProfilePath != "" { + seccompProfile, err := ioutil.ReadFile(b.CommonBuildOpts.SeccompProfilePath) + if err != nil { + return errors.Wrapf(err, "opening seccomp profile (%s) failed", b.CommonBuildOpts.SeccompProfilePath) + } + seccompConfig, err := seccomp.LoadProfile(string(seccompProfile), spec) + if err != nil { + return errors.Wrapf(err, "loading seccomp profile (%s) failed", b.CommonBuildOpts.SeccompProfilePath) + } + spec.Linux.Seccomp = seccompConfig + } else { + seccompConfig, err := seccomp.GetDefaultProfile(spec) + if err != nil { + return errors.Wrapf(err, "loading seccomp profile (%s) failed", b.CommonBuildOpts.SeccompProfilePath) + } + spec.Linux.Seccomp = seccompConfig + } + } + + cgroupMnt := specs.Mount{ + Destination: "/sys/fs/cgroup", + Type: "cgroup", + Source: "cgroup", + Options: []string{"nosuid", "noexec", "nodev", "relatime", "ro"}, + } + g.AddMount(cgroupMnt) + + bindFiles := []string{"/etc/hosts", "/etc/resolv.conf"} + err = b.setupMounts(mountPoint, spec, options.Mounts, bindFiles, b.Volumes(), b.CommonBuildOpts.Volumes, b.CommonBuildOpts.ShmSize) + if err != nil { + return errors.Wrapf(err, "error resolving mountpoints for container") + } + specbytes, err := json.Marshal(spec) + if err != nil { + return err + } + err = ioutils.AtomicWriteFile(filepath.Join(path, "config.json"), specbytes, 0600) + if err != nil { + return errors.Wrapf(err, "error storing runtime configuration") + } + logrus.Debugf("config = %v", string(specbytes)) + runtime := options.Runtime + if runtime == "" { + runtime = DefaultRuntime + } + args := append(options.Args, "run", "-b", path, Package+"-"+b.ContainerID) + cmd := exec.Command(runtime, args...) + cmd.Dir = mountPoint + cmd.Stdin = os.Stdin + cmd.Stdout = os.Stdout + if options.Quiet { + cmd.Stdout = nil + } + cmd.Stderr = os.Stderr + err = cmd.Run() + if err != nil { + logrus.Debugf("error running runc %v: %v", spec.Process.Args, err) + } + return err +} diff --git a/vendor/github.com/projectatomic/buildah/secrets.go b/vendor/github.com/projectatomic/buildah/secrets.go new file mode 100644 index 000000000..087bf6ba5 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/secrets.go @@ -0,0 +1,198 @@ +package buildah + +import ( + "bufio" + "fmt" + "io/ioutil" + "os" + "path/filepath" + "strings" + + rspec "github.com/opencontainers/runtime-spec/specs-go" + "github.com/opencontainers/selinux/go-selinux/label" + "github.com/pkg/errors" + "github.com/sirupsen/logrus" +) + +var ( + // DefaultMountsFile holds the default mount paths in the form + // "host_path:container_path" + DefaultMountsFile = "/usr/share/containers/mounts.conf" + // OverrideMountsFile holds the default mount paths in the form + // "host_path:container_path" overriden by the user + OverrideMountsFile = "/etc/containers/mounts.conf" +) + +// secretData stores the name of the file and the content read from it +type secretData struct { + name string + data []byte +} + +// saveTo saves secret data to given directory +func (s secretData) saveTo(dir string) error { + path := filepath.Join(dir, s.name) + if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil && !os.IsExist(err) { + return err + } + return ioutil.WriteFile(path, s.data, 0700) +} + +func readAll(root, prefix string) ([]secretData, error) { + path := filepath.Join(root, prefix) + + data := []secretData{} + + files, err := ioutil.ReadDir(path) + if err != nil { + if os.IsNotExist(err) { + return data, nil + } + + return nil, err + } + + for _, f := range files { + fileData, err := readFile(root, filepath.Join(prefix, f.Name())) + if err != nil { + // If the file did not exist, might be a dangling symlink + // Ignore the error + if os.IsNotExist(err) { + continue + } + return nil, err + } + data = append(data, fileData...) + } + + return data, nil +} + +func readFile(root, name string) ([]secretData, error) { + path := filepath.Join(root, name) + + s, err := os.Stat(path) + if err != nil { + return nil, err + } + + if s.IsDir() { + dirData, err := readAll(root, name) + if err != nil { + return nil, err + } + return dirData, nil + } + bytes, err := ioutil.ReadFile(path) + if err != nil { + return nil, err + } + return []secretData{{name: name, data: bytes}}, nil +} + +func getHostSecretData(hostDir string) ([]secretData, error) { + var allSecrets []secretData + hostSecrets, err := readAll(hostDir, "") + if err != nil { + return nil, errors.Wrapf(err, "failed to read secrets from %q", hostDir) + } + return append(allSecrets, hostSecrets...), nil +} + +func getMounts(filePath string) []string { + file, err := os.Open(filePath) + if err != nil { + logrus.Warnf("file %q not found, skipping...", filePath) + return nil + } + defer file.Close() + scanner := bufio.NewScanner(file) + if err = scanner.Err(); err != nil { + logrus.Warnf("error reading file %q, skipping...", filePath) + return nil + } + var mounts []string + for scanner.Scan() { + mounts = append(mounts, scanner.Text()) + } + return mounts +} + +// getHostAndCtrDir separates the host:container paths +func getMountsMap(path string) (string, string, error) { + arr := strings.SplitN(path, ":", 2) + if len(arr) == 2 { + return arr[0], arr[1], nil + } + return "", "", errors.Errorf("unable to get host and container dir") +} + +// secretMount copies the contents of host directory to container directory +// and returns a list of mounts +func secretMounts(filePath, mountLabel, containerWorkingDir string) ([]rspec.Mount, error) { + var mounts []rspec.Mount + defaultMountsPaths := getMounts(filePath) + for _, path := range defaultMountsPaths { + hostDir, ctrDir, err := getMountsMap(path) + if err != nil { + return nil, err + } + // skip if the hostDir path doesn't exist + if _, err = os.Stat(hostDir); os.IsNotExist(err) { + logrus.Warnf("%q doesn't exist, skipping", hostDir) + continue + } + + ctrDirOnHost := filepath.Join(containerWorkingDir, ctrDir) + if err = os.RemoveAll(ctrDirOnHost); err != nil { + return nil, fmt.Errorf("remove container directory failed: %v", err) + } + + if err = os.MkdirAll(ctrDirOnHost, 0755); err != nil { + return nil, fmt.Errorf("making container directory failed: %v", err) + } + + hostDir, err = resolveSymbolicLink(hostDir) + if err != nil { + return nil, err + } + + data, err := getHostSecretData(hostDir) + if err != nil { + return nil, errors.Wrapf(err, "getting host secret data failed") + } + for _, s := range data { + if err := s.saveTo(ctrDirOnHost); err != nil { + return nil, errors.Wrapf(err, "error saving data to container filesystem on host %q", ctrDirOnHost) + } + } + + err = label.Relabel(ctrDirOnHost, mountLabel, false) + if err != nil { + return nil, errors.Wrap(err, "error applying correct labels") + } + + m := rspec.Mount{ + Source: ctrDirOnHost, + Destination: ctrDir, + Type: "bind", + Options: []string{"bind"}, + } + + mounts = append(mounts, m) + } + return mounts, nil +} + +// resolveSymbolicLink resolves a possbile symlink path. If the path is a symlink, returns resolved +// path; if not, returns the original path. +func resolveSymbolicLink(path string) (string, error) { + info, err := os.Lstat(path) + if err != nil { + return "", err + } + if info.Mode()&os.ModeSymlink != os.ModeSymlink { + return path, nil + } + return filepath.EvalSymlinks(path) +} diff --git a/vendor/github.com/projectatomic/buildah/unmount.go b/vendor/github.com/projectatomic/buildah/unmount.go new file mode 100644 index 000000000..e1578bf71 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/unmount.go @@ -0,0 +1,11 @@ +package buildah + +// Unmount unmounts a build container. +func (b *Builder) Unmount() error { + err := b.store.Unmount(b.ContainerID) + if err == nil { + b.MountPoint = "" + err = b.Save() + } + return err +} diff --git a/vendor/github.com/projectatomic/buildah/util.go b/vendor/github.com/projectatomic/buildah/util.go new file mode 100644 index 000000000..33b5b9e83 --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/util.go @@ -0,0 +1,34 @@ +package buildah + +import ( + "github.com/containers/storage/pkg/chrootarchive" + "github.com/containers/storage/pkg/reexec" +) + +var ( + // CopyWithTar defines the copy method to use. + copyWithTar = chrootarchive.NewArchiver(nil).CopyWithTar + copyFileWithTar = chrootarchive.NewArchiver(nil).CopyFileWithTar + untarPath = chrootarchive.NewArchiver(nil).UntarPath +) + +// InitReexec is a wrapper for reexec.Init(). It should be called at +// the start of main(), and if it returns true, main() should return +// immediately. +func InitReexec() bool { + return reexec.Init() +} + +func copyStringStringMap(m map[string]string) map[string]string { + n := map[string]string{} + for k, v := range m { + n[k] = v + } + return n +} + +func copyStringSlice(s []string) []string { + t := make([]string, len(s)) + copy(t, s) + return t +} diff --git a/vendor/github.com/projectatomic/buildah/util/util.go b/vendor/github.com/projectatomic/buildah/util/util.go new file mode 100644 index 000000000..49d6fcc4e --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/util/util.go @@ -0,0 +1,221 @@ +package util + +import ( + "fmt" + "io" + "net/url" + "path" + "strings" + "time" + + "github.com/containers/image/directory" + dockerarchive "github.com/containers/image/docker/archive" + "github.com/containers/image/docker/reference" + ociarchive "github.com/containers/image/oci/archive" + "github.com/containers/image/pkg/sysregistries" + is "github.com/containers/image/storage" + "github.com/containers/image/tarball" + "github.com/containers/image/types" + "github.com/containers/storage" + "github.com/docker/distribution/registry/api/errcode" + "github.com/pkg/errors" + "github.com/sirupsen/logrus" +) + +const ( + minimumTruncatedIDLength = 3 +) + +var ( + // RegistryDefaultPathPrefix contains a per-registry listing of default prefixes + // to prepend to image names that only contain a single path component. + RegistryDefaultPathPrefix = map[string]string{ + "index.docker.io": "library", + "docker.io": "library", + } + // Transports contains the possible transports used for images + Transports = map[string]string{ + dockerarchive.Transport.Name(): "", + ociarchive.Transport.Name(): "", + directory.Transport.Name(): "", + tarball.Transport.Name(): "", + } + // DockerArchive is the transport we prepend to an image name + // when saving to docker-archive + DockerArchive = dockerarchive.Transport.Name() + // OCIArchive is the transport we prepend to an image name + // when saving to oci-archive + OCIArchive = ociarchive.Transport.Name() + // DirTransport is the transport for pushing and pulling + // images to and from a directory + DirTransport = directory.Transport.Name() + // TarballTransport is the transport for importing a tar archive + // and creating a filesystem image + TarballTransport = tarball.Transport.Name() +) + +// ResolveName checks if name is a valid image name, and if that name doesn't include a domain +// portion, returns a list of the names which it might correspond to in the registries. +func ResolveName(name string, firstRegistry string, sc *types.SystemContext, store storage.Store) []string { + if name == "" { + return nil + } + + // Maybe it's a truncated image ID. Don't prepend a registry name, then. + if len(name) >= minimumTruncatedIDLength { + if img, err := store.Image(name); err == nil && img != nil && strings.HasPrefix(img.ID, name) { + // It's a truncated version of the ID of an image that's present in local storage; + // we need to expand the ID. + return []string{img.ID} + } + } + + // If the image is from a different transport + split := strings.SplitN(name, ":", 2) + if len(split) == 2 { + if _, ok := Transports[split[0]]; ok { + return []string{split[1]} + } + } + + // If the image name already included a domain component, we're done. + named, err := reference.ParseNormalizedNamed(name) + if err != nil { + return []string{name} + } + if named.String() == name { + // Parsing produced the same result, so there was a domain name in there to begin with. + return []string{name} + } + if reference.Domain(named) != "" && RegistryDefaultPathPrefix[reference.Domain(named)] != "" { + // If this domain can cause us to insert something in the middle, check if that happened. + repoPath := reference.Path(named) + domain := reference.Domain(named) + defaultPrefix := RegistryDefaultPathPrefix[reference.Domain(named)] + "/" + if strings.HasPrefix(repoPath, defaultPrefix) && path.Join(domain, repoPath[len(defaultPrefix):]) == name { + // Yup, parsing just inserted a bit in the middle, so there was a domain name there to begin with. + return []string{name} + } + } + + // Figure out the list of registries. + registries, err := sysregistries.GetRegistries(sc) + if err != nil { + logrus.Debugf("unable to complete image name %q: %v", name, err) + return []string{name} + } + if sc.DockerInsecureSkipTLSVerify { + if unverifiedRegistries, err := sysregistries.GetInsecureRegistries(sc); err == nil { + registries = append(registries, unverifiedRegistries...) + } + } + + // Create all of the combinations. Some registries need an additional component added, so + // use our lookaside map to keep track of them. If there are no configured registries, at + // least return the name as it was passed to us. + candidates := []string{} + for _, registry := range append([]string{firstRegistry}, registries...) { + if registry == "" { + continue + } + middle := "" + if prefix, ok := RegistryDefaultPathPrefix[registry]; ok && strings.IndexRune(name, '/') == -1 { + middle = prefix + } + candidate := path.Join(registry, middle, name) + candidates = append(candidates, candidate) + } + if len(candidates) == 0 { + candidates = append(candidates, name) + } + return candidates +} + +// ExpandNames takes unqualified names, parses them as image names, and returns +// the fully expanded result, including a tag. Names which don't include a registry +// name will be marked for the most-preferred registry (i.e., the first one in our +// configuration). +func ExpandNames(names []string) ([]string, error) { + expanded := make([]string, 0, len(names)) + for _, n := range names { + name, err := reference.ParseNormalizedNamed(n) + if err != nil { + return nil, errors.Wrapf(err, "error parsing name %q", n) + } + name = reference.TagNameOnly(name) + tag := "" + digest := "" + if tagged, ok := name.(reference.NamedTagged); ok { + tag = ":" + tagged.Tag() + } + if digested, ok := name.(reference.Digested); ok { + digest = "@" + digested.Digest().String() + } + expanded = append(expanded, name.Name()+tag+digest) + } + return expanded, nil +} + +// FindImage locates the locally-stored image which corresponds to a given name. +func FindImage(store storage.Store, image string) (*storage.Image, error) { + var img *storage.Image + ref, err := is.Transport.ParseStoreReference(store, image) + if err == nil { + img, err = is.Transport.GetStoreImage(store, ref) + } + if err != nil { + img2, err2 := store.Image(image) + if err2 != nil { + if ref == nil { + return nil, errors.Wrapf(err, "error parsing reference to image %q", image) + } + return nil, errors.Wrapf(err, "unable to locate image %q", image) + } + img = img2 + } + return img, nil +} + +// AddImageNames adds the specified names to the specified image. +func AddImageNames(store storage.Store, image *storage.Image, addNames []string) error { + names, err := ExpandNames(addNames) + if err != nil { + return err + } + err = store.SetNames(image.ID, append(image.Names, names...)) + if err != nil { + return errors.Wrapf(err, "error adding names (%v) to image %q", names, image.ID) + } + return nil +} + +// GetFailureCause checks the type of the error "err" and returns a new +// error message that reflects the reason of the failure. +// In case err type is not a familiar one the error "defaultError" is returned. +func GetFailureCause(err, defaultError error) error { + switch nErr := errors.Cause(err).(type) { + case errcode.Errors: + return err + case errcode.Error, *url.Error: + return nErr + default: + return defaultError + } +} + +// GetLocalTime discover the UTC offset and then add that to the +// passed in time to arrive at the local time. +func GetLocalTime(localTime time.Time) time.Time { + t := time.Now() + _, offset := t.Local().Zone() + localTime = localTime.Add(time.Second * time.Duration(offset)) + return localTime +} + +// WriteError writes `lastError` into `w` if not nil and return the next error `err` +func WriteError(w io.Writer, err error, lastError error) error { + if lastError != nil { + fmt.Fprintln(w, lastError) + } + return err +} diff --git a/vendor/github.com/projectatomic/buildah/vendor.conf b/vendor/github.com/projectatomic/buildah/vendor.conf new file mode 100644 index 000000000..be0b04e4a --- /dev/null +++ b/vendor/github.com/projectatomic/buildah/vendor.conf @@ -0,0 +1,57 @@ +github.com/BurntSushi/toml master +github.com/Nvveen/Gotty master +github.com/blang/semver master +github.com/containers/image master +github.com/containers/storage master +github.com/docker/distribution 5f6282db7d65e6d72ad7c2cc66310724a57be716 +github.com/docker/docker b8571fd81c7d2223c9ecbf799c693e3ef1daaea9 +github.com/docker/engine-api master +github.com/docker/go-connections 3ede32e2033de7505e6500d6c868c2b9ed9f169d +github.com/docker/go-units 0dadbb0345b35ec7ef35e228dabb8de89a65bf52 +github.com/docker/docker-credential-helpers d68f9aeca33f5fd3f08eeae5e9d175edf4e731d1 +github.com/docker/libtrust aabc10ec26b754e797f9028f4589c5b7bd90dc20 +github.com/fsouza/go-dockerclient master +github.com/ghodss/yaml master +github.com/golang/glog master +github.com/gorilla/context master +github.com/gorilla/mux master +github.com/hashicorp/go-cleanhttp master +github.com/imdario/mergo master +github.com/mattn/go-runewidth master +github.com/mattn/go-shellwords master +github.com/mistifyio/go-zfs master +github.com/moby/moby f8806b18b4b92c5e1980f6e11c917fad201cd73c +github.com/mtrmac/gpgme master +github.com/opencontainers/go-digest aa2ec055abd10d26d539eb630a92241b781ce4bc +github.com/opencontainers/image-spec v1.0.0 +github.com/opencontainers/runc master +github.com/opencontainers/runtime-spec v1.0.0 +github.com/opencontainers/runtime-tools master +github.com/opencontainers/selinux b29023b86e4a69d1b46b7e7b4e2b6fda03f0b9cd +github.com/openshift/imagebuilder master +github.com/ostreedev/ostree-go aeb02c6b6aa2889db3ef62f7855650755befd460 +github.com/pborman/uuid master +github.com/pkg/errors master +github.com/sirupsen/logrus master +github.com/syndtr/gocapability master +github.com/tchap/go-patricia master +github.com/urfave/cli master +github.com/vbatts/tar-split v0.10.2 +golang.org/x/crypto master +golang.org/x/net master +golang.org/x/sys master +golang.org/x/text master +gopkg.in/cheggaaa/pb.v1 v1.0.13 +gopkg.in/yaml.v2 cd8b52f8269e0feb286dfeef29f8fe4d5b397e0b +k8s.io/apimachinery master +k8s.io/client-go master +k8s.io/kubernetes master +github.com/hashicorp/go-multierror master +github.com/hashicorp/errwrap master +github.com/xeipuuv/gojsonschema master +github.com/xeipuuv/gojsonreference master +github.com/containerd/continuity master +github.com/gogo/protobuf master +github.com/xeipuuv/gojsonpointer master +github.com/pquerna/ffjson d49c2bc1aa135aad0c6f4fc2056623ec78f5d5ac +github.com/projectatomic/libpod master |