Initial commit on compatible API

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Create service command

Use cd cmd/service && go build .

$ systemd-socket-activate -l 8081 cmd/service/service &
$ curl http://localhost:8081/v1.24/images/json

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Correct Makefile

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Two more stragglers

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Report errors back as http headers

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Split out handlers, updated output

Output aligned to docker structures

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Refactored routing, added more endpoints and types

* Encapsulated all the routing information in the handler_* files.
* Added more serviceapi/types, including podman additions. See Info

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Cleaned up code, implemented info content

* Move Content-Type check into serviceHandler
* Custom 404 handler showing the url, mostly for debugging
* Refactored images: better method names and explicit http codes
* Added content to /info
* Added podman fields to Info struct
* Added Container struct

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Add a bunch of endpoints

containers: stop, pause, unpause, wait, rm
images: tag, rmi, create (pull only)

Signed-off-by: baude <bbaude@redhat.com>

Add even more handlers

* Add serviceapi/Error() to improve error handling
* Better support for API return payloads
* Renamed unimplemented to unsupported these are generic endpoints
  we don't intend to ever support.  Swarm broken out since it uses
  different HTTP codes to signal that the node is not in a swarm.
* Added more types
* API Version broken out so it can be validated in the future

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Refactor to introduce ServiceWriter

Signed-off-by: Jhon Honce <jhonce@redhat.com>

populate pods endpoints

/libpod/pods/..

exists, kill, pause, prune, restart, remove, start, stop, unpause

Signed-off-by: baude <bbaude@redhat.com>

Add components to Version, fix Error body

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Add images pull output, fix swarm routes

* docker-py tests/integration/api_client_test.py pass 100%
* docker-py tests/integration/api_image_test.py pass 4/16
+ Test failures include services podman does not support

Signed-off-by: Jhon Honce <jhonce@redhat.com>

pods endpoint submission 2

add create and others; only top and stats is left.

Signed-off-by: baude <bbaude@redhat.com>

Update pull image to work from empty registry

Signed-off-by: Jhon Honce <jhonce@redhat.com>

pod create and container create

first pass at pod and container create.  the container create does not
quite work yet but it is very close.  pod create needs a partial
rewrite.  also broken off the DELETE (rm/rmi) to specific handler funcs.

Signed-off-by: baude <bbaude@redhat.com>

Add docker-py demos, GET .../containers/json

* Update serviceapi/types to reflect libpod not podman
* Refactored removeImage() to provide non-streaming return

Signed-off-by: Jhon Honce <jhonce@redhat.com>

create container part2

finished minimal config needed for create container.  started demo.py
for upcoming talk

Signed-off-by: baude <bbaude@redhat.com>

Stop server after honoring request

* Remove casting for method calls
* Improve WriteResponse()
* Update Container API type to match docker API

Signed-off-by: Jhon Honce <jhonce@redhat.com>

fix namespace assumptions

cleaned up namespace issues with libpod.

Signed-off-by: baude <bbaude@redhat.com>

wip

Signed-off-by: baude <bbaude@redhat.com>

Add sliding window when shutting down server

* Added a Timeout rather than closing down service on each call
* Added gorilla/schema dependency for Decode'ing query parameters
* Improved error handling
* Container logs returned and multiplexed for stdout and stderr
  * .../containers/{name}/logs?stdout=True&stderr=True
* Container stats
  * .../containers/{name}/stats

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Improve error handling

* Add check for at least one std stream required for /containers/{id}/logs
* Add check for state in /containers/{id}/top
* Fill in more fields for /info
* Fixed error checking in service start code

Signed-off-by: Jhon Honce <jhonce@redhat.com>

get rest  of image tests for pass

Signed-off-by: baude <bbaude@redhat.com>

linting our content

Signed-off-by: baude <bbaude@redhat.com>

more linting

Signed-off-by: baude <bbaude@redhat.com>

more linting

Signed-off-by: baude <bbaude@redhat.com>

pruning

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]apiv2 pods

migrate from using args in the url to using a json struct in body for
pod create.

Signed-off-by: baude <bbaude@redhat.com>

fix handler_images prune

prune's api changed slightly to deal with filters.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]enabled base container create tests

enabling the base container create tests which allow us to get more into
the stop, kill, etc tests. many new tests now pass.

Signed-off-by: baude <bbaude@redhat.com>

serviceapi errors: append error message to API message

I dearly hope this is not breaking any other tests but debugging
"Internal Server Error" is not helpful to any user.  In case, it
breaks tests, we can rever the commit - that's why it's a small one.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

serviceAPI: add containers/prune endpoint

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

add `service` make target

Also remove the non-functional sub-Makefile.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

add make targets for testing the service

 * `sudo make run-service` for running the service.

 * `DOCKERPY_TEST="tests/integration/api_container_test.py::ListContainersTest" \
 	make run-docker-py-tests`
   for running a specific tests.  Run all tests by leaving the env
   variable empty.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

Split handlers and server packages

The files were split to help contain bloat. The api/server package will
contain all code related to the functioning of the server while
api/handlers will have all the code related to implementing the end
points.

api/server/register_* will contain the methods for registering
endpoints.  Additionally, they will have the comments for generating the
swagger spec file.

See api/handlers/version.go for a small example handler,
api/handlers/containers.go contains much more complex handlers.

Signed-off-by: Jhon Honce <jhonce@redhat.com>

[CI:DOCS]enabled more tests

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]libpod endpoints

small refactor for libpod inclusion and began adding endpoints.

Signed-off-by: baude <bbaude@redhat.com>

Implement /build and /events

* Include crypto libraries for future ssh work

Signed-off-by: Jhon Honce <jhonce@redhat.com>

[CI:DOCS]more image implementations

convert from using for to query structs among other changes including
new endpoints.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add bindings for golang

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add volume endpoints for libpod

create, inspect, ls, prune, and rm

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]apiv2 healthcheck enablement

wire up container healthchecks for the api.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]Add mount endpoints

via the api, allow ability to mount a container and list container
mounts.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]Add search endpoint

add search endpoint with golang bindings

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]more apiv2 development

misc population of methods, etc

Signed-off-by: baude <bbaude@redhat.com>

rebase cleanup and epoch reset

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add more network endpoints

also, add some initial error handling and convenience functions for
standard endpoints.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]use helper funcs for bindings

use the methods developed to make writing bindings less duplicative and
easier to use.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add return info for prereview

begin to add return info and status codes for errors so that we can
review the apiv2

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]first pass at adding swagger docs for api

Signed-off-by: baude <bbaude@redhat.com>

1 files changed, 363 insertions, 0 deletions

diff --git a/pkg/api/handlers/generic/images.go b/pkg/api/handlers/generic/images.go
new file mode 100644
index 000000000..8029ee861
--- /dev/null
+++ b/pkg/api/handlers/generic/images.go
@@ -0,0 +1,363 @@
+package generic
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+	"strconv"
+	"strings"
+
+	"github.com/containers/buildah"
+	"github.com/containers/image/v5/manifest"
+	"github.com/containers/libpod/libpod"
+	image2 "github.com/containers/libpod/libpod/image"
+	"github.com/containers/libpod/pkg/api/handlers"
+	"github.com/containers/libpod/pkg/api/handlers/utils"
+	"github.com/containers/libpod/pkg/util"
+	"github.com/containers/storage"
+	"github.com/docker/docker/api/types"
+	"github.com/gorilla/mux"
+	"github.com/gorilla/schema"
+	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+)
+
+func ExportImage(w http.ResponseWriter, r *http.Request) {
+	// 200 ok
+	// 500 server
+	runtime := r.Context().Value("runtime").(*libpod.Runtime)
+
+	name := mux.Vars(r)["name"]
+	newImage, err := runtime.ImageRuntime().NewFromLocal(name)
+	if err != nil {
+		utils.ImageNotFound(w, name, errors.Wrapf(err, "Failed to find image %s", name))
+		return
+	}
+	tmpfile, err := ioutil.TempFile("", "api.tar")
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to create tempfile"))
+		return
+	}
+	if err := tmpfile.Close(); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to close tempfile"))
+		return
+	}
+	if err := newImage.Save(r.Context(), name, "docker-archive", tmpfile.Name(), []string{}, false, false); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "failed to save image"))
+		return
+	}
+	rdr, err := os.Open(tmpfile.Name())
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "failed to read the exported tarfile"))
+		return
+	}
+	defer rdr.Close()
+	defer os.Remove(tmpfile.Name())
+	utils.WriteResponse(w, http.StatusOK, rdr)
+}
+
+func PruneImages(w http.ResponseWriter, r *http.Request) {
+	// 200 no error
+	// 500 internal
+	var (
+		dangling bool = true
+		err      error
+	)
+	decoder := r.Context().Value("decoder").(*schema.Decoder)
+	runtime := r.Context().Value("runtime").(*libpod.Runtime)
+
+	query := struct {
+		filters map[string]string
+	}{
+		// This is where you can override the golang default value for one of fields
+	}
+
+	if err := decoder.Decode(&query, r.URL.Query()); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusBadRequest, errors.Wrapf(err, "Failed to parse parameters for %s", r.URL.String()))
+		return
+	}
+
+	// FIXME This is likely wrong due to it not being a map[string][]string
+
+	// until ts is not supported on podman prune
+	if len(query.filters["until"]) > 0 {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "until is not supported yet"))
+		return
+	}
+	// labels are not supported on podman prune
+	if len(query.filters["label"]) > 0 {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "labelis not supported yet"))
+		return
+	}
+
+	if len(query.filters["dangling"]) > 0 {
+		dangling, err = strconv.ParseBool(query.filters["dangling"])
+		if err != nil {
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "processing dangling filter"))
+			return
+		}
+	}
+	idr := []types.ImageDeleteResponseItem{}
+	//
+	// This code needs to be migrated to libpod to work correctly.  I could not
+	// work my around the information docker needs with the existing prune in libpod.
+	//
+	pruneImages, err := runtime.ImageRuntime().GetPruneImages(!dangling, []image2.ImageFilter{})
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to get images to prune"))
+		return
+	}
+	for _, p := range pruneImages {
+		repotags, err := p.RepoTags()
+		if err != nil {
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to get repotags for image"))
+			return
+		}
+		if err := p.Remove(r.Context(), true); err != nil {
+			if errors.Cause(err) == storage.ErrImageUsedByContainer {
+				logrus.Warnf("Failed to prune image %s as it is in use: %v", p.ID(), err)
+				continue
+			}
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "failed to prune image"))
+			return
+		}
+		// newimageevent is not export therefore we cannot record the event. this will be fixed
+		// when the prune is fixed in libpod
+		// defer p.newImageEvent(events.Prune)
+		response := types.ImageDeleteResponseItem{
+			Deleted: fmt.Sprintf("sha256:%s", p.ID()), // I ack this is not ideal
+		}
+		if len(repotags) > 0 {
+			response.Untagged = repotags[0]
+		}
+		idr = append(idr, response)
+	}
+	ipr := types.ImagesPruneReport{
+		ImagesDeleted:  idr,
+		SpaceReclaimed: 1, // TODO we cannot supply this right now
+	}
+	utils.WriteResponse(w, http.StatusOK, handlers.ImagesPruneReport{ImagesPruneReport: ipr})
+}
+
+func CommitContainer(w http.ResponseWriter, r *http.Request) {
+	var (
+		destImage string
+	)
+	decoder := r.Context().Value("decoder").(*schema.Decoder)
+	runtime := r.Context().Value("runtime").(*libpod.Runtime)
+
+	query := struct {
+		author    string
+		changes   string
+		comment   string
+		container string
+		//fromSrc   string  # fromSrc is currently unused
+		pause bool
+		repo  string
+		tag   string
+	}{
+		// This is where you can override the golang default value for one of fields
+	}
+
+	if err := decoder.Decode(&query, r.URL.Query()); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusBadRequest, errors.Wrapf(err, "Failed to parse parameters for %s", r.URL.String()))
+		return
+	}
+	rtc, err := runtime.GetConfig()
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "Decode()"))
+		return
+	}
+	sc := image2.GetSystemContext(rtc.SignaturePolicyPath, "", false)
+	tag := "latest"
+	options := libpod.ContainerCommitOptions{
+		Pause: true,
+	}
+	options.CommitOptions = buildah.CommitOptions{
+		SignaturePolicyPath:   rtc.SignaturePolicyPath,
+		ReportWriter:          os.Stderr,
+		SystemContext:         sc,
+		PreferredManifestType: manifest.DockerV2Schema2MediaType,
+	}
+
+	input := handlers.CreateContainerConfig{}
+	if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "Decode()"))
+		return
+	}
+
+	if len(query.tag) > 0 {
+		tag = query.tag
+	}
+	options.Message = query.comment
+	options.Author = query.author
+	options.Pause = query.pause
+	options.Changes = strings.Fields(query.changes)
+	ctr, err := runtime.LookupContainer(query.container)
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusNotFound, err)
+		return
+	}
+
+	// I know mitr hates this ... but doing for now
+	if len(query.repo) > 1 {
+		destImage = fmt.Sprintf("%s:%s", query.repo, tag)
+	}
+
+	commitImage, err := ctr.Commit(r.Context(), destImage, options)
+	if err != nil && !strings.Contains(err.Error(), "is not running") {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrapf(err, "CommitFailure"))
+		return
+	}
+	utils.WriteResponse(w, http.StatusOK, handlers.IDResponse{ID: commitImage.ID()}) // nolint
+}
+
+func CreateImageFromSrc(w http.ResponseWriter, r *http.Request) {
+	// 200 no error
+	// 404 repo does not exist or no read access
+	// 500 internal
+	decoder := r.Context().Value("decoder").(*schema.Decoder)
+	runtime := r.Context().Value("runtime").(*libpod.Runtime)
+
+	query := struct {
+		fromSrc string
+		changes []string
+	}{
+		// This is where you can override the golang default value for one of fields
+	}
+
+	if err := decoder.Decode(&query, r.URL.Query()); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusBadRequest, errors.Wrapf(err, "Failed to parse parameters for %s", r.URL.String()))
+		return
+	}
+	// fromSrc – Source to import. The value may be a URL from which the image can be retrieved or - to read the image from the request body. This parameter may only be used when importing an image.
+	source := query.fromSrc
+	if source == "-" {
+		f, err := ioutil.TempFile("", "api_load.tar")
+		if err != nil {
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "failed to create tempfile"))
+			return
+		}
+		source = f.Name()
+		if err := handlers.SaveFromBody(f, r); err != nil {
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "failed to write temporary file"))
+		}
+	}
+	iid, err := runtime.Import(r.Context(), source, "", query.changes, "", false)
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to import tarball"))
+		return
+	}
+	tmpfile, err := ioutil.TempFile("", "fromsrc.tar")
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to create tempfile"))
+		return
+	}
+	if err := tmpfile.Close(); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "unable to close tempfile"))
+		return
+	}
+	// Success
+	utils.WriteResponse(w, http.StatusOK, struct {
+		Status         string            `json:"status"`
+		Progress       string            `json:"progress"`
+		ProgressDetail map[string]string `json:"progressDetail"`
+		Id             string            `json:"id"`
+	}{
+		Status:         iid,
+		ProgressDetail: map[string]string{},
+		Id:             iid,
+	})
+
+}
+
+func CreateImageFromImage(w http.ResponseWriter, r *http.Request) {
+	// 200 no error
+	// 404 repo does not exist or no read access
+	// 500 internal
+	decoder := r.Context().Value("decoder").(*schema.Decoder)
+	runtime := r.Context().Value("runtime").(*libpod.Runtime)
+
+	query := struct {
+		fromImage string
+		tag       string
+	}{
+		// This is where you can override the golang default value for one of fields
+	}
+
+	if err := decoder.Decode(&query, r.URL.Query()); err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusBadRequest, errors.Wrapf(err, "Failed to parse parameters for %s", r.URL.String()))
+		return
+	}
+
+	/*
+	   fromImage – Name of the image to pull. The name may include a tag or digest. This parameter may only be used when pulling an image. The pull is cancelled if the HTTP connection is closed.
+	   repo – Repository name given to an image when it is imported. The repo may include a tag. This parameter may only be used when importing an image.
+	   tag – Tag or digest. If empty when pulling an image, this causes all tags for the given image to be pulled.
+	*/
+	fromImage := query.fromImage
+	if len(query.tag) < 1 {
+		fromImage = fmt.Sprintf("%s:%s", fromImage, query.tag)
+	}
+
+	// TODO
+	// We are eating the output right now because we haven't talked about how to deal with multiple responses yet
+	img, err := runtime.ImageRuntime().New(r.Context(), fromImage, "", "", nil, &image2.DockerRegistryOptions{}, image2.SigningOptions{}, nil, util.PullImageMissing)
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, err)
+		return
+	}
+
+	// Success
+	utils.WriteResponse(w, http.StatusOK, struct {
+		Status         string            `json:"status"`
+		Error          string            `json:"error"`
+		Progress       string            `json:"progress"`
+		ProgressDetail map[string]string `json:"progressDetail"`
+		Id             string            `json:"id"`
+	}{
+		Status:         fmt.Sprintf("pulling image (%s) from %s", img.Tag, strings.Join(img.Names(), ", ")),
+		ProgressDetail: map[string]string{},
+		Id:             img.ID(),
+	})
+}
+
+func GetImage(w http.ResponseWriter, r *http.Request) {
+	// 200 no error
+	// 404 no such
+	// 500 internal
+	name := mux.Vars(r)["name"]
+	newImage, err := handlers.GetImage(r, name)
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusNotFound, errors.Wrapf(err, "Failed to find image %s", name))
+		return
+	}
+	inspect, err := handlers.ImageDataToImageInspect(r.Context(), newImage)
+	if err != nil {
+		utils.Error(w, "Server error", http.StatusInternalServerError, errors.Wrapf(err, "Failed to convert ImageData to ImageInspect '%s'", inspect.ID))
+		return
+	}
+	utils.WriteResponse(w, http.StatusOK, inspect)
+}
+
+func GetImages(w http.ResponseWriter, r *http.Request) {
+	// 200 ok
+	// 500 internal
+	images, err := utils.GetImages(w, r)
+	if err != nil {
+		utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "Failed get images"))
+		return
+	}
+	var summaries = make([]*handlers.ImageSummary, len(images))
+	for j, img := range images {
+		is, err := handlers.ImageToImageSummary(img)
+		if err != nil {
+			utils.Error(w, "Something went wrong.", http.StatusInternalServerError, errors.Wrap(err, "Failed transform image summaries"))
+			return
+		}
+		summaries[j] = is
+	}
+	utils.WriteResponse(w, http.StatusOK, summaries)
+}