pkg/hooks: Version the hook structure and add 1.0.0 hooks

This shifts the matching logic out of libpod/container_internal and
into the hook package, where we can reuse it after vendoring into
CRI-O.  It also adds unit tests with almost-complete coverage.  Now
libpod is even more isolated from the hook internals, which makes it
fairly straightforward to bump the hook config file to 1.0.0.  I've
dubbed the old format 0.1.0, although it doesn't specify an explicit
version.  Motivation for some of my changes with 1.0.0:

* Add an explicit version field.  This will make any future JSON
  structure migrations more straightforward by avoiding the need for
  version-guessing heuristics.

* Collect the matching properties in a new When sub-structure.  This
  makes the root Hook structure easier to understand, because you
  don't have to read over all the matching properties when wrapping
  your head around Hook.

* Replace the old 'hook' and 'arguments' with a direct embedding of
  the runtime-spec's hook structure.  This provides access to
  additional upstream properties (args[0], env, and timeout) and
  avoids the complication of a CRI-O-specific analog structure.

* Add a 'when.always' property.  You can usually accomplish this
  effect in another way (e.g. when.commands = [".*"]), but having a
  boolean explicitly for this use-case makes for easier reading and
  writing.

* Replace the previous annotations array with an annotations map.  The
  0.1.0 approach matched only the values regardless of key, and that
  seems unreliable.

* Replace 'cmds' with 'when.commands', because while there are a few
  ways to abbreviate "commands", there's only one way to write it out
  in full ;).  This gives folks one less thing to remember when
  writing hook JSON.

* Replace the old "inject if any specified condition matches" with
  "inject if all specified conditions match".  This allows for more
  precise targeting.  Users that need more generous targeting can
  recover the previous behavior by creating a separate 1.0.0 hook file
  for each specified 0.1.0 condition.

I've added doc-compat support for the various pluralizations of the
0.1.0 properties.  Previously, the docs and code were not in
agreement.  More on this particular facet in [1].

I've updated the docs to point out that the annotations being matched
are the OCI config annotations.  This differs from CRI-O, where the
annotations used are the Kubernetes-supplied annotations [2,3].  For
example, io.kubernetes.cri-o.Volumes [4] is part of CRI-O's runtime
config annotations [5], but not part of the Kubernetes-supplied
annotations CRI-O uses for matching hooks.

The Monitor method supports the CRI-O use-case [6].  podman doesn't
need it directly, but CRI-O will need it when we vendor this package
there.

I've used nvidia-container-runtime-hook for the annotation examples
because Dan mentioned the Nvidia folks as the motivation behind
annotation matching.  The environment variables are documented in [7].
The 0.1.0 hook config, which does not allow for environment variables,
only works because runc currently leaks the host environment into the
hooks [8].  I haven't been able to find documentation for their usual
annotation trigger or hook-install path, so I'm just guessing there.

[1]: https://github.com/kubernetes-incubator/cri-o/pull/1235
[2]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L760
[3]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L772
[4]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/pkg/annotations/annotations.go#L97-L98
[5]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L830-L834
[6]: https://github.com/kubernetes-incubator/cri-o/pull/1345/
[7]: https://github.com/NVIDIA/nvidia-container-runtime/tree/v1.3.0-1#environment-variables-oci-spec
[8]: https://github.com/opencontainers/runc/pull/1738

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #686
Approved by: mheon

1 files changed, 175 insertions, 0 deletions

diff --git a/pkg/hooks/README.md b/pkg/hooks/README.md
new file mode 100644
index 000000000..e47000a09
--- /dev/null
+++ b/pkg/hooks/README.md
@@ -0,0 +1,175 @@
+# OCI Hooks Configuration
+
+For POSIX platforms, the [OCI runtime configuration][runtime-spec] supports [hooks][spec-hooks] for configuring custom actions related to the life cycle of the container.
+The way you enable the hooks above is by editing the OCI runtime configuration before running the OCI runtime (e.g. [`runc`][runc]).
+CRI-O and `podman create` create the OCI configuration for you, and this documentation allows developers to configure them to set their intended hooks.
+
+One problem with hooks is that the runtime actually stalls execution of the container before running the hooks and stalls completion of the container, until all hooks complete.
+This can cause some performance issues.
+Also a lot of hooks just check if certain configuration is set and then exit early, without doing anything.
+For example the [oci-systemd-hook][] only executes if the command is `init` or `systemd`, otherwise it just exits.
+This means if we automatically enabled all hooks, every container would have to execute `oci-systemd-hook`, even if they don't run systemd inside of the container.
+Performance would also suffer if we exectuted each hook at each stage ([pre-start][], [post-start][], and [post-stop][]).
+
+## Notational Conventions
+
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in [RFC 2119][rfc2119].
+
+## JSON Definition
+
+This package reads all [JSON][] files (ending with a `.json` extention) from a series of hook directories.
+For both `crio` and `podman`, hooks are read from `/usr/share/containers/oci/hooks.d/*.json`.
+
+For `crio`, hook JSON is also read from `/etc/containers/oci/hooks.d/*.json`.
+If files of with the same name exist in both directories, the one in `/etc/containers/oci/hooks.d` takes precedence.
+
+Each JSON file should contain an object with the following properties:
+
+### 1.0.0 Hook Schema
+
+* **`version`** (REQUIRED, string) Sets the hook-definition version.
+    For this schema version, the value MUST be 1.0.0.
+* **`hook`** (REQUIRED, object) The hook to inject, with the [hook-entry schema][spec-hooks] defined by the 1.0.1 OCI Runtime Specification.
+* **`when`** (REQUIRED, object) Conditions under which the hook is injected.
+    The following properties can be specified:
+
+    * **`always`** (OPTIONAL, boolean) If set `true`, this condition matches.
+    * **`annotations`** (OPTIONAL, object) If all `annotations` key/value pairs match a key/value pair from the [configured annotations][spec-annotations], this condition matches.
+        Both keys and values MUST be [POSIX extended regular expressions][POSIX-ERE].
+    * **`commands`** (OPTIONAL, array of strings) If the configured [`process.args[0]`][spec-process] matches an entry, this condition matches.
+        Entries MUST be [POSIX extended regular expressions][POSIX-ERE].
+    * **`hasBindMounts`** (OPTIONAL, boolean) If `hasBindMounts` is true and the caller requested host-to-container bind mounts (beyond those that CRI-O or libpod use by default), this condition matches.
+* **`stages`** (REQUIRED, array of strings) Stages when the hook MUST be injected.
+    Entries MUST be chosen from the 1.0.1 OCI Runtime Specification [hook stages][spec-hooks].
+
+If *all* of the conditions set in `when` match, then the `hook` MUST be injected for the stages set in `stages`.
+
+#### Example
+
+The following configuration injects [`oci-systemd-hook`][oci-systemd-hook] in the [pre-start][] and [post-stop][] stages if [`process.args[0]`][spec-process] ends with `/init` or `/systemd`:
+
+```console
+$ cat /etc/containers/oci/hooks.d/oci-systemd-hook.json
+{
+  "version": "1.0.0",
+  "hook": {
+    "path": "/usr/libexec/oci/hooks.d/oci-systemd-hook"
+  }
+  "when": {
+    "args": [".*/init$" , ".*/systemd$"],
+  },
+  "stages": ["prestart", "poststop"]
+}
+```
+
+The following example injects [`oci-umount --debug`][oci-umount] in the [pre-start][] phase if the container is configured to bind-mount host directories into the container.
+
+```console
+$ cat /etc/containers/oci/hooks.d/oci-umount.json
+{
+  "version": "1.0.0",
+  "hook": {
+    "path": "/usr/libexec/oci/hooks.d/oci-umount",
+    "args": ["oci-umount", "--debug"],
+  }
+  "when": {
+    "hasBindMounts": true,
+  },
+  "stages": ["prestart"]
+}
+```
+
+The following example injects [`nvidia-container-runtime-hook prestart`][nvidia-container-runtime-hook] with particular environment variables in the [pre-start][] phase if the container is configured with an `annotations` entry whose key matches `^com\.example\.department$` and whose value matches `.*fluid-dynamics.*`.
+
+```console
+$ cat /etc/containers/oci/hooks.d/nvidia.json
+{
+  "hook": {
+    "path": "/usr/sbin/nvidia-container-runtime-hook",
+    "args": ["nvidia-container-runtime-hook", "prestart"],
+    "env": [
+      "NVIDIA_REQUIRE_CUDA=cuda>=9.1",
+      "NVIDIA_VISIBLE_DEVICES=GPU-fef8089b"
+    ]
+  },
+  "when": {
+    "annotations": {
+      "^com\.example\.department$": ".*fluid-dynamics$"
+    }
+  },
+  "stages": ["prestart"]
+}
+```
+
+### 0.1.0 Hook Schema
+
+Previous versions of CRI-O and libpod supported the 0.1.0 hook schema:
+
+* **`hook`** (REQUIRED, string) Sets [`path`][spec-hooks] in the injected hook.
+* **`arguments`** (OPTIONAL, array of strings) Additional arguments to pass to the hook.
+    The injected hook's [`args`][spec-hooks] is `hook` with `arguments` appended.
+* **`stages`** (REQUIRED, array of strings) Stages when the hook MUST be injected.
+    `stage` is an allowed synonym for this property, but you MUST NOT set both `stages` and `stage`.
+    Entries MUST be chosen from:
+    * **`prestart`**, to inject [pre-start][].
+    * **`poststart`**, to inject [post-start][].
+    * **`poststop`**, to inject [post-stop][].
+* **`cmds`** (OPTIONAL, array of strings) The hook MUST be injected if the configured [`process.args[0]`][spec-process] matches an entry.
+    `cmd` is an allowed synonym for this property, but you MUST NOT set both `cmds` and `cmd`.
+    Entries MUST be [POSIX extended regular expressions][POSIX-ERE].
+* **`annotations`** (OPTIONAL, array of strings) The hook MUST be injected if an `annotations` entry matches a value from the [configured annotations][spec-annotations].
+    `annotation` is an allowed synonym for this property, but you MUST NOT set both `annotations` and `annotation`.
+    Entries MUST be [POSIX extended regular expressions][POSIX-ERE].
+* **`hasbindmounts`** (OPTIONAL, boolean) The hook MUST be injected if `hasBindMounts` is true and the caller requested host-to-container bind mounts (beyond those that CRI-O or libpod use by default).
+
+#### Example
+
+The following configuration injects [`oci-systemd-hook`][oci-systemd-hook] in the [pre-start][] and [post-stop][] stages if [`process.args[0]`][spec-process] ends with `/init` or `/systemd`:
+
+```console
+$ cat /etc/containers/oci/hooks.d/oci-systemd-hook.json
+{
+  "cmds": [".*/init$" , ".*/systemd$"],
+  "hook": "/usr/libexec/oci/hooks.d/oci-systemd-hook",
+  "stages": ["prestart", "poststop"]
+}
+```
+
+The following example injects [`oci-umount --debug`][oci-umount] in the [pre-start][] phase if the container is configured to bind-mount host directories into the container.
+
+```console
+$ cat /etc/containers/oci/hooks.d/oci-umount.json
+{
+  "hook": "/usr/libexec/oci/hooks.d/oci-umount",
+  "arguments": ["--debug"],
+  "hasbindmounts": true,
+  "stages": ["prestart"]
+}
+```
+
+The following example injects [`nvidia-container-runtime-hook prestart`][nvidia-container-runtime-hook] in the [pre-start][] phase if the container is configured with an `annotations` entry whose value matches `.*fluid-dynamics.*`.
+
+```console
+$ cat /etc/containers/oci/hooks.d/osystemd-hook.json
+{
+  "hook": "/usr/sbin/nvidia-container-runtime-hook",
+  "arguments": ["prestart"],
+  "annotations: [".*fluid-dynamics.*"],
+  "stages": ["prestart"]
+}
+```
+
+[JSON]: https://tools.ietf.org/html/rfc8259
+[nvidia-container-runtime-hook]: https://github.com/NVIDIA/nvidia-container-runtime/tree/master/hook/nvidia-container-runtime-hook
+[oci-systemd-hook]: https://github.com/projectatomic/oci-systemd-hook
+[oci-umount]: https://github.com/projectatomic/oci-umount
+[POSIX-ERE]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04
+[post-start]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#poststart
+[post-stop]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#poststop
+[pre-start]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#prestart
+[rfc2119]: http://tools.ietf.org/html/rfc2119
+[runc]: https://github.com/opencontainers/runc
+[runtime-spec]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/spec.md
+[spec-annotations]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#annotations
+[spec-hooks]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#posix-platform-hooks
+[spec-process]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config.md#process