diff options
Diffstat (limited to 'contrib/cirrus')
26 files changed, 988 insertions, 320 deletions
diff --git a/contrib/cirrus/README.md b/contrib/cirrus/README.md index 0d315c4f5..e175479f1 100644 --- a/contrib/cirrus/README.md +++ b/contrib/cirrus/README.md @@ -5,6 +5,7 @@ Similar to other integrated github CI/CD services, Cirrus utilizes a simple YAML-based configuration/description file: ``.cirrus.yml``. Ref: https://cirrus-ci.org/ + ## Workflow All tasks execute in parallel, unless there are conditions or dependencies @@ -12,24 +13,34 @@ which alter this behavior. Within each task, each script executes in sequence, so long as any previous script exited successfully. The overall state of each task (pass or fail) is set based on the exit status of the last script to execute. -### ``full_vm_testing`` Task -1. Unconditionally, spin up one VM per ``matrix: image_name`` item defined - in ``.cirrus.yml``. Once accessible, ``ssh`` into each VM and run the following - scripts. +### ``gating`` Task + +***N/B: Steps below are performed by automation*** + +1. Launch a purpose-built container in Cirrus's community cluster. + For container image details, please see + [the contributors guide](https://github.com/containers/libpod/blob/master/CONTRIBUTING.md#go-format-and-lint). + +3. ``validate``: Perform standard `make validate` source verification, + Should run for less than a minute or two. + +4. ``lint``: Execute regular `make lint` to check for any code cruft. + Should also run for less than a few minutes. -2. ``setup_environment.sh``: Configure root's ``.bash_profile`` - for all subsequent scripts (each run in a new shell). Any - distribution-specific environment variables are also defined - here. For example, setting tags/flags to use compiling. -3. ``verify_source.sh``: Perform per-distribution source - verification, lint-checking, etc. This acts as a minimal - gate, blocking extended use of VMs when a PR's code or commits - would otherwise not be accepted. Should run for less than a minute. +### ``testing`` Task -4. ``unit_test.sh``: Execute unit-testing, as defined by the ``Makefile``. - This should execute within 10-minutes, but often much faster. +***N/B: Steps below are performed by automation*** + +1. After `gating` passes, spin up one VM per + `matrix: image_name` item. Once accessible, ``ssh`` + into each VM as the `root` user. + +2. ``setup_environment.sh``: Configure root's `.bash_profile` + for all subsequent scripts (each run in a new shell). Any + distribution-specific environment variables are also defined + here. For example, setting tags/flags to use compiling. 5. ``integration_test.sh``: Execute integration-testing. This is much more involved, and relies on access to external @@ -37,39 +48,163 @@ task (pass or fail) is set based on the exit status of the last script to execut Total execution time is capped at 2-hours (includes all the above) but this script normally completes in less than an hour. -### ``build_vm_images`` Task -1. When a PR is merged (``$CIRRUS_BRANCH`` == ``master``), run another - round of the ``full_vm_testing`` task (above). +### ``optional_testing`` Task + +***N/B: Steps below are performed by automation*** + +1. Optionally executes in parallel with ``testing``. Requires + **prior** to job-start, the magic string ``***CIRRUS: SYSTEM TEST***`` + is found in the pull-request *description*. The *description* is the first + text-box under the main *summary* line in the github WebUI. -2. After confirming the tests all pass post-merge, spin up a special VM - capable of communicating with the GCE API. Once accessible, ``ssh`` into - the special VM and run the following scripts. +2. ``setup_environment.sh``: Same as for other tasks. -3. ``setup_environment.sh``: Configure root's ``.bash_profile`` - for all subsequent scripts (each run in a new shell). Any - distribution-specific environment variables are also defined - here. For example, setting tags/flags to use compiling. +3. ``system_test.sh``: Build both dependencies and libpod, install them, + then execute `make localsystem` from the repository root. -4. ``build_vm_images.sh``: Examine the merged PR's description on github. - If it contains the magic string ``***CIRRUS: REBUILD IMAGES***``, then - continue. Otherwise display a message, take no further action, and - exit successfully. This prevents production of new VM images unless - they are called for, thereby saving the cost of needlessly storing them. -5. If the magic string was found, utilize [the packer tool](http://packer.io/docs/) +### ``cache_images`` Task + +Modifying the contents of cache-images is done by making changes to +one or more of the ``./contrib/cirrus/packer/*_setup.sh`` files. Testing +those changes currently requires adding a temporary commit to a PR that +updates ``.cirrus.yml``: + +* Remove all task sections except ``cache_images_task``. +* Remove the ``only_if`` condition and ``depends_on`` dependencies + +The new image names will be displayed at the end of output, assuming the build +is successful, at that point the temporary commit may be removed. Finally, +the new names may be used as ``image_name`` values in ``.cirrus.yml``. + +***N/B: Steps below are performed by automation*** + +1. When a PR is merged (``$CIRRUS_BRANCH`` == ``master``), run another + round of the ``gating`` and ``testing`` tasks (above). + +2. Assuming tests pass, if the commit message contains the magic string + ``***CIRRUS: REBUILD IMAGES***``, then this task continues. Otherwise + simply mark the master branch as 'passed'. + +3. ``setup_environment.sh``: Same as for other tasks. + +4. ``build_vm_images.sh``: Utilize [the packer tool](http://packer.io/docs/) to produce new VM images. Create a new VM from each base-image, connect - to them with ``ssh``, and perform these steps as defined by the - ``libpod_images.json`` file. + to them with ``ssh``, and perform the steps as defined by the + ``$PACKER_BASE/libpod_images.json`` file: - 1. Copy the current state of the repository into ``/tmp/libpod``. + 1. On a base-image VM, as root, copy the current state of the repository + into ``/tmp/libpod``. 2. Execute distribution-specific scripts to prepare the image for - use by the ``full_vm_testing`` task (above). + use by the ``integration_testing`` task (above). For example, + ``fedora_setup.sh``. 3. If successful, shut down each VM and create a new GCE Image - named after the base image and the commit sha of the merge. - -***Note:*** The ``.cirrus.yml`` file must be manually updated with the new -images names, then the change sent in via a secondary pull-request. This -ensures that all the ``full_vm_testing`` tasks can pass with the new images, -before subjecting all future PRs to them. A workflow to automate this -process is described in comments at the end of the ``.cirrus.yml`` file. + named with the base image, and the commit sha of the merge. + +### Base-images + +Base-images are VM disk-images specially prepared for executing as GCE VMs. +In particular, they run services on startup similar in purpose/function +as the standard 'cloud-init' services. + +* The google services are required for full support of ssh-key management + and GCE OAuth capabilities. Google provides native images in GCE + with services pre-installed, for many platforms. For example, + RHEL, CentOS, and Ubuntu. + +* Google does ***not*** provide any images for Fedora or Fedora Atomic + Host (as of 11/2018), nor do they provide a base-image prepared to + run packer for creating other images in the ``build_vm_images`` Task + (above). + +* Base images do not need to be produced often, but doing so completely + manually would be time-consuming and error-prone. Therefor a special + semi-automatic *Makefile* target is provided to assist with producing + all the base-images: ``libpod_base_images`` + +To produce new base-images, including an `image-builder-image` (used by +the ``cache_images`` Task) some input parameters are required: + +* ``GCP_PROJECT_ID``: The complete GCP project ID string e.g. foobar-12345 + identifying where the images will be stored. + +* ``GOOGLE_APPLICATION_CREDENTIALS``: A *JSON* file containing + credentials for a GCE service account. This can be [a service + account](https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually) + or [end-user + credentials](https://cloud.google.com/docs/authentication/end-user#creating_your_client_credentials) + +* ``RHEL_IMAGE_FILE`` and ``RHEL_CSUM_FILE`` complete paths + to a `rhel-server-ec2-*.raw.xz` and it's cooresponding + checksum file. These must be supplied manually because + they're not available directly via URL like other images. + +* ``RHSM_COMMAND`` contains the complete string needed to register + the VM for installing package dependencies. The VM will be de-registered + upon completion. + +* Optionally, CSV's may be specified to ``PACKER_BUILDS`` + to limit the base-images produced. For example, + ``PACKER_BUILDS=fedora,image-builder-image``. + +If there is an existing 'image-builder-image' within GCE, it may be utilized +to produce base-images (in addition to cache-images). However it must be +created with support for nested-virtualization, and with elevated cloud +privileges (to access GCE, from within the GCE VM). For example: + +``` +$ alias pgcloud='sudo podman run -it --rm -e AS_ID=$UID + -e AS_USER=$USER -v $HOME:$HOME:z quay.io/cevich/gcloud_centos:latest' + +$ URL=https://www.googleapis.com/auth +$ SCOPES=$URL/userinfo.email,$URL/compute,$URL/devstorage.full_control + +# The --min-cpu-platform is critical for nested-virt. +$ pgcloud compute instances create $USER-making-images \ + --image-family image-builder-image \ + --boot-disk-size "200GB" \ + --min-cpu-platform "Intel Haswell" \ + --machine-type n1-standard-2 \ + --scopes $SCOPES +``` + +Alternatively, if there is no image-builder-image available yet, a bare-metal +CentOS 7 machine with network access to GCE is required. Software dependencies +can be obtained from the ``packer/image-builder-image_base_setup.sh`` script. + +In both cases, the following can be used to setup and build base-images. + +``` +$ IP_ADDRESS=1.2.3.4 # EXTERNAL_IP from command output above +$ rsync -av $PWD centos@$IP_ADDRESS:. +$ scp $GOOGLE_APPLICATION_CREDENTIALS centos@$IP_ADDRESS:. +$ ssh centos@$IP_ADDRESS +... +``` + +When ready, change to the ``packer`` sub-directory, and build the images: + +``` +$ cd libpod/contrib/cirrus/packer +$ make libpod_base_images GCP_PROJECT_ID=<VALUE> \ + GOOGLE_APPLICATION_CREDENTIALS=<VALUE> \ + RHEL_IMAGE_FILE=<VALUE> \ + RHEL_CSUM_FILE=<VALUE> \ + RHSM_COMMAND=<VALUE> \ + PACKER_BUILDS=<OPTIONAL> +``` + +Assuming this is successful (hence the semi-automatic part), packer will +produce a ``packer-manifest.json`` output file. This contains the base-image +names suitable for updating in ``.cirrus.yml``, `env` keys ``*_BASE_IMAGE``. + +On failure, it should be possible to determine the problem from the packer +output. Sometimes that means setting `PACKER_LOG=1` and troubleshooting +the nested virt calls. It's also possible to observe the (nested) qemu-kvm +console output. Simply set the ``TTYDEV`` parameter, for example: + +``` +$ make libpod_base_images ... TTYDEV=$(tty) + ... +``` diff --git a/contrib/cirrus/build_vm_images.sh b/contrib/cirrus/build_vm_images.sh index ffbb2d5d5..ecdf1d877 100755 --- a/contrib/cirrus/build_vm_images.sh +++ b/contrib/cirrus/build_vm_images.sh @@ -8,12 +8,13 @@ CNI_COMMIT $CNI_COMMIT CRIO_COMMIT $CRIO_COMMIT RUNC_COMMIT $RUNC_COMMIT PACKER_BUILDS $PACKER_BUILDS +BUILT_IMAGE_SUFFIX $BUILT_IMAGE_SUFFIX CENTOS_BASE_IMAGE $CENTOS_BASE_IMAGE UBUNTU_BASE_IMAGE $UBUNTU_BASE_IMAGE FEDORA_BASE_IMAGE $FEDORA_BASE_IMAGE +FAH_BASE_IMAGE $FAH_BASE_IMAGE RHEL_BASE_IMAGE $RHEL_BASE_IMAGE RHSM_COMMAND $RHSM_COMMAND -BUILT_IMAGE_SUFFIX $BUILT_IMAGE_SUFFIX SERVICE_ACCOUNT $SERVICE_ACCOUNT GCE_SSH_USERNAME $GCE_SSH_USERNAME GCP_PROJECT_ID $GCP_PROJECT_ID @@ -22,36 +23,30 @@ SCRIPT_BASE $SCRIPT_BASE PACKER_BASE $PACKER_BASE " -require_regex '\*\*\*\s*CIRRUS:\s*REBUILD\s*IMAGES\s*\*\*\*' 'Not re-building VM images' - show_env_vars # Everything here is running on the 'image-builder-image' GCE image # Assume basic dependencies are all met, but there could be a newer version # of the packer binary PACKER_FILENAME="packer_${PACKER_VER}_linux_amd64.zip" -mkdir -p "$HOME/packer" -cd "$HOME/packer" -# image_builder_image has packer pre-installed, check if same version requested -if ! [[ -r "$PACKER_FILENAME" ]] +if [[ -d "$HOME/packer" ]] then - curl -L -O https://releases.hashicorp.com/packer/$PACKER_VER/$PACKER_FILENAME - curl -L https://releases.hashicorp.com/packer/${PACKER_VER}/packer_${PACKER_VER}_SHA256SUMS | \ - grep 'linux_amd64' > ./sha256sums - sha256sum --check ./sha256sums - unzip -o $PACKER_FILENAME - ./packer --help &> /dev/null # verify exit(0) + cd "$HOME/packer" + # image_builder_image has packer pre-installed, check if same version requested + if [[ -r "$PACKER_FILENAME" ]] + then + cp $PACKER_FILENAME "$GOSRC/$PACKER_BASE/" + cp packer "$GOSRC/$PACKER_BASE/" + fi fi set -x -cd "$GOSRC" -# N/B: /usr/sbin/packer is a DIFFERENT tool, and will exit 0 given the args below :( -TEMPLATE="./$PACKER_BASE/libpod_images.json" - -$HOME/packer/packer inspect "$TEMPLATE" - -#$HOME/packer/packer build -machine-readable "-only=$PACKER_BUILDS" "$TEMPLATE" | tee /tmp/packer_log.csv -$HOME/packer/packer build "-only=$PACKER_BUILDS" "$TEMPLATE" - -# TODO: Report back to PR names of built images +cd "$GOSRC/$PACKER_BASE" +make libpod_images \ + PACKER_BUILDS=$PACKER_BUILDS \ + PACKER_VER=$PACKER_VER \ + GOSRC=$GOSRC \ + SCRIPT_BASE=$SCRIPT_BASE \ + PACKER_BASE=$PACKER_BASE \ + BUILT_IMAGE_SUFFIX=$BUILT_IMAGE_SUFFIX diff --git a/contrib/cirrus/integration_test.sh b/contrib/cirrus/integration_test.sh index 226053724..a50bd448f 100755 --- a/contrib/cirrus/integration_test.sh +++ b/contrib/cirrus/integration_test.sh @@ -9,7 +9,7 @@ OS_RELEASE_ID $OS_RELEASE_ID OS_RELEASE_VER $OS_RELEASE_VER " -show_env_vars +clean_env set -x cd "$GOSRC" @@ -19,10 +19,13 @@ case "${OS_RELEASE_ID}-${OS_RELEASE_VER}" in make test-binaries "BUILDTAGS=$BUILDTAGS" SKIP_USERNS=1 make localintegration "BUILDTAGS=$BUILDTAGS" ;; - fedora-28) ;& # Continue to the next item + fedora-29) ;& # Continue to the next item + fedora-28) ;& centos-7) ;& rhel-7) - stub 'integration testing not working on $OS_RELEASE_ID' + make install PREFIX=/usr ETCDIR=/etc + make test-binaries + make localintegration ;; *) bad_os_id_ver ;; esac diff --git a/contrib/cirrus/lib.sh b/contrib/cirrus/lib.sh index 2fa91258b..3b567b7a7 100644 --- a/contrib/cirrus/lib.sh +++ b/contrib/cirrus/lib.sh @@ -4,8 +4,21 @@ # to be sourced by other scripts, not called directly. # Under some contexts these values are not set, make sure they are. -USER="$(whoami)" -HOME="$(getent passwd $USER | cut -d : -f 6)" +export USER="$(whoami)" +export HOME="$(getent passwd $USER | cut -d : -f 6)" + +# These are normally set by cirrus, if not use some reasonable defaults +ENVLIB=${ENVLIB:-.bash_profile} +CIRRUS_WORKING_DIR=${CIRRUS_WORKING_DIR:-/var/tmp/go/src/github.com/containers/libpod} +SCRIPT_BASE=${SCRIPT_BASE:-./contrib/cirrus} +PACKER_BASE=${PACKER_BASE:-./contrib/cirrus/packer} +CIRRUS_BUILD_ID=${CIRRUS_BUILD_ID:-DEADBEEF} # a human +cd "$CIRRUS_WORKING_DIR" +CIRRUS_BASE_SHA=${CIRRUS_BASE_SHA:-$(git rev-parse upstream/master || git rev-parse origin/master)} +CIRRUS_CHANGE_IN_REPO=${CIRRUS_CHANGE_IN_REPO:-$(git rev-parse HEAD)} +CIRRUS_REPO_NAME=${CIRRUS_REPO_NAME:-libpod} +cd - + if ! [[ "$PATH" =~ "/usr/local/bin" ]] then export PATH="$PATH:/usr/local/bin" @@ -73,6 +86,18 @@ PACKER_BUILDS $PACKER_BUILDS do [[ -z "$NAME" ]] || echo "export $NAME=\"$VALUE\"" done + echo "" + echo "##### $(go version) #####" + echo "" +} + +# Unset environment variables not needed for testing purposes +clean_env() { + req_env_var " + UNSET_ENV_VARS $UNSET_ENV_VARS + " + echo "Unsetting $(echo $UNSET_ENV_VARS | wc -w) environment variables" + unset -v UNSET_ENV_VARS $UNSET_ENV_VARS || true # don't fail on read-only } # Return a GCE image-name compatible string representation of distribution name @@ -98,15 +123,17 @@ stub() { ircmsg() { req_env_var " - SCRIPT_BASE $SCRIPT_BASE - GOSRC $GOSRC CIRRUS_TASK_ID $CIRRUS_TASK_ID 1 $1 " - SCRIPT="$GOSRC/$SCRIPT_BASE/podbot.py" + # Sometimes setup_environment.sh didn't run + SCRIPT="$(dirname $0)/podbot.py" NICK="podbot_$CIRRUS_TASK_ID" NICK="${NICK:0:15}" # Any longer will break things + set +e $SCRIPT $NICK $1 + echo "Ignoring exit($?)" + set -e } # Run sudo in directory with GOPATH set @@ -117,23 +144,6 @@ cdsudo() { sudo --preserve-env=GOPATH --non-interactive bash -c "$CMD" } -# Skip a build if $1 does not match in the PR Title/Description with message $2 -require_regex() { - req_env_var " - CIRRUS_CHANGE_MESSAGE $CIRRUS_CHANGE_MESSAGE - 1 $1 - 2 $2 - " - regex="$1" - msg="$2" - if ! echo "$CIRRUS_CHANGE_MESSAGE" | egrep -q "$regex" - then - echo "***** The PR Title/Description did not match the regular expression: $MAGIC_RE" - echo "***** $msg" - exit 0 - fi -} - # Helper/wrapper script to only show stderr/stdout on non-zero exit install_ooe() { req_env_var "SCRIPT_BASE $SCRIPT_BASE" @@ -171,6 +181,19 @@ install_cni_plugins() { sudo cp bin/* /usr/libexec/cni } +install_runc_from_git(){ + wd=$(pwd) + DEST="$GOPATH/src/github.com/opencontainers/runc" + rm -rf "$DEST" + ooe.sh git clone https://github.com/opencontainers/runc.git "$DEST" + cd "$DEST" + ooe.sh git fetch origin --tags + ooe.sh git checkout -q "$RUNC_COMMIT" + ooe.sh make static BUILDTAGS="seccomp selinux" + sudo install -m 755 runc /usr/bin/runc + cd $wd +} + install_runc(){ OS_RELEASE_ID=$(os_release_id) echo "Installing RunC from commit $RUNC_COMMIT" @@ -193,14 +216,7 @@ install_runc(){ cd "$GOPATH/src/github.com/containers/libpod" ooe.sh sudo make install.libseccomp.sudo fi - DEST="$GOPATH/src/github.com/opencontainers/runc" - rm -rf "$DEST" - ooe.sh git clone https://github.com/opencontainers/runc.git "$DEST" - cd "$DEST" - ooe.sh git fetch origin --tags - ooe.sh git checkout -q "$RUNC_COMMIT" - ooe.sh make static BUILDTAGS="seccomp selinux" - sudo install -m 755 runc /usr/bin/runc + install_runc_from_git } install_buildah() { @@ -277,21 +293,29 @@ install_varlink(){ } _finalize(){ + set +e # Don't fail at the very end + set +e # make errors non-fatal echo "Removing leftover giblets from cloud-init" cd / sudo rm -rf /var/lib/cloud/instance? sudo rm -rf /root/.ssh/* sudo rm -rf /home/* + sudo rm -rf /tmp/* + sudo rm -rf /tmp/.??* + sync + sudo fstrim -av } rh_finalize(){ + set +e # Don't fail at the very end # Allow root ssh-logins if [[ -r /etc/cloud/cloud.cfg ]] then sudo sed -re 's/^disable_root:.*/disable_root: 0/g' -i /etc/cloud/cloud.cfg fi echo "Resetting to fresh-state for usage as cloud-image." - sudo $(type -P dnf || type -P yum) clean all + PKG=$(type -P dnf || type -P yum || echo "") + [[ -z "$PKG" ]] || sudo $PKG clean all # not on atomic sudo rm -rf /var/cache/{yum,dnf} sudo rm -f /etc/udev/rules.d/*-persistent-*.rules sudo touch /.unconfigured # force firstboot to run @@ -299,7 +323,35 @@ rh_finalize(){ } ubuntu_finalize(){ + set +e # Don't fail at the very end echo "Resetting to fresh-state for usage as cloud-image." sudo rm -rf /var/cache/apt _finalize } + +rhel_exit_handler() { + set +ex + req_env_var " + GOPATH $GOPATH + RHSMCMD $RHSMCMD + " + cd / + sudo rm -rf "$RHSMCMD" + sudo rm -rf "$GOPATH" + sudo subscription-manager remove --all + sudo subscription-manager unregister + sudo subscription-manager clean +} + +rhsm_enable() { + req_env_var " + RHSM_COMMAND $RHSM_COMMAND + " + export GOPATH="$(mktemp -d)" + export RHSMCMD="$(mktemp)" + trap "rhel_exit_handler" EXIT + # Avoid logging sensitive details + echo "$RHSM_COMMAND" > "$RHSMCMD" + ooe.sh sudo bash "$RHSMCMD" + sudo rm -rf "$RHSMCMD" +} diff --git a/contrib/cirrus/optional_system_test.sh b/contrib/cirrus/optional_system_test.sh deleted file mode 100755 index 705dda5ad..000000000 --- a/contrib/cirrus/optional_system_test.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/bash - -set -e -source $(dirname $0)/lib.sh - -MAGIC_RE='\*\*\*\s*CIRRUS:\s*SYSTEM\s*TEST\s*\*\*\*' -if ! echo "$CIRRUS_CHANGE_MESSAGE" | egrep -q "$MAGIC_RE" -then - echo "Skipping system-testing because PR title or description" - echo "does not match regular expression: $MAGIC_RE" - exit 0 -fi - -req_env_var " -GOSRC $GOSRC -OS_RELEASE_ID $OS_RELEASE_ID -OS_RELEASE_VER $OS_RELEASE_VER -" - -show_env_vars - -set -x -cd "$GOSRC" -make localsystem diff --git a/contrib/cirrus/packer/.gitignore b/contrib/cirrus/packer/.gitignore new file mode 100644 index 000000000..8f7bdeaf7 --- /dev/null +++ b/contrib/cirrus/packer/.gitignore @@ -0,0 +1,7 @@ +*json +packer +packer*zip +packer_cache +cidata* +meta-data +user-data diff --git a/contrib/cirrus/packer/Makefile b/contrib/cirrus/packer/Makefile new file mode 100644 index 000000000..9bf27373e --- /dev/null +++ b/contrib/cirrus/packer/Makefile @@ -0,0 +1,108 @@ + +# N/B: PACKER_BUILDS variable is required. Should contain CSV of +# builder name(s) from applicable YAML file, +# e.g for names see libpod_images.yml + +PACKER_VER ?= 1.3.1 +PACKER_DIST_FILENAME := packer_${PACKER_VER}_linux_amd64.zip + +# Only needed for libpod_base_images target +TIMESTAMP := $(shell date +%s) +GOSRC ?= $(shell realpath "./../../../") +PACKER_BASE ?= contrib/cirrus/packer +SCRIPT_BASE ?= contrib/cirrus + +# For debugging nested-virt, use +#TTYDEV := $(shell tty) +TTYDEV := /dev/null + +.PHONY: all +all: libpod_images + +%.json: %.yml + @python3 -c 'import json,yaml; json.dump( yaml.load(open("$<").read()), open("$@","w"), indent=2);' + +${PACKER_DIST_FILENAME}: + @curl -L --silent --show-error \ + -O https://releases.hashicorp.com/packer/${PACKER_VER}/${PACKER_DIST_FILENAME} + +packer: ${PACKER_DIST_FILENAME} + @curl -L --silent --show-error \ + https://releases.hashicorp.com/packer/${PACKER_VER}/packer_${PACKER_VER}_SHA256SUMS \ + | grep 'linux_amd64' > /tmp/packer_sha256sums + @sha256sum --check /tmp/packer_sha256sums + @unzip -o ${PACKER_DIST_FILENAME} + @touch --reference=Makefile ${PACKER_DIST_FILENAME} + +.PHONY: test +test: libpod_base_images.json libpod_images.json packer + ./packer inspect libpod_base_images.json > /dev/null + ./packer inspect libpod_images.json > /dev/null + @echo "All good" + +.PHONY: libpod_images +libpod_images: libpod_images.json packer +ifndef PACKER_BUILDS + $(error PACKER_BUILDS is undefined, expected builder-names CSV) +endif + ./packer build -only=${PACKER_BUILDS} \ + -var GOSRC=$(GOSRC) \ + -var PACKER_BASE=$(PACKER_BASE) \ + -var SCRIPT_BASE=$(SCRIPT_BASE) \ + libpod_images.json + @echo "" + @echo "Finished. The images mentioned above, and in packer-manifest.json" + @echo "can be used in .cirrus.yml as values for the 'image_name' keys" + @echo "" + +cidata.ssh: + ssh-keygen -f $@ -P "" -q + +cidata.ssh.pub: cidata.ssh + touch $@ + +meta-data: + echo "local-hostname: localhost.localdomain" > $@ + +user-data: cidata.ssh.pub + bash make-user-data.sh + +cidata.iso: user-data meta-data + genisoimage -output cidata.iso -volid cidata -input-charset utf-8 -joliet -rock user-data meta-data + +# This is intended to be run by a human, with admin access to the libpod GCE project. +.PHONY: libpod_base_images +libpod_base_images: libpod_base_images.json cidata.iso cidata.ssh packer +ifndef GCP_PROJECT_ID + $(error GCP_PROJECT_ID is undefined, expected complete GCP project ID string e.g. foobar-12345) +endif +ifndef GOOGLE_APPLICATION_CREDENTIALS + $(error GOOGLE_APPLICATION_CREDENTIALS is undefined, expected absolute path to JSON file, like $HOME/.config/gcloud/legacy_credentials/*/adc.json) +endif +ifndef RHEL_IMAGE_FILE + $(error RHEL_IMAGE_FILE is undefined, expected full path to a rhel-server-ec2-*.raw.xz file) +endif +ifndef RHEL_CSUM_FILE + $(error RHEL_CSUM_FILE is undefined, expected full path to a rhel-server-ec2-*.raw.xz.SHA256SUM file) +endif +ifndef RHSM_COMMAND + $(error RHSM_COMMAND is undefined, expected string required for temporarily registering VM) +endif + PACKER_CACHE_DIR=/tmp ./packer build \ + -var TIMESTAMP=$(TIMESTAMP) \ + -var TTYDEV=$(TTYDEV) \ + -var GCP_PROJECT_ID=$(GCP_PROJECT_ID) \ + -var GOOGLE_APPLICATION_CREDENTIALS=$(GOOGLE_APPLICATION_CREDENTIALS) \ + -var GOSRC=$(GOSRC) \ + -var PACKER_BASE=$(PACKER_BASE) \ + -var SCRIPT_BASE=$(SCRIPT_BASE) \ + -var RHEL_BASE_IMAGE_NAME=$(shell basename $(RHEL_IMAGE_FILE) | tr -d '[[:space:]]' | sed -r -e 's/\.x86_64\.raw\.xz//' | tr '[[:upper:]]' '[[:lower:]]' | tr '[[:punct:]]' '-') \ + -var RHEL_IMAGE_FILE=$(RHEL_IMAGE_FILE) \ + -var RHEL_CSUM_FILE=$(RHEL_CSUM_FILE) \ + -var 'RHSM_COMMAND=$(RHSM_COMMAND)' \ + -only $(PACKER_BUILDS) \ + libpod_base_images.json + @echo "" + @echo "Finished. The images mentioned above, and in packer-manifest.json" + @echo "can be used in .cirrus.yml as values for the *_BASE_IMAGE keys." + @echo "" diff --git a/contrib/cirrus/packer/README.md b/contrib/cirrus/packer/README.md index 8ff6947e9..9a07ed960 100644 --- a/contrib/cirrus/packer/README.md +++ b/contrib/cirrus/packer/README.md @@ -1,2 +1,3 @@ These are definitions and scripts consumed by packer to produce the -various distribution images used for CI testing. +various distribution images used for CI testing. For more details +see the [Cirrus CI documentation](../README.md) diff --git a/contrib/cirrus/packer/centos_setup.sh b/contrib/cirrus/packer/centos_setup.sh index 7b2308739..a13050569 100644 --- a/contrib/cirrus/packer/centos_setup.sh +++ b/contrib/cirrus/packer/centos_setup.sh @@ -29,6 +29,7 @@ ooe.sh sudo yum -y install \ btrfs-progs-devel \ bzip2 \ device-mapper-devel \ + emacs-nox \ findutils \ glib2-devel \ glibc-static \ @@ -63,6 +64,7 @@ ooe.sh sudo yum -y install \ runc \ skopeo-containers \ unzip \ + vim \ which \ xz diff --git a/contrib/cirrus/packer/fah_base-setup.sh b/contrib/cirrus/packer/fah_base-setup.sh new file mode 100644 index 000000000..606c4f336 --- /dev/null +++ b/contrib/cirrus/packer/fah_base-setup.sh @@ -0,0 +1,45 @@ + +# N/B: This script is not intended to be run by humans. It is used to configure the +# FAH base image for importing, so that it will boot in GCE. + +set -e + +# Load in library (copied by packer, before this script was run) +source $GOSRC/$SCRIPT_BASE/lib.sh + +install_ooe + +if [[ "$1" == "pre" ]] +then + echo "Upgrading Atomic Host" + setenforce 0 + ooe.sh atomic host upgrade + + echo "Configuring Repositories" + ooe.sh sudo tee /etc/yum.repos.d/ngompa-gce-oslogin.repo <<EOF +[ngompa-gce-oslogin] +name=Copr repo for gce-oslogin owned by ngompa +baseurl=https://copr-be.cloud.fedoraproject.org/results/ngompa/gce-oslogin/fedora-\$releasever-\$basearch/ +type=rpm-md +skip_if_unavailable=True +gpgcheck=1 +gpgkey=https://copr-be.cloud.fedoraproject.org/results/ngompa/gce-oslogin/pubkey.gpg +repo_gpgcheck=0 +enabled=1 +enabled_metadata=1 +EOF + echo "Installing necessary packages and google services" + # Google services are enabled by default, upon install. + ooe.sh rpm-ostree install rng-tools google-compute-engine google-compute-engine-oslogin + echo "Rebooting..." + systemctl reboot # Required for upgrade + package installs to be active +elif [[ "$1" == "post" ]] +then + echo "Enabling necessary services" + systemctl enable rngd # Must reboot before enabling + rh_finalize + echo "SUCCESS!" +else + echo "Expected to be called with 'pre' or 'post'" + exit 6 +fi diff --git a/contrib/cirrus/packer/fah_setup.sh b/contrib/cirrus/packer/fah_setup.sh new file mode 100644 index 000000000..2e053b396 --- /dev/null +++ b/contrib/cirrus/packer/fah_setup.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# This script is called by packer on the subject fah VM, to setup the podman +# build/test environment. It's not intended to be used outside of this context. + +set -e + +# Load in library (copied by packer, before this script was run) +source /tmp/libpod/$SCRIPT_BASE/lib.sh + +req_env_var " +SCRIPT_BASE $SCRIPT_BASE +" + +install_ooe + +ooe.sh sudo atomic host upgrade + +ooe.sh sudo rpm-ostree uninstall cloud-init + +rh_finalize + +echo "SUCCESS!" diff --git a/contrib/cirrus/packer/fedora_base-setup.sh b/contrib/cirrus/packer/fedora_base-setup.sh new file mode 100644 index 000000000..c0a1e422c --- /dev/null +++ b/contrib/cirrus/packer/fedora_base-setup.sh @@ -0,0 +1,27 @@ +#!/bin/bash + +# N/B: This script is not intended to be run by humans. It is used to configure the +# fedora base image for importing, so that it will boot in GCE + +set -e + +# Load in library (copied by packer, before this script was run) +source $GOSRC/$SCRIPT_BASE/lib.sh + +[[ "$1" == "post" ]] || exit 0 # nothing to do + +install_ooe + +echo "Updating packages" +ooe.sh dnf -y update + +echo "Installing necessary packages and google services" +ooe.sh dnf -y copr enable ngompa/gce-oslogin +ooe.sh dnf -y install rng-tools google-compute-engine google-compute-engine-oslogin + +echo "Enabling services" +ooe.sh systemctl enable rngd + +rh_finalize + +echo "SUCCESS!" diff --git a/contrib/cirrus/packer/fedora_setup.sh b/contrib/cirrus/packer/fedora_setup.sh index f9fea04a7..01c468901 100644 --- a/contrib/cirrus/packer/fedora_setup.sh +++ b/contrib/cirrus/packer/fedora_setup.sh @@ -10,6 +10,7 @@ source /tmp/libpod/$SCRIPT_BASE/lib.sh req_env_var " SCRIPT_BASE $SCRIPT_BASE +FEDORA_CNI_COMMIT $FEDORA_CNI_COMMIT CNI_COMMIT $CNI_COMMIT CRIO_COMMIT $CRIO_COMMIT CRIU_COMMIT $CRIU_COMMIT @@ -27,8 +28,8 @@ ooe.sh sudo dnf install -y \ atomic-registries \ btrfs-progs-devel \ bzip2 \ - conmon \ device-mapper-devel \ + emacs-nox \ findutils \ git \ glib2-devel \ @@ -65,11 +66,14 @@ ooe.sh sudo dnf install -y \ runc \ skopeo-containers \ slirp4netns \ + unzip \ + vim \ which \ xz install_varlink +CNI_COMMIT=$FEDORA_CNI_COMMIT install_cni_plugins install_buildah diff --git a/contrib/cirrus/packer/image-builder-image_base-setup.sh b/contrib/cirrus/packer/image-builder-image_base-setup.sh new file mode 100644 index 000000000..b8e2824a7 --- /dev/null +++ b/contrib/cirrus/packer/image-builder-image_base-setup.sh @@ -0,0 +1,75 @@ +#!/bin/bash + +# This script is called by packer on a vanilla CentOS VM, to setup the image +# used for building images FROM base images. It's not intended to be used +# outside of this context. + +set -e + +[[ "$1" == "post" ]] || exit 0 # pre stage not needed + +# Load in library (copied by packer, before this script was run) +source $GOSRC/$SCRIPT_BASE/lib.sh + +req_env_var " + TIMESTAMP $TIMESTAMP + GOSRC $GOSRC + SCRIPT_BASE $SCRIPT_BASE + PACKER_BASE $PACKER_BASE +" + +install_ooe + +echo "Updating packages" +ooe.sh sudo yum -y update + +echo "Configuring repositories" +ooe.sh sudo yum -y install centos-release-scl epel-release + +echo "Installing packages" +ooe.sh sudo yum -y install \ + genisoimage \ + golang \ + google-cloud-sdk \ + libvirt \ + libvirt-admin \ + libvirt-client \ + libvirt-daemon \ + make \ + python34 \ + python34 \ + python34-PyYAML \ + python34-PyYAML \ + qemu-img \ + qemu-kvm \ + qemu-kvm-tools \ + qemu-user \ + rsync \ + unzip \ + util-linux \ + vim + +sudo ln -s /usr/libexec/qemu-kvm /usr/bin/ + +sudo tee /etc/modprobe.d/kvm-nested.conf <<EOF +options kvm-intel nested=1 +options kvm-intel enable_shadow_vmcs=1 +options kvm-intel enable_apicv=1 +options kvm-intel ept=1 +EOF + +echo "Installing packer" +sudo mkdir -p /root/$(basename $PACKER_BASE) +sudo cp $GOSRC/$PACKER_BASE/*packer* /root/$(basename $PACKER_BASE) +sudo mkdir -p /root/$(basename $SCRIPT_BASE) +sudo cp $GOSRC/$SCRIPT_BASE/*.sh /root/$(basename $SCRIPT_BASE) + +install_scl_git + +echo "Cleaning up" +cd / +rm -rf $GOSRC + +rh_finalize + +echo "SUCCESS!" diff --git a/contrib/cirrus/packer/libpod_base_images.yml b/contrib/cirrus/packer/libpod_base_images.yml new file mode 100644 index 000000000..109b9b8d5 --- /dev/null +++ b/contrib/cirrus/packer/libpod_base_images.yml @@ -0,0 +1,193 @@ +--- + +variables: + # Complete local path to this repository (Required) + GOSRC: + # Relative path to this (packer) subdirectory (Required) + PACKER_BASE: + # Relative path to cirrus scripts subdirectory (Required) + SCRIPT_BASE: + # Unique ID for naming new base-images (required) + TIMESTAMP: + # Required for output from qemu builders + TTYDEV: + # RHEL images require click-through agreements to obtain (required) + RHEL_BASE_IMAGE_NAME: + RHEL_IMAGE_FILE: + RHEL_CSUM_FILE: + # RHEL requires a subscription to install/update packages + RHSM_COMMAND: + + # Latest Fedora release + FEDORA_IMAGE_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/29/Cloud/x86_64/images/Fedora-Cloud-Base-29-1.2.x86_64.qcow2" + FEDORA_CSUM_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/29/Cloud/x86_64/images/Fedora-Cloud-29-1.2-x86_64-CHECKSUM" + FEDORA_BASE_IMAGE_NAME: 'fedora-cloud-base-29-1-2' # Name to use in GCE + # Prior Fedora release + PRIOR_FEDORA_IMAGE_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2" + PRIOR_FEDORA_CSUM_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-28-1.1-x86_64-CHECKSUM" + PRIOR_FEDORA_BASE_IMAGE_NAME: 'fedora-cloud-base-28-1-1' # Name to use in GCE + FAH_IMAGE_URL: "https://dl.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-29-20181025.1/AtomicHost/x86_64/images/Fedora-AtomicHost-29-20181025.1.x86_64.qcow2" + FAH_CSUM_URL: "https://dl.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-29-20181025.1/AtomicHost/x86_64/images/Fedora-AtomicHost-29-20181025.1-x86_64-CHECKSUM" + FAH_BASE_IMAGE_NAME: 'fedora-atomichost-29-20181025-1' # Name to use in GCE + + # The name of the image in GCE used for packer build libpod_images.yml + IBI_BASE_NAME: 'image-builder-image' + CIDATA_ISO: 'cidata.iso' # produced by Makefile + + # Path to json file (required, likely ~/.config/gcloud/legacy_credentials/*/adc.json) + GOOGLE_APPLICATION_CREDENTIALS: + # The complete project ID (required, not the short name) + GCP_PROJECT_ID: + # Pre-existing storage bucket w/ lifecycle-enabled + XFERBUCKET: "packer-import" # pre-created, globally unique, lifecycle-enabled + +# Don't leak sensitive values in error messages / output +sensitive-variables: + - 'GOOGLE_APPLICATION_CREDENTIALS' + - 'GCP_PROJECT_ID' + - 'RHSM_COMMAND' + +# What images to produce in which cloud +builders: + - name: '{{user `IBI_BASE_NAME`}}' + type: 'googlecompute' + image_name: '{{user `IBI_BASE_NAME`}}-{{user `TIMESTAMP`}}' + image_family: '{{user `IBI_BASE_NAME`}}' + source_image_project_id: 'centos-cloud' + source_image_family: 'centos-7' + project_id: '{{user `GCP_PROJECT_ID`}}' + account_file: '{{user `GOOGLE_APPLICATION_CREDENTIALS`}}' + communicator: 'ssh' + ssh_username: 'centos' + ssh_pty: 'true' + # The only supported zone in Cirrus-CI, as of addition of this comment + zone: 'us-central1-a' + # Enable nested virtualization in case it's ever needed + image_licenses: + - 'https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx' + min_cpu_platform: "Intel Broadwell" # nested-virt requirement + + - &nested_virt + name: 'fedora' + type: 'qemu' + accelerator: "kvm" + iso_url: '{{user `FEDORA_IMAGE_URL`}}' + disk_image: true + format: "raw" + disk_size: 5120 + iso_checksum_url: '{{user `FEDORA_CSUM_URL`}}' + iso_checksum_type: "sha256" + output_directory: '/tmp/{{build_name}}' + vm_name: "disk.raw" # actually qcow2, name required for post-processing + boot_wait: '5s' + shutdown_command: 'shutdown -h now' + headless: true + qemu_binary: "/usr/libexec/qemu-kvm" + qemuargs: # List-of-list format required to override packer-generated args + - - "-m" + - "1024" + - - "-cpu" + - "host" + - - "-device" + - "virtio-rng-pci" + - - "-chardev" + - "tty,id=pts,path={{user `TTYDEV`}}" + - - "-device" + - "isa-serial,chardev=pts" + - - "-cdrom" + - "{{user `CIDATA_ISO`}}" + - - "-netdev" + - "user,id=net0,hostfwd=tcp::{{ .SSHHostPort }}-:22" + - - "-device" + - "virtio-net,netdev=net0" + communicator: 'ssh' + ssh_private_key_file: 'cidata.ssh' + ssh_username: 'root' + + - <<: *nested_virt + name: 'prior_fedora' + iso_url: '{{user `PRIOR_FEDORA_IMAGE_URL`}}' + iso_checksum_url: '{{user `PRIOR_FEDORA_CSUM_URL`}}' + + - <<: *nested_virt + name: 'fah' + iso_url: '{{user `FAH_IMAGE_URL`}}' + iso_checksum_url: '{{user `FAH_CSUM_URL`}}' + disk_size: 10240 + + - <<: *nested_virt + name: 'rhel' + iso_url: 'file://{{user `RHEL_IMAGE_FILE`}}' + iso_checksum_url: 'file://{{user `RHEL_CSUM_FILE`}}' + disk_size: 10240 + +provisioners: + - type: 'shell' + inline: + - 'mkdir -p /tmp/libpod/{{user `SCRIPT_BASE`}}' + - 'mkdir -p /tmp/libpod/{{user `PACKER_BASE`}}' + + - type: 'file' + source: '{{user `GOSRC`}}/.cirrus.yml' + destination: '/tmp/libpod/.cirrus.yml' + + - type: 'file' + source: '{{user `GOSRC`}}/{{user `SCRIPT_BASE`}}/' + destination: '/tmp/libpod/{{user `SCRIPT_BASE`}}/' + + - type: 'file' + source: '{{user `GOSRC`}}/{{user `PACKER_BASE`}}/' + destination: '/tmp/libpod/{{user `PACKER_BASE`}}/' + + - &shell_script + type: 'shell' + inline: + - 'chmod +x /tmp/libpod/{{user `PACKER_BASE`}}/{{build_name}}_base-setup.sh' + - '/tmp/libpod/{{user `PACKER_BASE`}}/{{build_name}}_base-setup.sh pre' + expect_disconnect: true # Allow this to reboot the VM + environment_vars: + - 'TIMESTAMP={{user `TIMESTAMP`}}' + - 'GOSRC=/tmp/libpod' + - 'SCRIPT_BASE={{user `SCRIPT_BASE`}}' + - 'PACKER_BASE={{user `PACKER_BASE`}}' + - 'RHSM_COMMAND={{user `RHSM_COMMAND`}}' + + - <<: *shell_script + inline: ['{{user `GOSRC`}}/{{user `PACKER_BASE`}}/{{build_name}}_base-setup.sh'] + expect_disconnect: false + pause_before: '10s' + inline: + - '/tmp/libpod/{{user `PACKER_BASE`}}/{{build_name}}_base-setup.sh post' + +post-processors: + - - type: "compress" + only: ['fedora', 'prior_fedora', 'fah', 'rhel'] + output: '/tmp/{{build_name}}/disk.raw.tar.gz' + format: '.tar.gz' + compression_level: 9 + - &gcp_import + only: ['fedora'] + type: "googlecompute-import" + project_id: '{{user `GCP_PROJECT_ID`}}' + account_file: '{{user `GOOGLE_APPLICATION_CREDENTIALS`}}' + bucket: '{{user `XFERBUCKET`}}' + gcs_object_name: '{{build_name}}-{{user `TIMESTAMP`}}-{{uuid}}.tar.gz' + image_name: "{{user `FEDORA_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}" + image_description: 'Based on {{user `FEDORA_IMAGE_URL`}}' + image_family: '{{user `FEDORA_BASE_IMAGE_NAME`}}' + - <<: *gcp_import + only: ['prior_fedora'] + image_name: "{{user `PRIOR_FEDORA_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}" + image_description: 'Based on {{user `PRIOR_FEDORA_IMAGE_URL`}}' + image_family: '{{user `PRIOR_FEDORA_BASE_IMAGE_NAME`}}' + - <<: *gcp_import + only: ['fah'] + image_name: "{{user `FAH_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}" + image_description: 'Based on {{user `FAH_IMAGE_URL`}}' + image_family: '{{user `FAH_BASE_IMAGE_NAME`}}' + - <<: *gcp_import + only: ['rhel'] + image_name: "{{user `RHEL_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}" + image_description: 'Based on {{user `RHEL_IMAGE_FILE`}}' + image_family: '{{user `RHEL_BASE_IMAGE_NAME`}}' + - type: 'manifest' diff --git a/contrib/cirrus/packer/libpod_images.json b/contrib/cirrus/packer/libpod_images.json deleted file mode 100644 index 9dac3e8ea..000000000 --- a/contrib/cirrus/packer/libpod_images.json +++ /dev/null @@ -1,130 +0,0 @@ -{ - "variables": { - "FEDORA_CNI_COMMIT": "{{env `FEDORA_CNI_COMMIT`}}", - "CNI_COMMIT": "{{env `CNI_COMMIT`}}", - "CRIO_COMMIT": "{{env `CRIO_COMMIT`}}", - "CRIU_COMMIT": "{{env `CRIU_COMMIT`}}", - "RUNC_COMMIT": "{{env `RUNC_COMMIT`}}", - - "CENTOS_BASE_IMAGE": "{{env `CENTOS_BASE_IMAGE`}}" , - "UBUNTU_BASE_IMAGE": "{{env `UBUNTU_BASE_IMAGE`}}", - "FEDORA_BASE_IMAGE": "{{env `FEDORA_BASE_IMAGE`}}", - "RHEL_BASE_IMAGE": "{{env `RHEL_BASE_IMAGE`}}", - - "GOSRC": "{{env `GOSRC`}}", - "PACKER_BASE": "{{env `PACKER_BASE`}}", - "SCRIPT_BASE": "{{env `SCRIPT_BASE`}}", - - "SERVICE_ACCOUNT": "{{env `SERVICE_ACCOUNT`}}", - "GCP_PROJECT_ID": "{{env `GCP_PROJECT_ID`}}", - "BUILT_IMAGE_SUFFIX": "{{env `BUILT_IMAGE_SUFFIX`}}", - "GCE_SSH_USERNAME": "{{env `GCE_SSH_USERNAME`}}", - "RHSM_COMMAND": "{{env `RHSM_COMMAND`}}" - }, - "sensitive-variables": [ - "GCP_PROJECT_ID", "SERVICE_ACCOUNT", "GCE_SSH_USERNAME", "RHSM_COMMAND" - ], - "builders": [ - { - "name": "rhel-7", - "type": "googlecompute", - "project_id": "{{user `GCP_PROJECT_ID`}}", - "zone": "us-central1-a", - "source_image": "{{user `RHEL_BASE_IMAGE`}}", - "image_name": "{{user `RHEL_BASE_IMAGE`}}{{user `BUILT_IMAGE_SUFFIX`}}", - "image_family": "{{user `RHEL_BASE_IMAGE`}}-libpod", - "service_account_email": "{{user `SERVICE_ACCOUNT`}}", - "communicator": "ssh", - "ssh_username": "ec2-user", - "ssh_pty": "true" - },{ - "name": "centos-7", - "type": "googlecompute", - "project_id": "{{user `GCP_PROJECT_ID`}}", - "zone": "us-central1-a", - "source_image": "{{user `CENTOS_BASE_IMAGE`}}", - "image_name": "{{user `CENTOS_BASE_IMAGE`}}{{user `BUILT_IMAGE_SUFFIX`}}", - "image_family": "{{user `CENTOS_BASE_IMAGE`}}-libpod", - "service_account_email": "{{user `SERVICE_ACCOUNT`}}", - "communicator": "ssh", - "ssh_username": "{{user `GCE_SSH_USERNAME`}}", - "ssh_pty": "true" - },{ - "name": "fedora-28", - "type": "googlecompute", - "project_id": "{{user `GCP_PROJECT_ID`}}", - "zone": "us-central1-a", - "source_image": "{{user `FEDORA_BASE_IMAGE`}}", - "image_name": "{{user `FEDORA_BASE_IMAGE`}}{{user `BUILT_IMAGE_SUFFIX`}}", - "image_family": "{{user `FEDORA_BASE_IMAGE`}}-libpod", - "service_account_email": "{{user `SERVICE_ACCOUNT`}}", - "communicator": "ssh", - "ssh_username": "fedora", - "ssh_pty": "true" - },{ - "name": "ubuntu-18", - "type": "googlecompute", - "project_id": "{{user `GCP_PROJECT_ID`}}", - "zone": "us-central1-a", - "source_image": "{{user `UBUNTU_BASE_IMAGE`}}", - "image_name": "{{user `UBUNTU_BASE_IMAGE`}}{{user `BUILT_IMAGE_SUFFIX`}}", - "image_family": "{{user `UBUNTU_BASE_IMAGE`}}-libpod", - "service_account_email": "{{user `SERVICE_ACCOUNT`}}", - "communicator": "ssh", - "ssh_username": "{{user `GCE_SSH_USERNAME`}}", - "ssh_pty": "true" - } - ], - "provisioners": [ - { - "type": "file", - "source": "{{user `GOSRC`}}", - "destination": "/tmp/libpod" - },{ - "type": "shell", - "only": ["rhel-7"], - "script": "{{user `GOSRC`}}/{{user `PACKER_BASE`}}/rhel_setup.sh", - "environment_vars": [ - "SCRIPT_BASE={{user `SCRIPT_BASE`}}", - "CNI_COMMIT={{user `CNI_COMMIT`}}", - "CRIO_COMMIT={{user `CRIO_COMMIT`}}", - "CRIU_COMMIT={{user `CRIU_COMMIT`}}", - "RUNC_COMMIT={{user `RUNC_COMMIT`}}", - "RHSM_COMMAND={{user `RHSM_COMMAND`}}" - ] - },{ - "type": "shell", - "only": ["centos-7"], - "script": "{{user `GOSRC`}}/{{user `PACKER_BASE`}}/centos_setup.sh", - "environment_vars": [ - "SCRIPT_BASE={{user `SCRIPT_BASE`}}", - "CNI_COMMIT={{user `CNI_COMMIT`}}", - "CRIO_COMMIT={{user `CRIO_COMMIT`}}", - "CRIU_COMMIT={{user `CRIU_COMMIT`}}", - "RUNC_COMMIT={{user `RUNC_COMMIT`}}" - ] - },{ - "type": "shell", - "only": ["fedora-28"], - "script": "{{user `GOSRC`}}/{{user `PACKER_BASE`}}/fedora_setup.sh", - "environment_vars": [ - "SCRIPT_BASE={{user `SCRIPT_BASE`}}", - "CNI_COMMIT={{user `FEDORA_CNI_COMMIT`}}", - "CRIO_COMMIT={{user `CRIO_COMMIT`}}", - "CRIU_COMMIT={{user `CRIU_COMMIT`}}", - "RUNC_COMMIT={{user `RUNC_COMMIT`}}" - ] - },{ - "type": "shell", - "only": ["ubuntu-18"], - "script": "{{user `GOSRC`}}/{{user `PACKER_BASE`}}/ubuntu_setup.sh", - "environment_vars": [ - "SCRIPT_BASE={{user `SCRIPT_BASE`}}", - "CNI_COMMIT={{user `CNI_COMMIT`}}", - "CRIO_COMMIT={{user `CRIO_COMMIT`}}", - "CRIU_COMMIT={{user `CRIU_COMMIT`}}", - "RUNC_COMMIT={{user `RUNC_COMMIT`}}" - ] - } - ] -} diff --git a/contrib/cirrus/packer/libpod_images.yml b/contrib/cirrus/packer/libpod_images.yml new file mode 100644 index 000000000..d31c11a8d --- /dev/null +++ b/contrib/cirrus/packer/libpod_images.yml @@ -0,0 +1,96 @@ +--- + +# All of these are required +variables: + # Names of GCE Base images to start from, in .cirrus.yml + RHEL_BASE_IMAGE: '{{env `RHEL_BASE_IMAGE`}}' + CENTOS_BASE_IMAGE: '{{env `CENTOS_BASE_IMAGE`}}' + UBUNTU_BASE_IMAGE: '{{env `UBUNTU_BASE_IMAGE`}}' + FEDORA_BASE_IMAGE: '{{env `FEDORA_BASE_IMAGE`}}' + PRIOR_FEDORA_BASE_IMAGE: '{{env `PRIOR_FEDORA_BASE_IMAGE`}}' + FAH_BASE_IMAGE: '{{env `FAH_BASE_IMAGE`}}' + + # libpod dependencies to build and install into images + FEDORA_CNI_COMMIT: "{{env `FEDORA_CNI_COMMIT`}}" + CNI_COMMIT: "{{env `CNI_COMMIT`}}" + CRIO_COMMIT: "{{env `CRIO_COMMIT`}}" + CRIU_COMMIT: "{{env `CRIU_COMMIT`}}" + RUNC_COMMIT: "{{env `RUNC_COMMIT`}}" + + BUILT_IMAGE_SUFFIX: '{{env `BUILT_IMAGE_SUFFIX`}}' + GOSRC: '{{env `GOSRC`}}' + PACKER_BASE: '{{env `PACKER_BASE`}}' + SCRIPT_BASE: '{{env `SCRIPT_BASE`}}' + + # Protected credentials, decrypted by Cirrus at runtime + GCE_SSH_USERNAME: '{{env `GCE_SSH_USERNAME`}}' + GCP_PROJECT_ID: '{{env `GCP_PROJECT_ID`}}' + RHSM_COMMAND: '{{env `RHSM_COMMAND`}}' + SERVICE_ACCOUNT: '{{env `SERVICE_ACCOUNT`}}' + GOOGLE_APPLICATION_CREDENTIALS: '{{env `GOOGLE_APPLICATION_CREDENTIALS`}}' + +# Don't leak sensitive values in error messages / output +sensitive-variables: + - 'GCE_SSH_USERNAME' + - 'GCP_PROJECT_ID' + - 'RHSM_COMMAND' + - 'SERVICE_ACCOUNT' + +# What images to produce in which cloud +builders: + # v----- is a YAML anchor, allows referencing this object by name (below) + - &gce_hosted_image + name: 'ubuntu-18' + type: 'googlecompute' + image_name: '{{build_name}}{{user `BUILT_IMAGE_SUFFIX`}}' + image_family: '{{build_name}}-libpod' + source_image: '{{user `UBUNTU_BASE_IMAGE`}}' + disk_size: 20 + project_id: '{{user `GCP_PROJECT_ID`}}' + service_account_email: '{{user `SERVICE_ACCOUNT`}}' + communicator: 'ssh' + ssh_username: '{{user `GCE_SSH_USERNAME`}}' + ssh_pty: 'true' + # The only supported zone in Cirrus-CI, as of addition of this comment + zone: 'us-central1-a' + + # v----- is a YAML alias, allows partial re-use of the anchor object + - <<: *gce_hosted_image + name: 'rhel-7' + source_image: '{{user `RHEL_BASE_IMAGE`}}' + + - <<: *gce_hosted_image + name: 'centos-7' + source_image: '{{user `CENTOS_BASE_IMAGE`}}' + + - <<: *gce_hosted_image + name: 'fedora-29' + source_image: '{{user `FEDORA_BASE_IMAGE`}}' + + - <<: *gce_hosted_image + name: 'fedora-28' + source_image: '{{user `PRIOR_FEDORA_BASE_IMAGE`}}' + + - <<: *gce_hosted_image + name: 'fah-29' + source_image: '{{user `FAH_BASE_IMAGE`}}' + +# The brains of the operation, making actual modifications to the base-image. +provisioners: + - type: 'file' + source: '{{user `GOSRC`}}' + destination: '/tmp/libpod' + + - type: 'shell' + script: '{{user `GOSRC`}}/{{user `PACKER_BASE`}}/{{split build_name "-" 0}}_setup.sh' + environment_vars: + - 'SCRIPT_BASE={{user `SCRIPT_BASE`}}' + - 'CNI_COMMIT={{user `CNI_COMMIT`}}' + - 'FEDORA_CNI_COMMIT={{user `FEDORA_CNI_COMMIT`}}' + - 'CRIO_COMMIT={{user `CRIO_COMMIT`}}' + - 'CRIU_COMMIT={{user `CRIU_COMMIT`}}' + - 'RUNC_COMMIT={{user `RUNC_COMMIT`}}' + - 'RHSM_COMMAND={{user `RHSM_COMMAND`}}' + +post-processors: + - - type: 'manifest' diff --git a/contrib/cirrus/packer/make-user-data.sh b/contrib/cirrus/packer/make-user-data.sh new file mode 100644 index 000000000..7f7fa1c1a --- /dev/null +++ b/contrib/cirrus/packer/make-user-data.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +# This script is utilized by Makefile, it's not intended to be run by humans + +cat <<EOF > user-data +#cloud-config +timezone: US/Eastern +growpart: + mode: auto +disable_root: false +ssh_pwauth: True +ssh_import_id: [root] +ssh_authorized_keys: + - $(cat cidata.ssh.pub) +users: + - name: root + primary-group: root + homedir: /root + system: true +EOF diff --git a/contrib/cirrus/packer/prior_fedora_base-setup.sh b/contrib/cirrus/packer/prior_fedora_base-setup.sh new file mode 120000 index 000000000..998a5d9fd --- /dev/null +++ b/contrib/cirrus/packer/prior_fedora_base-setup.sh @@ -0,0 +1 @@ +fedora_base-setup.sh
\ No newline at end of file diff --git a/contrib/cirrus/packer/rhel_base-setup.sh b/contrib/cirrus/packer/rhel_base-setup.sh new file mode 100644 index 000000000..8b2073d4f --- /dev/null +++ b/contrib/cirrus/packer/rhel_base-setup.sh @@ -0,0 +1,52 @@ +#!/bin/bash + +# N/B: This script is not intended to be run by humans. It is used to configure the +# rhel base image for importing, so that it will boot in GCE + +set -e + +[[ "$1" == "post" ]] || exit 0 # pre stage is not needed + +# Load in library (copied by packer, before this script was run) +source $GOSRC/$SCRIPT_BASE/lib.sh + +req_env_var " + RHSM_COMMAND $RHSM_COMMAND +" + +install_ooe + +echo "Setting up repos" +# Frequently needed +ooe.sh sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm + +# Required for google to manage ssh keys +ooe.sh sudo tee /etc/yum.repos.d/google-cloud-sdk.repo << EOM +[google-cloud-compute] +name=google-cloud-compute +baseurl=https://packages.cloud.google.com/yum/repos/google-cloud-compute-el7-x86_64 +enabled=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg + https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg +EOM + +rhsm_enable + +echo "Installing/removing packages" +ooe.sh sudo yum -y install google-compute-engine google-compute-engine-oslogin +ooe.sh sudo yum -y erase "cloud-init" "rh-amazon-rhui-client*" || true +ooe.sh sudo systemctl enable \ + google-accounts-daemon \ + google-clock-skew-daemon \ + google-instance-setup \ + google-network-daemon \ + google-shutdown-scripts \ + google-startup-scripts + +rhel_exit_handler # release subscription! + +rh_finalize + +echo "SUCCESS!" diff --git a/contrib/cirrus/packer/rhel_setup.sh b/contrib/cirrus/packer/rhel_setup.sh index d296713fc..99376fd65 100644 --- a/contrib/cirrus/packer/rhel_setup.sh +++ b/contrib/cirrus/packer/rhel_setup.sh @@ -18,24 +18,7 @@ RHSM_COMMAND $RHSM_COMMAND install_ooe -export GOPATH="$(mktemp -d)" -export RHSMCMD="$(mktemp)" - -exit_handler() { - set +ex - cd / - sudo rm -rf "$RHSMCMD" - sudo rm -rf "$GOPATH" - sudo subscription-manager remove --all - sudo subscription-manager unregister - sudo subscription-manager clean -} -trap "exit_handler" EXIT - -# Avoid logging sensitive details -echo "$RHSM_COMMAND" > "$RHSMCMD" -ooe.sh sudo bash "$RHSMCMD" -sudo rm -rf "$RHSMCMD" +rhsm_enable ooe.sh sudo yum -y erase "rh-amazon-rhui-client*" ooe.sh sudo subscription-manager repos "--disable=*" @@ -47,26 +30,12 @@ ooe.sh sudo subscription-manager repos \ ooe.sh sudo yum -y update -# Frequently needed -ooe.sh sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm - -# Required for google to manage ssh keys -sudo tee -a /etc/yum.repos.d/google-cloud-sdk.repo << EOM -[google-cloud-compute] -name=google-cloud-compute -baseurl=https://packages.cloud.google.com/yum/repos/google-cloud-compute-el7-x86_64 -enabled=1 -gpgcheck=1 -repo_gpgcheck=1 -gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg - https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg -EOM - ooe.sh sudo yum -y install \ atomic-registries \ btrfs-progs-devel \ bzip2 \ device-mapper-devel \ + emacs-nox \ findutils \ glib2-devel \ glibc-static \ @@ -74,8 +43,6 @@ ooe.sh sudo yum -y install \ golang \ golang-github-cpuguy83-go-md2man \ golang-github-cpuguy83-go-md2man \ - google-compute-engine \ - google-compute-engine-oslogin \ gpgme-devel \ iptables \ libassuan-devel \ @@ -103,6 +70,7 @@ ooe.sh sudo yum -y install \ runc \ skopeo-containers \ unzip \ + vim \ which \ xz @@ -118,7 +86,7 @@ install_criu install_packer_copied_files -exit_handler # release subscription! +rhel_exit_handler # release subscription! rh_finalize diff --git a/contrib/cirrus/packer/ubuntu_setup.sh b/contrib/cirrus/packer/ubuntu_setup.sh index 4cf1f335b..af5671c90 100644 --- a/contrib/cirrus/packer/ubuntu_setup.sh +++ b/contrib/cirrus/packer/ubuntu_setup.sh @@ -21,11 +21,14 @@ install_ooe export GOPATH="$(mktemp -d)" trap "sudo rm -rf $GOPATH" EXIT +# Avoid getting stuck waiting for user input +export DEBIAN_FRONTEND=noninteractive + # Try twice as workaround for minor networking problems echo "Updating system and installing package dependencies" -ooe.sh sudo apt-get -qq update || sudo apt-get -qq update -ooe.sh sudo apt-get -qq upgrade || sudo apt-get -qq upgrade -ooe.sh sudo apt-get -qq install --no-install-recommends \ +ooe.sh sudo -E apt-get -qq update || sudo -E apt-get -qq update +ooe.sh sudo -E apt-get -qq upgrade || sudo -E apt-get -qq upgrade +ooe.sh sudo -E apt-get -qq install --no-install-recommends \ apparmor \ autoconf \ automake \ @@ -34,6 +37,7 @@ ooe.sh sudo apt-get -qq install --no-install-recommends \ build-essential \ curl \ e2fslibs-dev \ + emacs-nox \ gawk \ gettext \ go-md2man \ @@ -54,6 +58,8 @@ ooe.sh sudo apt-get -qq install --no-install-recommends \ libostree-dev \ libprotobuf-c0-dev \ libprotobuf-dev \ + libseccomp-dev \ + libseccomp2 \ libtool \ libudev-dev \ lsof \ @@ -71,6 +77,7 @@ ooe.sh sudo apt-get -qq install --no-install-recommends \ python3-setuptools \ socat \ unzip \ + vim \ xz-utils echo "Fixing Ubuntu kernel not enabling swap accounting by default" diff --git a/contrib/cirrus/setup_environment.sh b/contrib/cirrus/setup_environment.sh index 167db127f..174bd3daf 100755 --- a/contrib/cirrus/setup_environment.sh +++ b/contrib/cirrus/setup_environment.sh @@ -4,7 +4,6 @@ set -e source $(dirname $0)/lib.sh req_env_var " -CI $CI USER $USER HOME $HOME ENVLIB $ENVLIB @@ -16,12 +15,11 @@ CIRRUS_BUILD_ID $CIRRUS_BUILD_ID" cd "$CIRRUS_WORKING_DIR" # for clarity of initial conditions # Verify basic dependencies -for depbin in go rsync unzip sha256sum curl make +for depbin in go rsync unzip sha256sum curl make python3 git do if ! type -P "$depbin" &> /dev/null then - echo "ERROR: $depbin binary not found in $PATH" - exit 2 + echo "***** WARNING: $depbin binary not found in $PATH *****" fi done @@ -35,14 +33,15 @@ then # N/B: Single-quote items evaluated every time, double-quotes only once (right now). for envstr in \ "$MARK" \ + "export EPOCH_TEST_COMMIT=\"$CIRRUS_BASE_SHA\"" \ "export HEAD=\"$CIRRUS_CHANGE_IN_REPO\"" \ "export TRAVIS=\"1\"" \ "export GOSRC=\"$CIRRUS_WORKING_DIR\"" \ "export OS_RELEASE_ID=\"$(os_release_id)\"" \ "export OS_RELEASE_VER=\"$(os_release_ver)\"" \ - "export OS_REL_VER=\"${OS_RELEASE_ID}-${OS_RELEASE_VER}\"" \ + "export OS_REL_VER=\"$(os_release_id)-$(os_release_ver)\"" \ "export BUILT_IMAGE_SUFFIX=\"-$CIRRUS_REPO_NAME-${CIRRUS_CHANGE_IN_REPO:0:8}\"" \ - "export GOPATH=\"/go\"" \ + "export GOPATH=\"/var/tmp/go\"" \ 'export PATH="$HOME/bin:$GOPATH/bin:/usr/local/bin:$PATH"' \ 'export LD_LIBRARY_PATH="/usr/local/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"' do @@ -53,9 +52,16 @@ then # Some setup needs to vary between distros case "${OS_RELEASE_ID}-${OS_RELEASE_VER}" in ubuntu-18) + # Always install runc on Ubuntu + install_runc_from_git envstr='export BUILDTAGS="seccomp $($GOSRC/hack/btrfs_tag.sh) $($GOSRC/hack/btrfs_installed_tag.sh) $($GOSRC/hack/ostree_tag.sh) varlink exclude_graphdriver_devicemapper"' ;; - fedora-28) ;& # Continue to the next item + fedora-29) ;& # Continue to the next item + fedora-28) + RUNC="https://kojipkgs.fedoraproject.org/packages/runc/1.0.0/55.dev.git578fe65.fc${OS_RELEASE_VER}/x86_64/runc-1.0.0-55.dev.git578fe65.fc${OS_RELEASE_VER}.x86_64.rpm" + echo ">>>>> OVERRIDING RUNC WITH $RUNC <<<<<" + dnf -y install "$RUNC" + ;& # Continue to the next item centos-7) ;& rhel-7) envstr='unset BUILDTAGS' # Use default from Makefile diff --git a/contrib/cirrus/success.sh b/contrib/cirrus/success.sh index d1daf9043..2b0cf4655 100755 --- a/contrib/cirrus/success.sh +++ b/contrib/cirrus/success.sh @@ -1,22 +1,20 @@ #!/bin/bash set -e + source $(dirname $0)/lib.sh req_env_var " - CIRRUS_TASK_NAME $CIRRUS_TASK_NAME CIRRUS_BRANCH $CIRRUS_BRANCH - OS_RELEASE_ID $OS_RELEASE_ID - OS_RELEASE_VER $OS_RELEASE_VER - CIRRUS_REPO_CLONE_URL $CIRRUS_REPO_CLONE_URL + CIRRUS_BUILD_ID $CIRRUS_BUILD_ID " -REF_URL="$(echo $CIRRUS_REPO_CLONE_URL | sed 's/.git$//g')" +REF=$(basename $CIRRUS_BRANCH) # PR number or branch named +URL="https://cirrus-ci.com/build/$CIRRUS_BUILD_ID" + if [[ "$CIRRUS_BRANCH" =~ "pull" ]] then - REF_URL="$REF_URL/$CIRRUS_BRANCH" # pull request URL + ircmsg "Cirrus-CI testing successful for PR #$REF: $URL" else - REF_URL="$REF_URL/commits/$CIRRUS_BRANCH" # branch merge + ircmsg "Cirrus-CI testing branch $REF successful: $URL" fi - -ircmsg "Cirrus-CI $CIRRUS_TASK_NAME on $OS_RELEASE_ID-$OS_RELEASE_VER successful for $REF_URL" diff --git a/contrib/cirrus/verify_source.sh b/contrib/cirrus/system_test.sh index 860bafc00..66974f8c6 100755 --- a/contrib/cirrus/verify_source.sh +++ b/contrib/cirrus/system_test.sh @@ -4,11 +4,12 @@ set -e source $(dirname $0)/lib.sh req_env_var " +GOSRC $GOSRC OS_RELEASE_ID $OS_RELEASE_ID OS_RELEASE_VER $OS_RELEASE_VER " -show_env_vars +clean_env set -x cd "$GOSRC" @@ -16,15 +17,17 @@ cd "$GOSRC" case "${OS_RELEASE_ID}-${OS_RELEASE_VER}" in ubuntu-18) make install.tools "BUILDTAGS=$BUILDTAGS" - make validate "BUILDTAGS=$BUILDTAGS" - # make lint "BUILDTAGS=$BUILDTAGS" + make "BUILDTAGS=$BUILDTAGS" + make test-binaries "BUILDTAGS=$BUILDTAGS" ;; fedora-28) ;& centos-7) ;& rhel-7) make install.tools - make validate - # make lint + make + make test-binaries ;; *) bad_os_id_ver ;; esac + +make localsystem diff --git a/contrib/cirrus/unit_test.sh b/contrib/cirrus/unit_test.sh index cacc23045..61d9dc73d 100755 --- a/contrib/cirrus/unit_test.sh +++ b/contrib/cirrus/unit_test.sh @@ -9,22 +9,23 @@ OS_RELEASE_ID $OS_RELEASE_ID OS_RELEASE_VER $OS_RELEASE_VER " -show_env_vars +clean_env set -x cd "$GOSRC" case "${OS_RELEASE_ID}-${OS_RELEASE_VER}" in ubuntu-18) + make install.tools "BUILDTAGS=$BUILDTAGS" make localunit "BUILDTAGS=$BUILDTAGS" make "BUILDTAGS=$BUILDTAGS" ;; - fedora-28) + fedora-29) ;& # Continue to the next item + fedora-28) ;& + centos-7) ;& + rhel-7) + make install.tools make localunit make ;; - centos-7) ;& # Continue to the next item - rhel-7) - stub 'unit testing not working on $OS_RELEASE_ID' - ;; *) bad_os_id_ver ;; esac |