Cirrus: Document and codify base-image production

A number of images required for future testing are not present in GCE.
Importing them is a long proscribed process prone to errors and
complications.

Improve this situation by documenting, and encoding the majority of the
steps required.  Due to the required complexity, these are clearly
identified as 'semi-automated'.  This means a discerning eye is
sometimes needed to address unforeseen problems (networking issues,
format or packaging changes, etc).

Nevertheless, having these steps in writing, will reduce current and
future  maintenance burden while supporting future testing needs of
RHEL, Fedora and Fedora Atomic Host.

Also:

* Add necessary configuration, scripts, and Makefile updates needed to
  prepare RHEL, Fedora, & FAH cloud images for use in GCE.  This
  is a complex, multi-step process where the cloud image is booted
  un a local user-mod qemu-kvm instance, where it can be modified.
  From there, it's converted into a specific format, and imported into
  GCE.  Lastly, the imported raw disk data is made available as a GCE
  VM image.

  Note: As of this commit, the RHEL base-image builds (CentOS has native
  image), however neither RHEL or CentOS cache-images build correctly.

* Left testing on FAH disabled, the GCE/Cirrus integration needs needs more
  work.  Specifically, the python3-based google startup script service
  throws a permission-denied (as root) when trying to create a temp.
  directory.  Did not investigate further, though manually running the
  startup script does allow the libpod tests to start running.

* Enabled Fedora 29 image to execute tests and general use.

* Utilize the standardized F28-based container image  for gating
  of more the intensive unit and integration testing.  Update
  documentation to reflect this as the standard platform for
  these checks.  Rename tasks with shorter names and to better
  reflect their purpose.

* Cirrus: Trim unnecessary env vars before testing since the vast
  majority are only required for orchestration purposes.  Since most
  are defined within `.cirrus.yml`, it's a good place to store the
  list of undesirables.  Since each of the cirrus-scripts runs in
  it's own shell, unsetting these near the end will have no
  consequence.  Also trim down the number of calls to show_env_vars()

Signed-off-by: Chris Evich <cevich@redhat.com>

1 files changed, 45 insertions, 1 deletions

diff --git a/contrib/cirrus/lib.sh b/contrib/cirrus/lib.sh
index ff5925d5d..985264f22 100644
--- a/contrib/cirrus/lib.sh
+++ b/contrib/cirrus/lib.sh
@@ -73,6 +73,18 @@ PACKER_BUILDS $PACKER_BUILDS
     do
         [[ -z "$NAME" ]] || echo "export $NAME=\"$VALUE\""
     done
+    echo ""
+    echo "##### $(go version) #####"
+    echo ""
+}
+
+# Unset environment variables not needed for testing purposes
+clean_env() {
+    req_env_var "
+        UNSET_ENV_VARS $UNSET_ENV_VARS
+    "
+    echo "Unsetting $(echo $UNSET_ENV_VARS | wc -w) environment variables"
+    unset -v UNSET_ENV_VARS $UNSET_ENV_VARS || true  # don't fail on read-only
 }
 
 # Return a GCE image-name compatible string representation of distribution name
@@ -269,6 +281,8 @@ install_varlink(){
 }
 
 _finalize(){
+    set +e  # Don't fail at the very end
+    set +e  # make errors non-fatal
     echo "Removing leftover giblets from cloud-init"
     cd /
     sudo rm -rf /var/lib/cloud/instance?
@@ -281,13 +295,15 @@ _finalize(){
 }
 
 rh_finalize(){
+    set +e  # Don't fail at the very end
     # Allow root ssh-logins
     if [[ -r /etc/cloud/cloud.cfg ]]
     then
         sudo sed -re 's/^disable_root:.*/disable_root: 0/g' -i /etc/cloud/cloud.cfg
     fi
     echo "Resetting to fresh-state for usage as cloud-image."
-    sudo $(type -P dnf || type -P yum) clean all
+    PKG=$(type -P dnf || type -P yum || echo "")
+    [[ -z "$PKG" ]] || sudo $PKG clean all  # not on atomic
     sudo rm -rf /var/cache/{yum,dnf}
     sudo rm -f /etc/udev/rules.d/*-persistent-*.rules
     sudo touch /.unconfigured  # force firstboot to run
@@ -295,7 +311,35 @@ rh_finalize(){
 }
 
 ubuntu_finalize(){
+    set +e  # Don't fail at the very end
     echo "Resetting to fresh-state for usage as cloud-image."
     sudo rm -rf /var/cache/apt
     _finalize
 }
+
+rhel_exit_handler() {
+    set +ex
+    req_env_var "
+        GOPATH $GOPATH
+        RHSMCMD $RHSMCMD
+    "
+    cd /
+    sudo rm -rf "$RHSMCMD"
+    sudo rm -rf "$GOPATH"
+    sudo subscription-manager remove --all
+    sudo subscription-manager unregister
+    sudo subscription-manager clean
+}
+
+rhsm_enable() {
+    req_env_var "
+        RHSM_COMMAND $RHSM_COMMAND
+    "
+    export GOPATH="$(mktemp -d)"
+    export RHSMCMD="$(mktemp)"
+    trap "rhel_exit_handler" EXIT
+    # Avoid logging sensitive details
+    echo "$RHSM_COMMAND" > "$RHSMCMD"
+    ooe.sh sudo bash "$RHSMCMD"
+    sudo rm -rf "$RHSMCMD"
+}