Cirrus: Document and codify base-image production

A number of images required for future testing are not present in GCE.
Importing them is a long proscribed process prone to errors and
complications.

Improve this situation by documenting, and encoding the majority of the
steps required.  Due to the required complexity, these are clearly
identified as 'semi-automated'.  This means a discerning eye is
sometimes needed to address unforeseen problems (networking issues,
format or packaging changes, etc).

Nevertheless, having these steps in writing, will reduce current and
future  maintenance burden while supporting future testing needs of
RHEL, Fedora and Fedora Atomic Host.

Also:

* Add necessary configuration, scripts, and Makefile updates needed to
  prepare RHEL, Fedora, & FAH cloud images for use in GCE.  This
  is a complex, multi-step process where the cloud image is booted
  un a local user-mod qemu-kvm instance, where it can be modified.
  From there, it's converted into a specific format, and imported into
  GCE.  Lastly, the imported raw disk data is made available as a GCE
  VM image.

  Note: As of this commit, the RHEL base-image builds (CentOS has native
  image), however neither RHEL or CentOS cache-images build correctly.

* Left testing on FAH disabled, the GCE/Cirrus integration needs needs more
  work.  Specifically, the python3-based google startup script service
  throws a permission-denied (as root) when trying to create a temp.
  directory.  Did not investigate further, though manually running the
  startup script does allow the libpod tests to start running.

* Enabled Fedora 29 image to execute tests and general use.

* Utilize the standardized F28-based container image  for gating
  of more the intensive unit and integration testing.  Update
  documentation to reflect this as the standard platform for
  these checks.  Rename tasks with shorter names and to better
  reflect their purpose.

* Cirrus: Trim unnecessary env vars before testing since the vast
  majority are only required for orchestration purposes.  Since most
  are defined within `.cirrus.yml`, it's a good place to store the
  list of undesirables.  Since each of the cirrus-scripts runs in
  it's own shell, unsetting these near the end will have no
  consequence.  Also trim down the number of calls to show_env_vars()

Signed-off-by: Chris Evich <cevich@redhat.com>

1 files changed, 52 insertions, 0 deletions

diff --git a/contrib/cirrus/packer/Makefile b/contrib/cirrus/packer/Makefile
index d3a34877f..9bf27373e 100644
--- a/contrib/cirrus/packer/Makefile
+++ b/contrib/cirrus/packer/Makefile
@@ -54,3 +54,55 @@ endif
 	@echo "Finished.  The images mentioned above, and in packer-manifest.json"
 	@echo "can be used in .cirrus.yml as values for the 'image_name' keys"
 	@echo ""
+
+cidata.ssh:
+	ssh-keygen -f $@ -P "" -q
+
+cidata.ssh.pub: cidata.ssh
+	touch $@
+
+meta-data:
+	echo "local-hostname: localhost.localdomain" > $@
+
+user-data: cidata.ssh.pub
+	bash make-user-data.sh
+
+cidata.iso: user-data meta-data
+	genisoimage -output cidata.iso -volid cidata -input-charset utf-8 -joliet -rock user-data meta-data
+
+# This is intended to be run by a human, with admin access to the libpod GCE project.
+.PHONY: libpod_base_images
+libpod_base_images: libpod_base_images.json cidata.iso cidata.ssh packer
+ifndef GCP_PROJECT_ID
+	$(error GCP_PROJECT_ID is undefined, expected complete GCP project ID string e.g. foobar-12345)
+endif
+ifndef GOOGLE_APPLICATION_CREDENTIALS
+	$(error GOOGLE_APPLICATION_CREDENTIALS is undefined, expected absolute path to JSON file, like $HOME/.config/gcloud/legacy_credentials/*/adc.json)
+endif
+ifndef RHEL_IMAGE_FILE
+	$(error RHEL_IMAGE_FILE is undefined, expected full path to a rhel-server-ec2-*.raw.xz file)
+endif
+ifndef RHEL_CSUM_FILE
+	$(error RHEL_CSUM_FILE is undefined, expected full path to a rhel-server-ec2-*.raw.xz.SHA256SUM file)
+endif
+ifndef RHSM_COMMAND
+	$(error RHSM_COMMAND is undefined, expected string required for temporarily registering VM)
+endif
+	PACKER_CACHE_DIR=/tmp ./packer build \
+		-var TIMESTAMP=$(TIMESTAMP) \
+		-var TTYDEV=$(TTYDEV) \
+		-var GCP_PROJECT_ID=$(GCP_PROJECT_ID) \
+		-var GOOGLE_APPLICATION_CREDENTIALS=$(GOOGLE_APPLICATION_CREDENTIALS) \
+		-var GOSRC=$(GOSRC) \
+		-var PACKER_BASE=$(PACKER_BASE) \
+		-var SCRIPT_BASE=$(SCRIPT_BASE) \
+		-var RHEL_BASE_IMAGE_NAME=$(shell basename $(RHEL_IMAGE_FILE) | tr -d '[[:space:]]' | sed -r -e 's/\.x86_64\.raw\.xz//' | tr '[[:upper:]]' '[[:lower:]]' | tr '[[:punct:]]' '-') \
+		-var RHEL_IMAGE_FILE=$(RHEL_IMAGE_FILE) \
+		-var RHEL_CSUM_FILE=$(RHEL_CSUM_FILE) \
+	    -var 'RHSM_COMMAND=$(RHSM_COMMAND)' \
+		-only $(PACKER_BUILDS) \
+		libpod_base_images.json
+	@echo ""
+	@echo "Finished.  The images mentioned above, and in packer-manifest.json"
+	@echo "can be used in .cirrus.yml as values for the *_BASE_IMAGE keys."
+	@echo ""