summaryrefslogtreecommitdiff
path: root/contrib
diff options
context:
space:
mode:
authorChris Evich <cevich@redhat.com>2020-08-27 17:31:59 -0400
committerChris Evich <cevich@redhat.com>2020-09-09 13:46:52 -0400
commited1e87ecb57d2280ef3b02e2b437ad9b378d21a2 (patch)
tree679a97d6d188926b0db87a4e6ecdf9869925fdd5 /contrib
parent5a09fd8f2b8e624a8d4155fd4fc89f51e544e2ca (diff)
downloadpodman-ed1e87ecb57d2280ef3b02e2b437ad9b378d21a2.tar.gz
podman-ed1e87ecb57d2280ef3b02e2b437ad9b378d21a2.tar.bz2
podman-ed1e87ecb57d2280ef3b02e2b437ad9b378d21a2.zip
Cirrus: Obsolete CI:IMG process & related files
All VM-building functionality has been migrated to https://github.com/containers/automation_images Some container-build functions are still maintained here but are on a very-short list to also be migrated to the repository linked above. Signed-off-by: Chris Evich <cevich@redhat.com>
Diffstat (limited to 'contrib')
-rw-r--r--contrib/cirrus/README.md182
-rw-r--r--contrib/cirrus/add_second_partition.sh3
-rwxr-xr-xcontrib/cirrus/build_vm_images.sh67
-rwxr-xr-xcontrib/cirrus/check_image.sh85
-rw-r--r--contrib/cirrus/git_authors_to_irc_nicks.csv12
-rw-r--r--contrib/cirrus/lib.sh178
-rwxr-xr-xcontrib/cirrus/lib.sh.t44
-rwxr-xr-xcontrib/cirrus/notice_branch_failure.sh19
-rw-r--r--contrib/cirrus/packer/.gitignore7
-rw-r--r--contrib/cirrus/packer/Makefile94
-rw-r--r--contrib/cirrus/packer/README.how-to-update-cirrus-vms89
-rw-r--r--contrib/cirrus/packer/README.md3
-rw-r--r--contrib/cirrus/packer/cloud-init/fedora/cloud-init.service20
-rw-r--r--contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/40_enable_root.cfg1
-rw-r--r--contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/50_custom_disk_setup.cfg4
-rw-r--r--contrib/cirrus/packer/cloud-init/ubuntu/cloud.cfg.d/40_enable_root.cfg1
-rw-r--r--contrib/cirrus/packer/fedora_base-setup.sh44
-rw-r--r--contrib/cirrus/packer/fedora_packaging.sh194
-rw-r--r--contrib/cirrus/packer/fedora_setup.sh34
-rw-r--r--contrib/cirrus/packer/image-builder-image_base-setup.sh71
-rw-r--r--contrib/cirrus/packer/libpod_base_images.yml164
-rw-r--r--contrib/cirrus/packer/libpod_images.yml86
-rw-r--r--contrib/cirrus/packer/make-user-data.sh20
-rw-r--r--contrib/cirrus/packer/prior-fedora_base-setup.sh44
-rwxr-xr-xcontrib/cirrus/packer/systemd_banish.sh28
-rw-r--r--contrib/cirrus/packer/ubuntu_packaging.sh175
-rw-r--r--contrib/cirrus/packer/ubuntu_setup.sh35
-rw-r--r--contrib/cirrus/packer/xfedora_setup.sh34
-rwxr-xr-xcontrib/cirrus/podbot.py105
-rwxr-xr-xcontrib/cirrus/setup_environment.sh21
-rwxr-xr-xcontrib/cirrus/success.sh66
31 files changed, 19 insertions, 1911 deletions
diff --git a/contrib/cirrus/README.md b/contrib/cirrus/README.md
index 977762293..f66560cc8 100644
--- a/contrib/cirrus/README.md
+++ b/contrib/cirrus/README.md
@@ -76,95 +76,6 @@ exercising cgroups v2 with Podman integration tests. Also depends on
having `SPECIALMODE` set to 'cgroupv2`
-### ``test_build_cache_images_task`` Task
-
-Modifying the contents of cache-images is tested by making changes to
-one or more of the ``./contrib/cirrus/packer/*_setup.sh`` files. Then
-in the PR description, add the magic string: ``[CI:IMG]``
-
-***N/B: Steps below are performed by automation***
-
-1. ``setup_environment.sh``: Same as for other tasks.
-
-2. ``build_vm_images.sh``: Utilize [the packer tool](http://packer.io/docs/)
- to produce new VM images. Create a new VM from each base-image, connect
- to them with ``ssh``, and perform the steps as defined by the
- ``$PACKER_BASE/libpod_images.yml`` file:
-
- 1. On a base-image VM, as root, copy the current state of the repository
- into ``/tmp/libpod``.
- 2. Execute distribution-specific scripts to prepare the image for
- use. For example, ``fedora_setup.sh``.
- 3. If successful, shut down each VM and record the names, and dates
- into a json manifest file.
- 4. Move the manifest file, into a google storage bucket object.
- This is a retained as a secondary method for tracking/auditing
- creation of VM images, should it ever be needed.
-
-### ``verify_test_built_images`` Task
-
-Only runs following successful ``test_build_cache_images_task`` task. Uses
-images following the standard naming format; ***however, only runs a limited
-sub-set of automated tests***. Validating newly built images fully, requires
-updating ``.cirrus.yml``.
-
-***N/B: Steps below are performed by automation***
-
-1. Using the just build VM images, launch VMs and wait for them to boot.
-
-2. Execute the `setup_environment.sh` as in the `testing` task.
-
-2. Execute the `integration_test.sh` as in the `testing` task.
-
-
-***Manual Steps:*** Assuming the automated steps pass, then
-you'll find the new image names displayed at the end of the
-`test_build_cache_images`. For example:
-
-
-```
-...cut...
-
-[+0747s] ==> Builds finished. The artifacts of successful builds are:
-[+0747s] --> ubuntu-18: A disk image was created: ubuntu-18-libpod-5664838702858240
-[+0747s] --> fedora-29: A disk image was created: fedora-29-libpod-5664838702858240
-[+0747s] --> fedora-30: A disk image was created: fedora-30-libpod-5664838702858240
-[+0747s] --> ubuntu-19: A disk image was created: ubuntu-19-libpod-5664838702858240
-```
-
-Notice the suffix on all the image names comes from the env. var. set in
-*.cirrus.yml*: `BUILT_IMAGE_SUFFIX: "-${CIRRUS_REPO_NAME}-${CIRRUS_BUILD_ID}"`.
-Edit `.cirrus.yml`, in the top-level `env` section, update the suffix variable
-used at runtime to launch VMs for testing:
-
-
-```yaml
-env:
- ...cut...
- ####
- #### Cache-image names to test with (double-quotes around names are critical)
- ###
- _BUILT_IMAGE_SUFFIX: "libpod-5664838702858240"
- FEDORA_CACHE_IMAGE_NAME: "fedora-30-${_BUILT_IMAGE_SUFFIX}"
- PRIOR_FEDORA_CACHE_IMAGE_NAME: "fedora-29-${_BUILT_IMAGE_SUFFIX}"
- ...cut...
-```
-
-***NOTES:***
-* If re-using the same PR with new images in `.cirrus.yml`,
- take care to also *update the PR description* to remove
- the magic ``[CI:IMG]`` string. Keeping it and
- `--force` pushing would needlessly cause Cirrus-CI to build
- and test images again.
-* In the future, if you need to review the log from the build that produced
- the referenced image:
-
- * Note the Build ID from the image name (for example `5664838702858240`).
- * Go to that build in the Cirrus-CI WebUI, using the build ID in the URL.
- (For example `https://cirrus-ci.com/build/5664838702858240`.
- * Choose the *test_build_cache_images* task.
- * Open the *build_vm_images* script section.
-
### `docs` Task
Builds swagger API documentation YAML and uploads to google storage (an online
@@ -226,99 +137,6 @@ gsutil cors set /path/to/file.json gs://libpod-master-releases
file. Therefore, if it is not functioning or misconfigured, a person must have altered it or
changes were made to the referring site (e.g. `docs.podman.io`).
-## Base-images
-
-Base-images are VM disk-images specially prepared for executing as GCE VMs.
-In particular, they run services on startup similar in purpose/function
-as the standard 'cloud-init' services.
-
-* The google services are required for full support of ssh-key management
- and GCE OAuth capabilities. Google provides native images in GCE
- with services pre-installed, for many platforms. For example,
- RHEL, CentOS, and Ubuntu.
-
-* Google does ***not*** provide any images for Fedora (as of 5/2019), nor do
- they provide a base-image prepared to run packer for creating other images
- in the ``test_build_vm_images`` Task (above).
-
-* Base images do not need to be produced often, but doing so completely
- manually would be time-consuming and error-prone. Therefore a special
- semi-automatic *Makefile* target is provided to assist with producing
- all the base-images: ``libpod_base_images``
-
-To produce new base-images, including an `image-builder-image` (used by
-the ``cache_images`` Task) some input parameters are required:
-
-* ``GCP_PROJECT_ID``: The complete GCP project ID string e.g. foobar-12345
- identifying where the images will be stored.
-
-* ``GOOGLE_APPLICATION_CREDENTIALS``: A *JSON* file containing
- credentials for a GCE service account. This can be [a service
- account](https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually)
- or [end-user
- credentials](https://cloud.google.com/docs/authentication/end-user#creating_your_client_credentials)
-
-* Optionally, CSV's may be specified to ``PACKER_BUILDS``
- to limit the base-images produced. For example,
- ``PACKER_BUILDS=fedora,image-builder-image``.
-
-If there is no existing 'image-builder-image' within GCE, a new
-one may be bootstrapped by creating a CentOS 7 VM with support for
-nested-virtualization, and with elevated cloud privileges (to access
-GCE, from within the GCE VM). For example:
-
-```
-$ alias pgcloud='sudo podman run -it --rm -e AS_ID=$UID
- -e AS_USER=$USER -v $HOME:$HOME:z quay.io/cevich/gcloud_centos:latest'
-
-$ URL=https://www.googleapis.com/auth
-$ SCOPES=$URL/userinfo.email,$URL/compute,$URL/devstorage.full_control
-
-# The --min-cpu-platform is critical for nested-virt.
-$ pgcloud compute instances create $USER-image-builder \
- --image-family centos-7 \
- --boot-disk-size "200GB" \
- --min-cpu-platform "Intel Haswell" \
- --machine-type n1-standard-2 \
- --scopes $SCOPES
-```
-
-Then from that VM, execute the
-``contrib/cirrus/packer/image-builder-image_base_setup.sh`` script.
-Shutdown the VM, and convert it into a new image-builder-image.
-
-Building new base images is done by first creating a VM from an
-image-builder-image and copying the credentials json file to it.
-
-```
-$ hack/get_ci_vm.sh image-builder-image-1541772081
-...in another terminal...
-$ pgcloud compute scp /path/to/gac.json $USER-image-builder-image-1541772081:.
-```
-
-Then, on the VM, change to the ``packer`` sub-directory, and build the images:
-
-```
-$ cd libpod/contrib/cirrus/packer
-$ make libpod_base_images GCP_PROJECT_ID=<VALUE> \
- GOOGLE_APPLICATION_CREDENTIALS=/path/to/gac.json \
- PACKER_BUILDS=<OPTIONAL>
-```
-
-Assuming this is successful (hence the semi-automatic part), packer will
-produce a ``packer-manifest.json`` output file. This contains the base-image
-names suitable for updating in ``.cirrus.yml``, `env` keys ``*_BASE_IMAGE``.
-
-On failure, it should be possible to determine the problem from the packer
-output. Sometimes that means setting `PACKER_LOG=1` and troubleshooting
-the nested virt calls. It's also possible to observe the (nested) qemu-kvm
-console output. Simply set the ``TTYDEV`` parameter, for example:
-
-```
-$ make libpod_base_images ... TTYDEV=$(tty)
- ...
-```
-
## `$SPECIALMODE`
Some tasks alter their behavior based on this value. A summary of supported
diff --git a/contrib/cirrus/add_second_partition.sh b/contrib/cirrus/add_second_partition.sh
index 3c2f9f056..d0407be86 100644
--- a/contrib/cirrus/add_second_partition.sh
+++ b/contrib/cirrus/add_second_partition.sh
@@ -7,8 +7,7 @@
SLASH_DEVICE="/dev/sda" # Always the case on GCP
# The unallocated space results from the difference in disk-size between VM Image
-# and runtime request. The check_image.sh test includes a minimum-space check,
-# with the Image size set initially lower by contrib/cirrus/packer/libpod_images.yml
+# and runtime request.
NEW_PART_START="50%"
NEW_PART_END="100%"
diff --git a/contrib/cirrus/build_vm_images.sh b/contrib/cirrus/build_vm_images.sh
deleted file mode 100755
index be1c82185..000000000
--- a/contrib/cirrus/build_vm_images.sh
+++ /dev/null
@@ -1,67 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-source $(dirname $0)/lib.sh
-
-BASE_IMAGE_VARS='FEDORA_BASE_IMAGE PRIOR_FEDORA_BASE_IMAGE UBUNTU_BASE_IMAGE PRIOR_UBUNTU_BASE_IMAGE'
-ENV_VARS="PACKER_BUILDS BUILT_IMAGE_SUFFIX $BASE_IMAGE_VARS SERVICE_ACCOUNT GCE_SSH_USERNAME GCP_PROJECT_ID PACKER_VER SCRIPT_BASE PACKER_BASE CIRRUS_BUILD_ID CIRRUS_CHANGE_IN_REPO"
-req_env_var $ENV_VARS
-# Must also be made available through make, into packer process
-export $ENV_VARS
-
-# Everything here is running on the 'image-builder-image' GCE image
-# Assume basic dependencies are all met, but there could be a newer version
-# of the packer binary
-PACKER_FILENAME="packer_${PACKER_VER}_linux_amd64.zip"
-if [[ -d "$HOME/packer" ]]
-then
- cd "$HOME/packer"
- # image_builder_image has packer pre-installed, check if same version requested
- if [[ -r "$PACKER_FILENAME" ]]
- then
- cp $PACKER_FILENAME "$GOSRC/$PACKER_BASE/"
- cp packer "$GOSRC/$PACKER_BASE/"
- fi
-fi
-
-cd "$GOSRC/$PACKER_BASE"
-# Add/update labels on base-images used in this build to prevent premature deletion
-ARGS="
-"
-for base_image_var in $BASE_IMAGE_VARS
-do
- # See entrypoint.sh in contrib/imgts and contrib/imgprune
- # These updates can take a while, run them in the background, check later
- gcloud compute images update \
- --update-labels=last-used=$(date +%s) \
- --update-labels=build-id=$CIRRUS_BUILD_ID \
- --update-labels=repo-ref=$CIRRUS_CHANGE_IN_REPO \
- --update-labels=project=$GCP_PROJECT_ID \
- ${!base_image_var} &
-done
-
-make libpod_images \
- PACKER_BUILDS=$PACKER_BUILDS \
- PACKER_VER=$PACKER_VER \
- GOSRC=$GOSRC \
- SCRIPT_BASE=$SCRIPT_BASE \
- PACKER_BASE=$PACKER_BASE \
- BUILT_IMAGE_SUFFIX=$BUILT_IMAGE_SUFFIX
-
-# Separate PR-produced images from those produced on master.
-if [[ "${CIRRUS_BRANCH:-}" == "master" ]]
-then
- POST_MERGE_BUCKET_SUFFIX="-master"
-else
- POST_MERGE_BUCKET_SUFFIX=""
-fi
-
-# When successful, upload manifest of produced images using a filename unique
-# to this build.
-URI="gs://packer-import${POST_MERGE_BUCKET_SUFFIX}/manifest${BUILT_IMAGE_SUFFIX}.json"
-gsutil cp packer-manifest.json "$URI"
-
-# Ensure any background 'gcloud compute images update' processes finish
-wait # No -n option in CentOS, this is the best that can be done :(
-
-echo "Finished. A JSON manifest of produced images is available at $URI"
diff --git a/contrib/cirrus/check_image.sh b/contrib/cirrus/check_image.sh
deleted file mode 100755
index 04867ca64..000000000
--- a/contrib/cirrus/check_image.sh
+++ /dev/null
@@ -1,85 +0,0 @@
-#!/usr/bin/env bash
-
-set -eo pipefail
-
-source $(dirname $0)/lib.sh
-
-EVIL_UNITS="$($CIRRUS_WORKING_DIR/$PACKER_BASE/systemd_banish.sh --list)"
-
-req_env_var PACKER_BUILDER_NAME RCLI EVIL_UNITS OS_RELEASE_ID CG_FS_TYPE
-
-NFAILS=0
-echo "Validating VM image"
-
-MIN_SLASH_GIGS=30
-read SLASH_DEVICE SLASH_FSTYPE SLASH_SIZE JUNK <<<$(findmnt --df --first-only --noheadings / | cut -d '.' -f 1)
-SLASH_SIZE_GIGS=$(echo "$SLASH_SIZE" | sed -r -e 's/G|g//')
-item_test "Minimum available disk space" $SLASH_SIZE_GIGS -gt $MIN_SLASH_GIGS || let "NFAILS+=1"
-
-MIN_MEM_MB=2000
-read JUNK TOTAL USED MEM_FREE JUNK <<<$(free -tm | tail -1)
-item_test 'Minimum available memory' $MEM_FREE -ge $MIN_MEM_MB || let "NFAILS+=1"
-
-# We're testing a custom-built podman; make sure there isn't a distro-provided
-# binary anywhere; that could potentially taint our results.
-remove_packaged_podman_files
-item_test "remove_packaged_podman_files() does it's job" -z "$(type -P podman)" || let "NFAILS+=1"
-
-MIN_ZIP_VER='3.0'
-VER_RE='.+([[:digit:]]+\.[[:digit:]]+).+'
-ACTUAL_VER=$(zip --version 2>&1 | egrep -m 1 "Zip$VER_RE" | sed -r -e "s/$VER_RE/\\1/")
-item_test "minimum zip version" "$MIN_ZIP_VER" = $(echo -e "$MIN_ZIP_VER\n$ACTUAL_VER" | sort -V | head -1) || let "NFAILS+=1"
-
-for REQ_UNIT in google-accounts-daemon.service \
- google-clock-skew-daemon.service \
- google-instance-setup.service \
- google-network-daemon.service \
- google-shutdown-scripts.service \
- google-startup-scripts.service
-do
- # enabled/disabled appears at the end of the line, on some Ubuntu's it appears twice
- service_status=$(systemctl list-unit-files --no-legend $REQ_UNIT | tac -s ' ' | head -1)
- item_test "required $REQ_UNIT status is enabled" \
- "$service_status" = "enabled" || let "NFAILS+=1"
-done
-
-for evil_unit in $EVIL_UNITS
-do
- # Exits zero if any unit matching pattern is running
- unit_status=$(systemctl is-active $evil_unit &> /dev/null; echo $?)
- item_test "No $evil_unit unit is present or active:" "$unit_status" -ne "0" || let "NFAILS+=1"
-done
-
-echo "Checking items specific to ${PACKER_BUILDER_NAME}${BUILT_IMAGE_SUFFIX}"
-case "$PACKER_BUILDER_NAME" in
- ubuntu*)
- item_test "On ubuntu, no periodic apt crap is enabled" -z "$(egrep $PERIODIC_APT_RE /etc/apt/apt.conf.d/*)"
- ;;
- fedora*)
- # Only runc -OR- crun should be installed, never both
- case "$CG_FS_TYPE" in
- tmpfs)
- HAS=runc
- HAS_NOT=crun
- ;;
- cgroup2fs)
- HAS=crun
- HAS_NOT=runc
- ;;
- esac
- HAS_RC=$(rpm -qV $HAS &> /dev/null; echo $?)
- HAS_NOT_RC=$(rpm -qV $HAS_NOT &> /dev/null; echo $?)
- item_test "With a cgroups-fs type $CG_FS_TYPE, the $HAS package is installed" $HAS_RC -eq 0
- item_test "With a cgroups-fs type $CG_FS_TYPE, the $HAS_NOT package is not installed" $HAS_NOT_RC -ne 0
- ;;
- xfedora*)
- echo "Kernel Command-line: $(cat /proc/cmdline)"
- item_test \
- "On ${PACKER_BUILDER_NAME} images, the /sys/fs/cgroup/unified directory does NOT exist" \
- "!" "-d" "/sys/fs/cgroup/unified" || let "NFAILS+=1"
- ;;
- *) echo "No vm-image specific items to check"
-esac
-
-echo "Total failed tests: $NFAILS"
-exit $NFAILS
diff --git a/contrib/cirrus/git_authors_to_irc_nicks.csv b/contrib/cirrus/git_authors_to_irc_nicks.csv
deleted file mode 100644
index a584cc76a..000000000
--- a/contrib/cirrus/git_authors_to_irc_nicks.csv
+++ /dev/null
@@ -1,12 +0,0 @@
-# Comma separated mapping of author e-mail, to Freenode IRC nick.
-# When no match is found here, the username portion of the e-mail is used.
-# Sorting is done at runtime - first-found e-mail match wins.
-# Comments (like this) and blank lines are ignored.
-
-bbaude@redhat.com,baude
-matthew.heon@pm.me,mheon
-matthew.heon@gmail.com,mheon
-emilien@redhat.com,EmilienM
-rothberg@redhat.com,vrothberg
-santiago@redhat.com,edsantiago
-gscrivan@redhat.com,giuseppe
diff --git a/contrib/cirrus/lib.sh b/contrib/cirrus/lib.sh
index 3292e9d14..f125dd76d 100644
--- a/contrib/cirrus/lib.sh
+++ b/contrib/cirrus/lib.sh
@@ -35,10 +35,8 @@ export PATH="$HOME/bin:$GOPATH/bin:/usr/local/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"
# Saves typing / in case location ever moves
SCRIPT_BASE=${SCRIPT_BASE:-./contrib/cirrus}
-PACKER_BASE=${PACKER_BASE:-./contrib/cirrus/packer}
# Important filepaths
SETUP_MARKER_FILEPATH="${SETUP_MARKER_FILEPATH:-/var/tmp/.setup_environment_sh_complete}"
-AUTHOR_NICKS_FILEPATH="${CIRRUS_WORKING_DIR}/${SCRIPT_BASE}/git_authors_to_irc_nicks.csv"
# Downloaded, but not installed packages.
PACKAGE_DOWNLOAD_DIR=/var/cache/download
@@ -61,22 +59,15 @@ CONTINUOUS_INTEGRATION="${CONTINUOUS_INTEGRATION:-false}"
CIRRUS_REPO_NAME=${CIRRUS_REPO_NAME:-libpod}
CIRRUS_BASE_SHA=${CIRRUS_BASE_SHA:-unknown$(date +%s)} # difficult to reliably discover
CIRRUS_BUILD_ID=${CIRRUS_BUILD_ID:-$RANDOM$(date +%s)} # must be short and unique
-# Vars. for image-building
-PACKER_VER="1.4.2"
-# CSV of cache-image names to build (see $PACKER_BASE/libpod_images.json)
-
-# List of cache imaes to build for 'CI:IMG' mode via build_vm_images.sh
-# Exists to support manual single-image building in case of emergency
-export PACKER_BUILDS="${PACKER_BUILDS:-ubuntu-20,ubuntu-19,fedora-32,fedora-31}"
-# Google cloud provides these, we just make copies (see $SCRIPT_BASE/README.md) for use
-export UBUNTU_BASE_IMAGE="ubuntu-2004-focal-v20200506"
-export PRIOR_UBUNTU_BASE_IMAGE="ubuntu-1910-eoan-v20200211"
-# Manually produced base-image names (see $SCRIPT_BASE/README.md)
-export FEDORA_BASE_IMAGE="fedora-cloud-base-32-1-6-1588257430"
-export PRIOR_FEDORA_BASE_IMAGE="fedora-cloud-base-31-1-9-1588257430"
-export BUILT_IMAGE_SUFFIX="${BUILT_IMAGE_SUFFIX:--$CIRRUS_REPO_NAME-${CIRRUS_BUILD_ID}}"
+
+OS_RELEASE_ID="$(source /etc/os-release; echo $ID)"
+# GCE image-name compatible string representation of distribution _major_ version
+OS_RELEASE_VER="$(source /etc/os-release; echo $VERSION_ID | cut -d '.' -f 1)"
+# Combined to ease soe usage
+OS_REL_VER="${OS_RELEASE_ID}-${OS_RELEASE_VER}"
+
# IN_PODMAN container image
-IN_PODMAN_IMAGE="quay.io/libpod/in_podman:$DEST_BRANCH"
+IN_PODMAN_IMAGE="quay.io/libpod/${OS_RELEASE_ID}_podman:$_BUILT_IMAGE_SUFFIX"
# Image for uploading releases
UPLDREL_IMAGE="quay.io/libpod/upldrel:master"
@@ -98,7 +89,7 @@ BIGTO="timeout_attempt_delay_command 300s 5 60s"
# Safe env. vars. to transfer from root -> $ROOTLESS_USER (go env handled separately)
ROOTLESS_ENV_RE='(CIRRUS_.+)|(ROOTLESS_.+)|(.+_IMAGE.*)|(.+_BASE)|(.*DIRPATH)|(.*FILEPATH)|(SOURCE.*)|(DEPEND.*)|(.+_DEPS_.+)|(OS_REL.*)|(.+_ENV_RE)|(TRAVIS)|(CI.+)|(REMOTE.*)'
# Unsafe env. vars for display
-SECRET_ENV_RE='(IRCID)|(ACCOUNT)|(GC[EP]..+)|(SSH)'
+SECRET_ENV_RE='(ACCOUNT)|(GC[EP]..+)|(SSH)'
SPECIALMODE="${SPECIALMODE:-none}"
RCLI="${RCLI:-false}"
@@ -111,22 +102,9 @@ then
else
ROOTLESS_USER="${ROOTLESS_USER:-$USER}"
fi
-
-# GCE image-name compatible string representation of distribution name
-OS_RELEASE_ID="$(source /etc/os-release; echo $ID)"
-# GCE image-name compatible string representation of distribution _major_ version
-OS_RELEASE_VER="$(source /etc/os-release; echo $VERSION_ID | cut -d '.' -f 1)"
-# Combined to ease soe usage
-OS_REL_VER="${OS_RELEASE_ID}-${OS_RELEASE_VER}"
# Type of filesystem used for cgroups
CG_FS_TYPE="$(stat -f -c %T /sys/fs/cgroup)"
-# When building images, the version of automation tooling to install
-INSTALL_AUTOMATION_VERSION=1.1.3
-
-# Installed into cache-images, supports overrides
-# by user-data in case of breakage or for debugging.
-CUSTOM_CLOUD_CONFIG_DEFAULTS="$GOSRC/$PACKER_BASE/cloud-init/$OS_RELEASE_ID/cloud.cfg.d"
# Pass in a list of one or more envariable names; exit non-zero with
# helpful error message if any value is empty
req_env_var() {
@@ -237,67 +215,6 @@ timeout_attempt_delay_command() {
fi
}
-ircmsg() {
- req_env_var CIRRUS_TASK_ID IRCID
- [[ -n "$*" ]] || die 9 "ircmsg() invoked without message text argument"
- # Sometimes setup_environment.sh didn't run
- SCRIPT="$(dirname $0)/podbot.py"
- NICK="podbot_$CIRRUS_TASK_ID"
- NICK="${NICK:0:15}" # Any longer will break things
- set +e
- $SCRIPT $NICK $@
- echo "Ignoring exit($?)"
- set -e
-}
-
-# This covers all possible human & CI workflow parallel & serial combinations
-# where at least one caller must definitively discover if within a commit range
-# there is at least one release tag not having any '-' characters (return 0)
-# or otherwise (return non-0).
-is_release() {
- unset RELVER
- local ret
- req_env_var CIRRUS_CHANGE_IN_REPO
- if [[ -n "$CIRRUS_TAG" ]]; then
- RELVER="$CIRRUS_TAG"
- elif [[ ! "$CIRRUS_BASE_SHA" =~ "unknown" ]]
- then
- # Normally not possible for this to be empty, except when unittesting.
- req_env_var CIRRUS_BASE_SHA
- local range="${CIRRUS_BASE_SHA}..${CIRRUS_CHANGE_IN_REPO}"
- if echo "${range}$CIRRUS_TAG" | grep -iq 'unknown'; then
- die 11 "is_release() unusable range ${range} or tag $CIRRUS_TAG"
- fi
-
- if type -P git &> /dev/null
- then
- git fetch --all --tags &> /dev/null|| \
- die 12 "is_release() failed to fetch tags"
- RELVER=$(git log --pretty='format:%d' $range | \
- grep '(tag:' | sed -r -e 's/\s+[(]tag:\s+(v[0-9].*)[)]/\1/' | \
- sort -uV | tail -1)
- ret=$?
- else
- warn -1 "Git command not found while checking for release"
- ret="-1"
- fi
- [[ "$ret" -eq "0" ]] || \
- die 13 "is_release() failed to parse tags"
- else # Not testing a PR, but neither CIRRUS_BASE_SHA or CIRRUS_TAG are set
- return 1
- fi
- if [[ -n "$RELVER" ]]; then
- echo "Found \$RELVER $RELVER"
- if echo "$RELVER" | grep -q '-'; then
- return 2 # development tag
- else
- return 0
- fi
- else
- return 1 # not a release
- fi
-}
-
setup_rootless() {
req_env_var ROOTLESS_USER GOPATH GOSRC SECRET_ENV_RE ROOTLESS_ENV_RE
@@ -369,20 +286,6 @@ setup_rootless() {
die 11 "Timeout exceeded waiting for localhost ssh capability"
}
-# Grab a newer version of git from software collections
-# https://www.softwarecollections.org/en/
-# and use it with a wrapper
-install_scl_git() {
- echo "Installing SoftwareCollections updated 'git' version."
- ooe.sh $SUDO yum -y install rh-git29
- cat << "EOF" | $SUDO tee /usr/bin/git
-#!/usr/bin/env bash
-
-scl enable rh-git29 -- git $@
-EOF
- $SUDO chmod 755 /usr/bin/git
-}
-
install_test_configs() {
echo "Installing cni config, policy and registry config"
req_env_var GOSRC SCRIPT_BASE
@@ -457,66 +360,3 @@ $FEDORA_BASE_IMAGE
$PRIOR_FEDORA_BASE_IMAGE
"
}
-
-systemd_banish() {
- $GOSRC/$PACKER_BASE/systemd_banish.sh
-}
-
-# This can be removed when the kernel bug fix is included in Fedora
-workaround_bfq_bug() {
- if [[ "$OS_RELEASE_ID" == "fedora" ]] && [[ $OS_RELEASE_VER -le 32 ]]; then
- warn "Switching io scheduler to 'deadline' to avoid RHBZ 1767539"
- warn "aka https://bugzilla.kernel.org/show_bug.cgi?id=205447"
- echo "mq-deadline" | sudo tee /sys/block/sda/queue/scheduler > /dev/null
- echo -n "IO Scheduler set to: "
- $SUDO cat /sys/block/sda/queue/scheduler
- fi
-}
-
-# Warning: DO NOT USE.
-# This is called by other functions as the very last step during the VM Image build
-# process. It's purpose is to "reset" the image, so all the first-boot operations
-# happen at test runtime (like generating new ssh host keys, resizing partitions, etc.)
-_finalize() {
- set +e # Don't fail at the very end
- if [[ -d "$CUSTOM_CLOUD_CONFIG_DEFAULTS" ]]
- then
- echo "Installing custom cloud-init defaults"
- $SUDO cp -v "$CUSTOM_CLOUD_CONFIG_DEFAULTS"/* /etc/cloud/cloud.cfg.d/
- else
- echo "Could not find any files in $CUSTOM_CLOUD_CONFIG_DEFAULTS"
- fi
- echo "Re-initializing so next boot does 'first-boot' setup again."
- cd /
- $SUDO rm -rf $GOPATH/src # Actual source will be cloned at runtime
- $SUDO rm -rf /var/lib/cloud/instanc*
- $SUDO rm -rf /root/.ssh/*
- $SUDO rm -rf /etc/ssh/*key*
- $SUDO rm -rf /etc/ssh/moduli
- $SUDO rm -rf /home/*
- $SUDO rm -rf /tmp/*
- $SUDO rm -rf /tmp/.??*
- $SUDO sync
- $SUDO fstrim -av
-}
-
-# Called during VM Image setup, not intended for general use.
-rh_finalize() {
- set +e # Don't fail at the very end
- echo "Resetting to fresh-state for usage as cloud-image."
- PKG=$(type -P dnf || type -P yum || echo "")
- $SUDO $PKG clean all
- $SUDO rm -rf /var/cache/{yum,dnf}
- $SUDO rm -f /etc/udev/rules.d/*-persistent-*.rules
- $SUDO touch /.unconfigured # force firstboot to run
- _finalize
-}
-
-# Called during VM Image setup, not intended for general use.
-ubuntu_finalize() {
- set +e # Don't fail at the very end
- echo "Resetting to fresh-state for usage as cloud-image."
- $LILTO $SUDOAPTGET autoremove
- $SUDO rm -rf /var/cache/apt
- _finalize
-}
diff --git a/contrib/cirrus/lib.sh.t b/contrib/cirrus/lib.sh.t
index 204af1245..643b5513d 100755
--- a/contrib/cirrus/lib.sh.t
+++ b/contrib/cirrus/lib.sh.t
@@ -84,7 +84,7 @@ BAR=1
test_rev "FOO BAR" 0 ''
###############################################################################
-# tests for test_okay()
+# tests for item_test()
function test_item_test {
local exp_msg=$1
@@ -118,46 +118,4 @@ test_item_test "ok okay enough" 0 "okay enough" "line 1
line2" "=" "line 1
line2"
-###############################################################################
-# tests for is_release()
-
-# N/B: Assuming tests run in their own process, so wiping out the local
-# CIRRUS_BASE_SHA CIRRUS_CHANGE_IN_REPO and CIRRUS_TAG will be okay.
-function test_is_release() {
- CIRRUS_BASE_SHA="$1"
- CIRRUS_CHANGE_IN_REPO="$2"
- CIRRUS_TAG="$3"
- local exp_status=$4
- local exp_msg=$5
- local msg
- msg=$(is_release)
- local status=$?
-
- check_result "$msg" "$exp_msg" "is_release(CIRRUS_BASE_SHA='$1' CIRRUS_CHANGE_IN_REPO='$2' CIRRUS_TAG='$3')"
- check_result "$status" "$exp_status" "is_release(...) returned $status"
-}
-
-# FROM TO TAG RET MSG
-test_is_release "" "" "" "9" "FATAL: is_release() requires \$CIRRUS_CHANGE_IN_REPO to be non-empty"
-test_is_release "x" "" "" "9" "FATAL: is_release() requires \$CIRRUS_CHANGE_IN_REPO to be non-empty"
-
-# post-merge / tag-push testing, FROM will be set 'unknown' by (lib.sh default)
-test_is_release "unknown" "x" "" "1" ""
-# post-merge / tag-push testing, oddball tag is set, FROM will be set 'unknown'
-test_is_release "unknown" "unknown" "test-tag" "2" "Found \$RELVER test-tag"
-# post-merge / tag-push testing, sane tag is set, FROM will be set 'unknown'
-test_is_release "unknown" "unknown" "0.0.0" "0" "Found \$RELVER 0.0.0"
-# hack/get_ci_vm or PR testing, FROM and TO are set, no tag is set
-test_is_release "x" "x" "" "1" ""
-
-# Negative-testing git with this function is very difficult, assume git works
-# test_is_release ... "is_release() failed to fetch tags"
-# test_is_release ... "is_release() failed to parse tags"
-
-BF_V1=$(git rev-parse v1.0.0^)
-AT_V1=$(git rev-parse v1.0.0)
-test_is_release "$BF_V1" "$BF_V1" "v9.8.7-dev" "2" "Found \$RELVER v9.8.7-dev"
-test_is_release "$BF_V1" "$AT_V1" "v9.8.7-dev" "2" "Found \$RELVER v9.8.7-dev"
-test_is_release "$BF_V1" "$AT_V1" "" "0" "Found \$RELVER v1.0.0"
-
exit $rc
diff --git a/contrib/cirrus/notice_branch_failure.sh b/contrib/cirrus/notice_branch_failure.sh
deleted file mode 100755
index b810bd266..000000000
--- a/contrib/cirrus/notice_branch_failure.sh
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-
-source $(dirname $0)/lib.sh
-
-# mIRC "escape" codes are the most standard, for a non-standard client-side interpretation.
-ETX="$(echo -n -e '\x03')"
-RED="${ETX}4"
-NOR="$(echo -n -e '\x0f')"
-
-if [[ "$CIRRUS_BRANCH" = "$DEST_BRANCH" ]]
-then
- BURL="https://cirrus-ci.com/build/$CIRRUS_BUILD_ID"
- ircmsg "${RED}[Action Recommended]: ${NOR}Post-merge testing on ${RED}$CIRRUS_BRANCH failed${NOR} in $CIRRUS_TASK_NAME on ${OS_RELEASE_ID}-${OS_RELEASE_VER}: $BURL. Please investigate, and re-run if appropriate."
-fi
-
-# This script assumed to be executed on failure
-die 1 "Testing Failed"
diff --git a/contrib/cirrus/packer/.gitignore b/contrib/cirrus/packer/.gitignore
deleted file mode 100644
index 8f7bdeaf7..000000000
--- a/contrib/cirrus/packer/.gitignore
+++ /dev/null
@@ -1,7 +0,0 @@
-*json
-packer
-packer*zip
-packer_cache
-cidata*
-meta-data
-user-data
diff --git a/contrib/cirrus/packer/Makefile b/contrib/cirrus/packer/Makefile
deleted file mode 100644
index c5a8e4cac..000000000
--- a/contrib/cirrus/packer/Makefile
+++ /dev/null
@@ -1,94 +0,0 @@
-PACKER_VER ?= 1.4.3
-GOARCH=$(shell go env GOARCH)
-ARCH=$(uname -m)
-PACKER_DIST_FILENAME := packer_${PACKER_VER}_linux_${GOARCH}.zip
-
-# Only needed for libpod_base_images target
-TIMESTAMP := $(shell date +%s)
-GOPATH ?= /var/tmp/go
-GOSRC ?= $(GOPATH)/src/github.com/containers/libpod
-PACKER_BASE ?= contrib/cirrus/packer
-SCRIPT_BASE ?= contrib/cirrus
-POST_MERGE_BUCKET_SUFFIX ?=
-
-UBUNTU_BASE_IMAGE = $(shell source ../lib.sh && echo "$$UBUNTU_BASE_IMAGE")
-PRIOR_UBUNTU_BASE_IMAGE = $(shell source ../lib.sh && echo "$$PRIOR_UBUNTU_BASE_IMAGE")
-
-# For debugging nested-virt, use
-#TTYDEV := $(shell tty)
-TTYDEV := /dev/null
-
-.PHONY: all
-all: libpod_images
-
-# Utility target for checking required parameters
-.PHONY: guard-%
-guard-%:
- @if [[ -z "$($*)" ]]; then \
- echo "Missing or empty required make variable '$*'."; \
- exit 1; \
- fi;
-
-%.json: %.yml
- @python3 -c 'import json,yaml; json.dump( yaml.safe_load(open("$<").read()), open("$@","w"), indent=2);'
-
-${PACKER_DIST_FILENAME}:
- @curl -L --silent --show-error \
- -O https://releases.hashicorp.com/packer/${PACKER_VER}/${PACKER_DIST_FILENAME}
-
-packer: ${PACKER_DIST_FILENAME}
- @curl -L --silent --show-error \
- https://releases.hashicorp.com/packer/${PACKER_VER}/packer_${PACKER_VER}_SHA256SUMS \
- | grep linux_${GOARCH} > /tmp/packer_sha256sums
- @sha256sum --check /tmp/packer_sha256sums
- @unzip -o ${PACKER_DIST_FILENAME}
- @touch --reference=Makefile ${PACKER_DIST_FILENAME}
-
-.PHONY: test
-test: libpod_base_images.json libpod_images.json packer
- ./packer inspect libpod_base_images.json > /dev/null
- ./packer inspect libpod_images.json > /dev/null
- @echo "All good"
-
-.PHONY: libpod_images
-libpod_images: guard-PACKER_BUILDS libpod_images.json packer
- ./packer build \
- -force \
- $(shell test -z "${PACKER_BUILDS}" || echo "-only=${PACKER_BUILDS}") \
- -var GOPATH=$(GOPATH) \
- -var GOSRC=$(GOSRC) \
- -var PACKER_BASE=$(PACKER_BASE) \
- -var SCRIPT_BASE=$(SCRIPT_BASE) \
- libpod_images.json
-
-cidata.ssh:
- ssh-keygen -f $@ -P "" -q
-
-cidata.ssh.pub: cidata.ssh
- touch $@
-
-meta-data:
- echo "local-hostname: localhost.localdomain" > $@
-
-user-data: cidata.ssh.pub
- bash make-user-data.sh
-
-cidata.iso: user-data meta-data
- genisoimage -output cidata.iso -volid cidata -input-charset utf-8 -joliet -rock user-data meta-data
-
-# This is intended to be run by a human, with admin access to the libpod GCE project.
-.PHONY: libpod_base_images
-libpod_base_images: guard-GCP_PROJECT_ID guard-GOOGLE_APPLICATION_CREDENTIALS libpod_base_images.json cidata.iso cidata.ssh packer
- PACKER_CACHE_DIR=/tmp ./packer build \
- $(shell test -z "${PACKER_BUILDS}" || echo "-only=${PACKER_BUILDS}") \
- -force \
- -var TIMESTAMP=$(TIMESTAMP) \
- -var TTYDEV=$(TTYDEV) \
- -var GCP_PROJECT_ID=$(GCP_PROJECT_ID) \
- -var GOOGLE_APPLICATION_CREDENTIALS=$(GOOGLE_APPLICATION_CREDENTIALS) \
- -var GOSRC=$(GOSRC) \
- -var PACKER_BASE=$(PACKER_BASE) \
- -var SCRIPT_BASE=$(SCRIPT_BASE) \
- -var UBUNTU_BASE_IMAGE=$(UBUNTU_BASE_IMAGE) \
- -var PRIOR_UBUNTU_BASE_IMAGE=$(PRIOR_UBUNTU_BASE_IMAGE) \
- libpod_base_images.json
diff --git a/contrib/cirrus/packer/README.how-to-update-cirrus-vms b/contrib/cirrus/packer/README.how-to-update-cirrus-vms
deleted file mode 100644
index ac2902ffb..000000000
--- a/contrib/cirrus/packer/README.how-to-update-cirrus-vms
+++ /dev/null
@@ -1,89 +0,0 @@
-This document briefly describes how to update VMs on Cirrus.
-
-Examples of when you need to do this:
-
- - to update crun, conmon, or some other package(s)
- - to add and/or remove an OS (eg drop f31, add f33)
- - to change system config (eg containers.conf or other /etc files)
- - to change kernel command-line (boot time) options
-
-This is a TWO-STEP process: you need to submit a PR with a magic [CI:IMG]
-description string, wait for it to finish, grab a magic string from the
-results, then resubmit without [CI:IMG].
-
-Procedure, Part One of Two:
-
- 1) Create a working branch:
-
- $ git co -b my_branch_name
-
- 2) Make your changes. Typically, zero or more of the following files:
-
- .cirrus.yml
- contrib/cirrus/packer/*_packaging.sh
-
- I said zero because sometimes you just want to update VMs
- with the latest in dnf or ubuntu repos. That doesn't require
- changing anything here, simply running new dnf/apt installs.
-
- 3) Commit your changes. Be sure to include the magic [CI:IMG] string:
-
- $ git commit -asm'[CI:IMG] this is my commit message'
-
- 4) Submit your PR:
-
- $ gh pr create --fill --web
-
-
- -------------------------- INTERMISSION --------------------------
- ...in which we wait for CI to turn green. In particular, although
- we only really need 'test_build_cache_images' (45 minutes or so)
- to get the required magic number strings, please be a decent
- human being and wait for 'verify_test_built_images' (another hour)
- so we can all have confidence in our process. Thank you.
- -------------------------- INTERMISSION --------------------------
-
-
-Procedure, Part Two of Two:
-
- 1) When 'test_build_cache_images' completes, click it, then click
- 'View more details on Cirrus CI', then expand the 'Run build_vm_image'
- accordion. This gives you a garishly colorful display of lines.
- Each color is a different VM.
-
- 2) Verify that each VM has the packages you require. (The garish log
- doesn't actually list this for all packages, so you may need to
- look in the 'verify_test_built_images' log for each individual
- VM. Click the 'package_versions' accordion.)
-
- 3) At the bottom of this log you will see a block like:
-
- Builds finished. The artifacts of successful builds are:
- ubuntu-19: A disk image was created: ubuntu-19-podman-6439450735542272
- fedora-31: A disk image was created: fedora-31-podman-6439450735542272
- .....
-
- The long numbers at the end should (MUST!) be all identical.
-
- 4) Edit .cirrus.yml locally. Find '_BUILT_IMAGE_SUFFIX' near the
- top. Copy that long number ("6439450735542272", above) and paste
- it here, replacing the previous long number.
-
- 5) Wait for CI to turn green. I know you might have skipped that,
- because 'test_build_cache_images' finishes long before 'verify',
- and maybe you're in a hurry, but come on. Be responsible.
-
- 6) Edit the PR description in github: remove '[CI:IMG]' from the
- title. Again, *in github*, in the web UI, use the 'Edit' button
- at top right next to the PR title. Remove the '[CI:IMG]' string
- from the PR title, press Save. If you forget to do this, the
- VM-building steps will run again (taking a long time) but it
- will be a waste of time.
-
- 7) Update your PR:
-
- $ git add .cirrus.yml (to get the new magic IMAGE_SUFFIX string)
- $ git commit --amend (remove [CI:IMG] for consistency with 6)
- $ git push --force
-
-You can probably take it from here.
diff --git a/contrib/cirrus/packer/README.md b/contrib/cirrus/packer/README.md
deleted file mode 100644
index 9a07ed960..000000000
--- a/contrib/cirrus/packer/README.md
+++ /dev/null
@@ -1,3 +0,0 @@
-These are definitions and scripts consumed by packer to produce the
-various distribution images used for CI testing. For more details
-see the [Cirrus CI documentation](../README.md)
diff --git a/contrib/cirrus/packer/cloud-init/fedora/cloud-init.service b/contrib/cirrus/packer/cloud-init/fedora/cloud-init.service
deleted file mode 100644
index 4d2197d87..000000000
--- a/contrib/cirrus/packer/cloud-init/fedora/cloud-init.service
+++ /dev/null
@@ -1,20 +0,0 @@
-[Unit]
-Description=Initial cloud-init job (metadata service crawler)
-DefaultDependencies=no
-Wants=cloud-init-local.service
-After=cloud-init-local.service
-Wants=google-network-daemon.service
-After=google-network-daemon.service
-Before=systemd-user-sessions.service
-
-[Service]
-Type=oneshot
-ExecStart=/usr/bin/cloud-init init
-RemainAfterExit=yes
-TimeoutSec=0
-
-# Output needs to appear in instance console output
-StandardOutput=journal+console
-
-[Install]
-WantedBy=cloud-init.target
diff --git a/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/40_enable_root.cfg b/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/40_enable_root.cfg
deleted file mode 100644
index 672d1907b..000000000
--- a/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/40_enable_root.cfg
+++ /dev/null
@@ -1 +0,0 @@
-disable_root: 0
diff --git a/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/50_custom_disk_setup.cfg b/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/50_custom_disk_setup.cfg
deleted file mode 100644
index c0fdf0e23..000000000
--- a/contrib/cirrus/packer/cloud-init/fedora/cloud.cfg.d/50_custom_disk_setup.cfg
+++ /dev/null
@@ -1,4 +0,0 @@
-#cloud-config
-growpart:
- mode: false
-resize_rootfs: false
diff --git a/contrib/cirrus/packer/cloud-init/ubuntu/cloud.cfg.d/40_enable_root.cfg b/contrib/cirrus/packer/cloud-init/ubuntu/cloud.cfg.d/40_enable_root.cfg
deleted file mode 100644
index 672d1907b..000000000
--- a/contrib/cirrus/packer/cloud-init/ubuntu/cloud.cfg.d/40_enable_root.cfg
+++ /dev/null
@@ -1 +0,0 @@
-disable_root: 0
diff --git a/contrib/cirrus/packer/fedora_base-setup.sh b/contrib/cirrus/packer/fedora_base-setup.sh
deleted file mode 100644
index bf29a1aec..000000000
--- a/contrib/cirrus/packer/fedora_base-setup.sh
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/usr/bin/env bash
-
-# N/B: This script is not intended to be run by humans. It is used to configure the
-# fedora base image for importing, so that it will boot in GCE
-
-set -e
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-echo "Updating packages"
-dnf -y update
-
-echo "Installing necessary packages and google services"
-dnf -y install rng-tools google-compute-engine-tools google-compute-engine-oslogin ethtool
-
-echo "Enabling services"
-systemctl enable rngd
-
-# There is a race that can happen on boot between the GCE services configuring
-# the VM, and cloud-init trying to do similar activities. Use a customized
-# unit file to make sure cloud-init starts after the google-compute-* services.
-echo "Setting cloud-init service to start after google-network-daemon.service"
-cp -v $GOSRC/$PACKER_BASE/cloud-init/fedora/cloud-init.service /etc/systemd/system/
-
-# ref: https://cloud.google.com/compute/docs/startupscript
-# The mechanism used by Cirrus-CI to execute tasks on the system is through an
-# "agent" process launched as a GCP startup-script (from the metadata service).
-# This agent is responsible for cloning the repository and executing all task
-# scripts and other operations. Therefor, on SELinux-enforcing systems, the
-# service must be labeled properly to ensure it's child processes can
-# run with the proper contexts.
-METADATA_SERVICE_CTX=unconfined_u:unconfined_r:unconfined_t:s0
-METADATA_SERVICE_PATH=systemd/system/google-startup-scripts.service
-sed -r -e \
- "s/Type=oneshot/Type=oneshot\nSELinuxContext=$METADATA_SERVICE_CTX/" \
- /lib/$METADATA_SERVICE_PATH > /etc/$METADATA_SERVICE_PATH
-
-# Ensure there are no disruptive periodic services enabled by default in image
-systemd_banish
-
-rh_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/packer/fedora_packaging.sh b/contrib/cirrus/packer/fedora_packaging.sh
deleted file mode 100644
index fcf9eb93f..000000000
--- a/contrib/cirrus/packer/fedora_packaging.sh
+++ /dev/null
@@ -1,194 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called from fedora_setup.sh and various Dockerfiles.
-# It's not intended to be used outside of those contexts. It assumes the lib.sh
-# library has already been sourced, and that all "ground-up" package-related activity
-# needs to be done, including repository setup and initial update.
-
-set -e
-
-echo "Updating/Installing repos and packages for $OS_REL_VER"
-
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var GOSRC SCRIPT_BASE BIGTO INSTALL_AUTOMATION_VERSION FEDORA_BASE_IMAGE PRIOR_FEDORA_BASE_IMAGE
-
-# Pre-req. to install automation tooing
-$LILTO $SUDO dnf install -y git
-
-# Install common automation tooling (i.e. ooe.sh)
-curl --silent --show-error --location \
- --url "https://raw.githubusercontent.com/containers/automation/master/bin/install_automation.sh" | \
- $SUDO env INSTALL_PREFIX=/usr/share /bin/bash -s - "$INSTALL_AUTOMATION_VERSION"
-# Reload installed environment right now (happens automatically in a new process)
-source /usr/share/automation/environment
-
-# Set this to 1 to NOT enable updates-testing repository
-DISABLE_UPDATES_TESTING=${DISABLE_UPDATES_TESTING:0}
-
-# Do not enable updates-testing on the previous Fedora release
-if ((DISABLE_UPDATES_TESTING!=0)); then
- warn "Enabling updates-testing repository for image based on $FEDORA_BASE_IMAGE"
- $LILTO $SUDO ooe.sh dnf install -y 'dnf-command(config-manager)'
- $LILTO $SUDO ooe.sh dnf config-manager --set-enabled updates-testing
-else
- warn "NOT enabling updates-testing repository for image based on $PRIOR_FEDORA_BASE_IMAGE"
-fi
-
-$BIGTO ooe.sh $SUDO dnf update -y
-
-# Fedora, as of 31, uses cgroups v2 by default. runc does not support
-# cgroups v2, only crun does. (As of 2020-07-30 runc support is
-# forthcoming but not even close to ready yet). To ensure a reliable
-# runtime environment, force-remove runc if it is present.
-# However, because a few other repos. which use these images still need
-# it, ensure the runc package is cached in $PACKAGE_DOWNLOAD_DIR so
-# it may be swap it in when required.
-REMOVE_PACKAGES=(runc)
-
-INSTALL_PACKAGES=(\
- autoconf
- automake
- bash-completion
- bats
- bridge-utils
- btrfs-progs-devel
- buildah
- bzip2
- conmon
- container-selinux
- containernetworking-plugins
- containers-common
- criu
- crun
- curl
- device-mapper-devel
- dnsmasq
- e2fsprogs-devel
- emacs-nox
- file
- findutils
- fuse3
- fuse3-devel
- gcc
- git
- glib2-devel
- glibc-devel
- glibc-static
- gnupg
- go-md2man
- golang
- gpgme
- gpgme-devel
- grubby
- hostname
- httpd-tools
- iproute
- iptables
- jq
- krb5-workstation
- libassuan
- libassuan-devel
- libblkid-devel
- libcap-devel
- libffi-devel
- libgpg-error-devel
- libguestfs-tools
- libmsi1
- libnet
- libnet-devel
- libnl3-devel
- libseccomp
- libseccomp-devel
- libselinux-devel
- libtool
- libvarlink-util
- libxml2-devel
- libxslt-devel
- lsof
- make
- mlocate
- msitools
- nfs-utils
- nmap-ncat
- openssl
- openssl-devel
- ostree-devel
- pandoc
- pkgconfig
- podman
- policycoreutils
- procps-ng
- protobuf
- protobuf-c
- protobuf-c-devel
- protobuf-devel
- python2
- python3-PyYAML
- python3-dateutil
- python3-libselinux
- python3-libsemanage
- python3-libvirt
- python3-psutil
- python3-pytoml
- python3-requests
- redhat-rpm-config
- rpcbind
- rsync
- sed
- selinux-policy-devel
- skopeo
- skopeo-containers
- slirp4netns
- socat
- tar
- unzip
- vim
- wget
- which
- xz
- zip
- zlib-devel
-)
-DOWNLOAD_PACKAGES=(\
- "cri-o-$(get_kubernetes_version)*"
- cri-tools
- "kubernetes-$(get_kubernetes_version)*"
- runc
- oci-umount
- parallel
-)
-
-echo "Installing general build/test dependencies for Fedora '$OS_RELEASE_VER'"
-$BIGTO ooe.sh $SUDO dnf install -y ${INSTALL_PACKAGES[@]}
-
-# AD-HOC CODE FOR SPECIAL-CASE SITUATIONS!
-# On 2020-07-23 we needed this code to upgrade crun on f31, a build
-# that is not yet in stable. Since CI:IMG PRs are a two-step process,
-# the key part is that we UN-COMMENT-THIS-OUT during the first step,
-# then re-comment it on the second (once we have the built images).
-# That way this will be dead code in future CI:IMG PRs but will
-# serve as an example for anyone in a similar future situation.
-# $BIGTO ooe.sh $SUDO dnf --enablerepo=updates-testing -y upgrade crun
-
-[[ ${#REMOVE_PACKAGES[@]} -eq 0 ]] || \
- $LILTO ooe.sh $SUDO dnf erase -y "${REMOVE_PACKAGES[@]}"
-
-if [[ ${#DOWNLOAD_PACKAGES[@]} -gt 0 ]]; then
- echo "Downloading packages for optional installation at runtime, as needed."
- # Required for cri-o
- ooe.sh $SUDO dnf -y module enable cri-o:$(get_kubernetes_version)
- $SUDO mkdir -p "$PACKAGE_DOWNLOAD_DIR"
- cd "$PACKAGE_DOWNLOAD_DIR"
- $LILTO ooe.sh $SUDO dnf download -y --resolve "${DOWNLOAD_PACKAGES[@]}"
-fi
-
-echo "Installing runtime tooling"
-# Save some runtime by having these already available
-cd $GOSRC
-# Required since initially go was not installed
-source $GOSRC/$SCRIPT_BASE/lib.sh
-echo "Go environment has been setup:"
-go env
-$SUDO make install.tools
-$SUDO $GOSRC/hack/install_catatonit.sh
diff --git a/contrib/cirrus/packer/fedora_setup.sh b/contrib/cirrus/packer/fedora_setup.sh
deleted file mode 100644
index 16ae87d8a..000000000
--- a/contrib/cirrus/packer/fedora_setup.sh
+++ /dev/null
@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called by packer on the subject fedora VM, to setup the podman
-# build/test environment. It's not intended to be used outside of this context.
-
-set -e
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var SCRIPT_BASE PACKER_BASE INSTALL_AUTOMATION_VERSION PACKER_BUILDER_NAME GOSRC FEDORA_BASE_IMAGE OS_RELEASE_ID OS_RELEASE_VER
-
-workaround_bfq_bug
-
-# Do not enable updates-testing on the previous Fedora release
-if [[ "$PRIOR_FEDORA_BASE_IMAGE" =~ "${OS_RELEASE_ID}-cloud-base-${OS_RELEASE_VER}" ]]; then
- DISABLE_UPDATES_TESTING=1
-else
- DISABLE_UPDATES_TESTING=0
-fi
-
-bash $PACKER_BASE/fedora_packaging.sh
-# Load installed environment right now (happens automatically in a new process)
-source /usr/share/automation/environment
-
-echo "Enabling cgroup management from containers"
-ooe.sh sudo setsebool container_manage_cgroup true
-
-# Ensure there are no disruptive periodic services enabled by default in image
-systemd_banish
-
-rh_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/packer/image-builder-image_base-setup.sh b/contrib/cirrus/packer/image-builder-image_base-setup.sh
deleted file mode 100644
index 26fbe2903..000000000
--- a/contrib/cirrus/packer/image-builder-image_base-setup.sh
+++ /dev/null
@@ -1,71 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called by packer on a vanilla CentOS VM, to setup the image
-# used for building images FROM base images. It's not intended to be used
-# outside of this context.
-
-set -e
-
-[[ "$1" == "post" ]] || exit 0 # pre stage not needed
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var TIMESTAMP GOSRC SCRIPT_BASE PACKER_BASE
-
-install_ooe
-
-echo "Updating packages"
-ooe.sh sudo yum -y update
-
-echo "Configuring repositories"
-ooe.sh sudo yum -y install centos-release-scl epel-release
-
-echo "Installing packages"
-ooe.sh sudo yum -y install \
- genisoimage \
- golang \
- google-cloud-sdk \
- libvirt \
- libvirt-admin \
- libvirt-client \
- libvirt-daemon \
- make \
- python36 \
- python36-PyYAML \
- qemu-img \
- qemu-kvm \
- qemu-kvm-tools \
- qemu-user \
- rsync \
- rng-tools \
- unzip \
- util-linux \
- vim
-
-sudo systemctl enable rngd
-
-sudo ln -s /usr/libexec/qemu-kvm /usr/bin/
-
-sudo tee /etc/modprobe.d/kvm-nested.conf <<EOF
-options kvm-intel nested=1
-options kvm-intel enable_shadow_vmcs=1
-options kvm-intel enable_apicv=1
-options kvm-intel ept=1
-EOF
-
-echo "Installing packer"
-sudo mkdir -p /root/$(basename $PACKER_BASE)
-sudo cp $GOSRC/$PACKER_BASE/*packer* /root/$(basename $PACKER_BASE)
-sudo mkdir -p /root/$(basename $SCRIPT_BASE)
-sudo cp $GOSRC/$SCRIPT_BASE/*.sh /root/$(basename $SCRIPT_BASE)
-
-install_scl_git
-
-echo "Cleaning up"
-cd /
-rm -rf $GOSRC
-
-rh_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/packer/libpod_base_images.yml b/contrib/cirrus/packer/libpod_base_images.yml
deleted file mode 100644
index f53bfafc5..000000000
--- a/contrib/cirrus/packer/libpod_base_images.yml
+++ /dev/null
@@ -1,164 +0,0 @@
----
-
-variables:
- # Complete local path to this repository (Required)
- GOSRC:
- # Relative path to this (packer) subdirectory (Required)
- PACKER_BASE:
- # Relative path to cirrus scripts subdirectory (Required)
- SCRIPT_BASE:
- # Unique ID for naming new base-images (required)
- TIMESTAMP:
- # Required for output from qemu builders
- TTYDEV:
-
- # Ubuntu releases are merely copied to this project for control purposes
- UBUNTU_BASE_IMAGE:
- PRIOR_UBUNTU_BASE_IMAGE:
-
- # Latest Fedora release
- FEDORA_IMAGE_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.qcow2"
- FEDORA_CSUM_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-32-1.6-x86_64-CHECKSUM"
- FEDORA_BASE_IMAGE_NAME: 'fedora-cloud-base-32-1-6'
-
- # Prior Fedora release
- PRIOR_FEDORA_IMAGE_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/31/Cloud/x86_64/images/Fedora-Cloud-Base-31-1.9.x86_64.qcow2"
- PRIOR_FEDORA_CSUM_URL: "https://dl.fedoraproject.org/pub/fedora/linux/releases/31/Cloud/x86_64/images/Fedora-Cloud-31-1.9-x86_64-CHECKSUM"
- PRIOR_FEDORA_BASE_IMAGE_NAME: 'fedora-cloud-base-31-1-9'
-
- # The name of the image in GCE used for packer build libpod_images.yml
- IBI_BASE_NAME: 'image-builder-image'
- CIDATA_ISO: 'cidata.iso' # produced by Makefile
-
- # Path to json file (required, likely ~/.config/gcloud/legacy_credentials/*/adc.json)
- GOOGLE_APPLICATION_CREDENTIALS:
- # The complete project ID (required, not the short name)
- GCP_PROJECT_ID:
- # Pre-existing storage bucket w/ lifecycle-enabled
- XFERBUCKET: "packer-import" # pre-created, globally unique, lifecycle-enabled
-
-# Don't leak sensitive values in error messages / output
-sensitive-variables:
- - 'GOOGLE_APPLICATION_CREDENTIALS'
- - 'GCP_PROJECT_ID'
-
-# What images to produce in which cloud
-builders:
- - &nested_virt
- name: 'fedora'
- type: 'qemu'
- accelerator: "kvm"
- iso_url: '{{user `FEDORA_IMAGE_URL`}}'
- disk_image: true
- format: "raw"
- disk_size: 5120
- iso_checksum_url: '{{user `FEDORA_CSUM_URL`}}'
- iso_checksum_type: "sha256"
- output_directory: '/tmp/{{build_name}}'
- vm_name: "disk.raw" # actually qcow2, name required for post-processing
- boot_wait: '5s'
- shutdown_command: 'shutdown -h now'
- headless: true
- qemu_binary: "/usr/libexec/qemu-kvm"
- qemuargs: # List-of-list format required to override packer-generated args
- - - "-m"
- - "1024"
- - - "-cpu"
- - "host"
- - - "-device"
- - "virtio-rng-pci"
- - - "-chardev"
- - "tty,id=pts,path={{user `TTYDEV`}}"
- - - "-device"
- - "isa-serial,chardev=pts"
- - - "-cdrom"
- - "{{user `CIDATA_ISO`}}"
- - - "-netdev"
- - "user,id=net0,hostfwd=tcp::{{ .SSHHostPort }}-:22"
- - - "-device"
- - "virtio-net,netdev=net0"
- communicator: 'ssh'
- ssh_private_key_file: 'cidata.ssh'
- ssh_username: 'root'
-
- - <<: *nested_virt
- name: 'prior-fedora'
- iso_url: '{{user `PRIOR_FEDORA_IMAGE_URL`}}'
- iso_checksum_url: '{{user `PRIOR_FEDORA_CSUM_URL`}}'
-
- - &imgcopy
- name: 'ubuntu'
- type: 'googlecompute'
- image_name: '{{user `UBUNTU_BASE_IMAGE`}}'
- image_family: '{{build_name}}-base'
- source_image: '{{user `UBUNTU_BASE_IMAGE`}}'
- source_image_project_id: 'ubuntu-os-cloud'
- project_id: '{{user `GCP_PROJECT_ID`}}'
- account_file: '{{user `GOOGLE_APPLICATION_CREDENTIALS`}}'
- startup_script_file: "systemd_banish.sh"
- zone: 'us-central1-a'
- disk_size: 20
- communicator: 'none'
-
- - <<: *imgcopy
- name: 'prior-ubuntu'
- image_name: '{{user `PRIOR_UBUNTU_BASE_IMAGE`}}'
- source_image: '{{user `PRIOR_UBUNTU_BASE_IMAGE`}}'
-
-provisioners:
- - type: 'shell'
- only: ['fedora', 'prior-fedora']
- inline:
- - 'mkdir -p /tmp/libpod/{{user `SCRIPT_BASE`}}'
- - 'mkdir -p /tmp/libpod/{{user `PACKER_BASE`}}'
-
- - type: 'file'
- only: ['fedora', 'prior-fedora']
- source: '{{user `GOSRC`}}/.cirrus.yml'
- destination: '/tmp/libpod/.cirrus.yml'
-
- - type: 'file'
- only: ['fedora', 'prior-fedora']
- source: '{{user `GOSRC`}}/{{user `SCRIPT_BASE`}}/'
- destination: '/tmp/libpod/{{user `SCRIPT_BASE`}}/'
-
- - type: 'file'
- only: ['fedora', 'prior-fedora']
- source: '{{user `GOSRC`}}/{{user `PACKER_BASE`}}/'
- destination: '/tmp/libpod/{{user `PACKER_BASE`}}/'
-
- - &shell_script
- only: ['fedora', 'prior-fedora']
- type: 'shell'
- inline:
- - 'chmod +x /tmp/libpod/{{user `PACKER_BASE`}}/*.sh'
- - '/tmp/libpod/{{user `PACKER_BASE`}}/{{build_name}}_base-setup.sh'
- expect_disconnect: true # Allow this to reboot the VM if needed
- environment_vars:
- - 'TIMESTAMP={{user `TIMESTAMP`}}'
- - 'GOSRC=/tmp/libpod'
- - 'SCRIPT_BASE={{user `SCRIPT_BASE`}}'
- - 'PACKER_BASE={{user `PACKER_BASE`}}'
-
-post-processors:
- - - type: "compress"
- only: ['fedora', 'prior-fedora']
- output: '/tmp/{{build_name}}/disk.raw.tar.gz'
- format: '.tar.gz'
- compression_level: 9
- - &gcp_import
- only: ['fedora']
- type: "googlecompute-import"
- project_id: '{{user `GCP_PROJECT_ID`}}'
- account_file: '{{user `GOOGLE_APPLICATION_CREDENTIALS`}}'
- bucket: '{{user `XFERBUCKET`}}'
- gcs_object_name: '{{build_name}}-{{user `TIMESTAMP`}}.tar.gz'
- image_name: "{{user `FEDORA_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}"
- image_description: 'Based on {{user `FEDORA_IMAGE_URL`}}'
- image_family: '{{build_name}}-base'
- - <<: *gcp_import
- only: ['prior-fedora']
- image_name: "{{user `PRIOR_FEDORA_BASE_IMAGE_NAME`}}-{{user `TIMESTAMP`}}"
- image_description: 'Based on {{user `PRIOR_FEDORA_IMAGE_URL`}}'
- image_family: '{{build_name}}-base'
- - type: 'manifest'
diff --git a/contrib/cirrus/packer/libpod_images.yml b/contrib/cirrus/packer/libpod_images.yml
deleted file mode 100644
index 38f5a8250..000000000
--- a/contrib/cirrus/packer/libpod_images.yml
+++ /dev/null
@@ -1,86 +0,0 @@
----
-
-# All of these are required
-variables:
- BUILT_IMAGE_SUFFIX: '{{env `BUILT_IMAGE_SUFFIX`}}'
- GOPATH: '{{env `GOPATH`}}'
- GOSRC: '{{env `GOSRC`}}'
- PACKER_BASE: '{{env `PACKER_BASE`}}'
- SCRIPT_BASE: '{{env `SCRIPT_BASE`}}'
-
- # Base-image names are required. Using image family-names breaks parallelism
- UBUNTU_BASE_IMAGE: '{{env `UBUNTU_BASE_IMAGE`}}'
- PRIOR_UBUNTU_BASE_IMAGE: '{{env `PRIOR_UBUNTU_BASE_IMAGE`}}'
- FEDORA_BASE_IMAGE: '{{env `FEDORA_BASE_IMAGE`}}'
- PRIOR_FEDORA_BASE_IMAGE: '{{env `PRIOR_FEDORA_BASE_IMAGE`}}'
-
- # Protected credentials, decrypted by Cirrus at runtime
- GCE_SSH_USERNAME: '{{env `GCE_SSH_USERNAME`}}'
- GCP_PROJECT_ID: '{{env `GCP_PROJECT_ID`}}'
- SERVICE_ACCOUNT: '{{env `SERVICE_ACCOUNT`}}'
- GOOGLE_APPLICATION_CREDENTIALS: '{{env `GOOGLE_APPLICATION_CREDENTIALS`}}'
-
-# Don't leak sensitive values in error messages / output
-sensitive-variables:
- - 'GCE_SSH_USERNAME'
- - 'GCP_PROJECT_ID'
- - 'SERVICE_ACCOUNT'
-
-# What images to produce in which cloud
-builders:
- # v----- is a YAML anchor, allows referencing this object by name (below)
- - &gce_hosted_image
- name: 'ubuntu-20'
- type: 'googlecompute'
- image_name: '{{build_name}}{{user `BUILT_IMAGE_SUFFIX`}}'
- image_family: '{{build_name}}-cache'
- source_image: '{{user `UBUNTU_BASE_IMAGE`}}' # precedence over family
- source_image_family: 'ubuntu-base' # for ref. only
- disk_size: 20 # REQUIRED: Runtime allocation > this value
- project_id: '{{user `GCP_PROJECT_ID`}}'
- service_account_email: '{{user `SERVICE_ACCOUNT`}}'
- communicator: 'ssh'
- ssh_username: '{{user `GCE_SSH_USERNAME`}}'
- ssh_pty: 'true'
- # The only supported zone in Cirrus-CI, as of addition of this comment
- zone: 'us-central1-a'
-
- # v----- is a YAML alias, allows partial re-use of the anchor object
- - <<: *gce_hosted_image
- name: 'ubuntu-19'
- source_image: '{{user `PRIOR_UBUNTU_BASE_IMAGE`}}'
- source_image_family: 'prior-ubuntu-base'
-
- - <<: *gce_hosted_image
- name: 'fedora-32'
- source_image: '{{user `FEDORA_BASE_IMAGE`}}'
- source_image_family: 'fedora-base'
-
- - <<: *gce_hosted_image
- name: 'fedora-31'
- source_image: '{{user `PRIOR_FEDORA_BASE_IMAGE`}}'
- source_image_family: 'prior-fedora-base'
-
-# The brains of the operation, making actual modifications to the base-image.
-provisioners:
- - type: 'shell'
- inline:
- - 'set -ex'
- # The 'file' provisioner item (below) will create the final component
- - 'mkdir -vp $(dirname {{user `GOSRC`}})'
-
- - type: 'file'
- source: '{{user `GOSRC`}}'
- destination: '{{user `GOSRC`}}'
-
- - type: 'shell'
- script: '{{user `GOSRC`}}/{{user `PACKER_BASE`}}/{{split build_name "-" 0}}_setup.sh'
- environment_vars:
- - 'PACKER_BUILDER_NAME={{build_name}}'
- - 'GOPATH={{user `GOPATH`}}'
- - 'GOSRC={{user `GOSRC`}}'
- - 'PACKER_BASE={{user `PACKER_BASE`}}'
- - 'SCRIPT_BASE={{user `SCRIPT_BASE`}}'
-
-post-processors:
- - type: 'manifest' # writes packer-manifest.json
diff --git a/contrib/cirrus/packer/make-user-data.sh b/contrib/cirrus/packer/make-user-data.sh
deleted file mode 100644
index 676a50f5c..000000000
--- a/contrib/cirrus/packer/make-user-data.sh
+++ /dev/null
@@ -1,20 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is utilized by Makefile, it's not intended to be run by humans
-
-cat <<EOF > user-data
-#cloud-config
-timezone: US/Eastern
-growpart:
- mode: auto
-disable_root: false
-ssh_pwauth: True
-ssh_import_id: [root]
-ssh_authorized_keys:
- - $(cat cidata.ssh.pub)
-users:
- - name: root
- primary-group: root
- homedir: /root
- system: true
-EOF
diff --git a/contrib/cirrus/packer/prior-fedora_base-setup.sh b/contrib/cirrus/packer/prior-fedora_base-setup.sh
deleted file mode 100644
index bf29a1aec..000000000
--- a/contrib/cirrus/packer/prior-fedora_base-setup.sh
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/usr/bin/env bash
-
-# N/B: This script is not intended to be run by humans. It is used to configure the
-# fedora base image for importing, so that it will boot in GCE
-
-set -e
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-echo "Updating packages"
-dnf -y update
-
-echo "Installing necessary packages and google services"
-dnf -y install rng-tools google-compute-engine-tools google-compute-engine-oslogin ethtool
-
-echo "Enabling services"
-systemctl enable rngd
-
-# There is a race that can happen on boot between the GCE services configuring
-# the VM, and cloud-init trying to do similar activities. Use a customized
-# unit file to make sure cloud-init starts after the google-compute-* services.
-echo "Setting cloud-init service to start after google-network-daemon.service"
-cp -v $GOSRC/$PACKER_BASE/cloud-init/fedora/cloud-init.service /etc/systemd/system/
-
-# ref: https://cloud.google.com/compute/docs/startupscript
-# The mechanism used by Cirrus-CI to execute tasks on the system is through an
-# "agent" process launched as a GCP startup-script (from the metadata service).
-# This agent is responsible for cloning the repository and executing all task
-# scripts and other operations. Therefor, on SELinux-enforcing systems, the
-# service must be labeled properly to ensure it's child processes can
-# run with the proper contexts.
-METADATA_SERVICE_CTX=unconfined_u:unconfined_r:unconfined_t:s0
-METADATA_SERVICE_PATH=systemd/system/google-startup-scripts.service
-sed -r -e \
- "s/Type=oneshot/Type=oneshot\nSELinuxContext=$METADATA_SERVICE_CTX/" \
- /lib/$METADATA_SERVICE_PATH > /etc/$METADATA_SERVICE_PATH
-
-# Ensure there are no disruptive periodic services enabled by default in image
-systemd_banish
-
-rh_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/packer/systemd_banish.sh b/contrib/cirrus/packer/systemd_banish.sh
deleted file mode 100755
index 2219f2a4f..000000000
--- a/contrib/cirrus/packer/systemd_banish.sh
+++ /dev/null
@@ -1,28 +0,0 @@
-#!/usr/bin/env bash
-
-set +e # Not all of these exist on every platform
-
-# This is intended to be executed on VMs as a startup script on initial-boot.
-# Alternatively, it may be executed with the '--list' option to return the list
-# of systemd units defined for disablement (useful for testing).
-
-EVIL_UNITS="cron crond atd apt-daily-upgrade apt-daily fstrim motd-news systemd-tmpfiles-clean"
-
-if [[ "$1" == "--list" ]]
-then
- echo "$EVIL_UNITS"
- exit 0
-fi
-
-echo "Disabling periodic services that could destabilize testing:"
-for unit in $EVIL_UNITS
-do
- echo "Banishing $unit (ignoring errors)"
- (
- sudo systemctl stop $unit
- sudo systemctl disable $unit
- sudo systemctl disable $unit.timer
- sudo systemctl mask $unit
- sudo systemctl mask $unit.timer
- ) &> /dev/null
-done
diff --git a/contrib/cirrus/packer/ubuntu_packaging.sh b/contrib/cirrus/packer/ubuntu_packaging.sh
deleted file mode 100644
index c478028b5..000000000
--- a/contrib/cirrus/packer/ubuntu_packaging.sh
+++ /dev/null
@@ -1,175 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called from ubuntu_setup.sh and various Dockerfiles.
-# It's not intended to be used outside of those contexts. It assumes the lib.sh
-# library has already been sourced, and that all "ground-up" package-related activity
-# needs to be done, including repository setup and initial update.
-
-set -e
-
-echo "Updating/Installing repos and packages for $OS_REL_VER"
-
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var GOSRC SCRIPT_BASE BIGTO SUDOAPTGET INSTALL_AUTOMATION_VERSION
-
-echo "Updating/configuring package repositories."
-$BIGTO $SUDOAPTGET update
-
-echo "Installing deps to add third-party repositories and automation tooling"
-$LILTO $SUDOAPTGET install software-properties-common git curl
-
-# Install common automation tooling (i.e. ooe.sh)
-curl --silent --show-error --location \
- --url "https://raw.githubusercontent.com/containers/automation/master/bin/install_automation.sh" | \
- $SUDO env INSTALL_PREFIX=/usr/share /bin/bash -s - "$INSTALL_AUTOMATION_VERSION"
-# Reload installed environment right now (happens automatically in a new process)
-source /usr/share/automation/environment
-
-$LILTO ooe.sh $SUDOAPTADD ppa:criu/ppa
-
-echo "Configuring/Instaling deps from Open build server"
-VERSION_ID=$(source /etc/os-release; echo $VERSION_ID)
-echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_$VERSION_ID/ /" \
- | ooe.sh $SUDO tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
-ooe.sh curl -L -o /tmp/Release.key "https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_${VERSION_ID}/Release.key"
-ooe.sh $SUDO apt-key add - < /tmp/Release.key
-
-INSTALL_PACKAGES=(\
- apache2-utils
- apparmor
- aufs-tools
- autoconf
- automake
- bash-completion
- bats
- bison
- btrfs-progs
- build-essential
- buildah
- bzip2
- conmon
- containernetworking-plugins
- containers-common
- coreutils
- cri-o-runc
- criu
- curl
- dnsmasq
- e2fslibs-dev
- emacs-nox
- file
- fuse3
- gawk
- gcc
- gettext
- git
- go-md2man
- golang-1.14
- iproute2
- iptables
- jq
- libaio-dev
- libapparmor-dev
- libbtrfs-dev
- libcap-dev
- libdevmapper-dev
- libdevmapper1.02.1
- libfuse-dev
- libfuse2
- libfuse3-dev
- libglib2.0-dev
- libgpgme11-dev
- liblzma-dev
- libnet1
- libnet1-dev
- libnl-3-dev
- libprotobuf-c-dev
- libprotobuf-dev
- libseccomp-dev
- libseccomp2
- libselinux-dev
- libsystemd-dev
- libtool
- libudev-dev
- libvarlink
- lsof
- make
- netcat
- openssl
- pkg-config
- podman
- protobuf-c-compiler
- protobuf-compiler
- python-dateutil
- python-protobuf
- python2
- python3-dateutil
- python3-pip
- python3-psutil
- python3-pytoml
- python3-requests
- python3-setuptools
- rsync
- runc
- scons
- skopeo
- slirp4netns
- socat
- sudo
- unzip
- vim
- wget
- xz-utils
- zip
- zlib1g-dev
-)
-DOWNLOAD_PACKAGES=(\
- cri-o-$(get_kubernetes_version)
- cri-tools
- parallel
-)
-
-# These aren't resolvable on Ubuntu 20
-if [[ "$OS_RELEASE_VER" -le 19 ]]; then
- INSTALL_PACKAGES+=(\
- python-future
- python-minimal
- yum-utils
- )
-else
- INSTALL_PACKAGES+=(\
- python-is-python3
- )
-fi
-
-# Do this at the last possible moment to avoid dpkg lock conflicts
-echo "Upgrading all packages"
-$BIGTO ooe.sh $SUDOAPTGET upgrade
-
-echo "Installing general testing and system dependencies"
-# Necessary to update cache of newly added repos
-$LILTO ooe.sh $SUDOAPTGET update
-$BIGTO ooe.sh $SUDOAPTGET install "${INSTALL_PACKAGES[@]}"
-
-if [[ ${#DOWNLOAD_PACKAGES[@]} -gt 0 ]]; then
- echo "Downloading packages for optional installation at runtime, as needed."
- $SUDO ln -s /var/cache/apt/archives "$PACKAGE_DOWNLOAD_DIR"
- $LILTO ooe.sh $SUDOAPTGET install --download-only "${DOWNLOAD_PACKAGES[@]}"
-fi
-
-echo "Configuring Go environment"
-# There are multiple (otherwise conflicting) versions of golang available
-# on Ubuntu. Being primarily localized by env. vars and defaults, dropping
-# a symlink is the appropriate way to "install" a specific version system-wide.
-$SUDO ln -sf /usr/lib/go-1.14/bin/go /usr/bin/go
-# Initially go was not installed
-cd $GOSRC
-source $SCRIPT_BASE/lib.sh
-echo "Go environment has been setup:"
-go env
-
-echo "Building/Installing runtime tooling"
-$SUDO hack/install_catatonit.sh
-$SUDO make install.libseccomp.sudo
-$SUDO make install.tools GO_BUILD='go build' # -mod=vendor breaks this
diff --git a/contrib/cirrus/packer/ubuntu_setup.sh b/contrib/cirrus/packer/ubuntu_setup.sh
deleted file mode 100644
index d650e6c76..000000000
--- a/contrib/cirrus/packer/ubuntu_setup.sh
+++ /dev/null
@@ -1,35 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called by packer on the subject Ubuntu VM, to setup the podman
-# build/test environment. It's not intended to be used outside of this context.
-
-set -e
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var SCRIPT_BASE PACKER_BASE INSTALL_AUTOMATION_VERSION PACKER_BUILDER_NAME GOSRC UBUNTU_BASE_IMAGE OS_RELEASE_ID OS_RELEASE_VER
-
-# Ensure there are no disruptive periodic services enabled by default in image
-systemd_banish
-
-# Stop disruption upon boot ASAP after booting
-echo "Disabling all packaging activity on boot"
-for filename in $(sudo ls -1 /etc/apt/apt.conf.d); do \
- echo "Checking/Patching $filename"
- sudo sed -i -r -e "s/$PERIODIC_APT_RE/"'\10"\;/' "/etc/apt/apt.conf.d/$filename"; done
-
-bash $PACKER_BASE/ubuntu_packaging.sh
-
-# Load installed environment right now (happens automatically in a new process)
-source /usr/share/automation/environment
-
-echo "Making Ubuntu kernel to enable cgroup swap accounting as it is not the default."
-SEDCMD='s/^GRUB_CMDLINE_LINUX="(.*)"/GRUB_CMDLINE_LINUX="\1 cgroup_enable=memory swapaccount=1"/g'
-ooe.sh sudo sed -re "$SEDCMD" -i /etc/default/grub.d/*
-ooe.sh sudo sed -re "$SEDCMD" -i /etc/default/grub
-ooe.sh sudo update-grub
-
-ubuntu_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/packer/xfedora_setup.sh b/contrib/cirrus/packer/xfedora_setup.sh
deleted file mode 100644
index 16ae87d8a..000000000
--- a/contrib/cirrus/packer/xfedora_setup.sh
+++ /dev/null
@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-
-# This script is called by packer on the subject fedora VM, to setup the podman
-# build/test environment. It's not intended to be used outside of this context.
-
-set -e
-
-# Load in library (copied by packer, before this script was run)
-source $GOSRC/$SCRIPT_BASE/lib.sh
-
-req_env_var SCRIPT_BASE PACKER_BASE INSTALL_AUTOMATION_VERSION PACKER_BUILDER_NAME GOSRC FEDORA_BASE_IMAGE OS_RELEASE_ID OS_RELEASE_VER
-
-workaround_bfq_bug
-
-# Do not enable updates-testing on the previous Fedora release
-if [[ "$PRIOR_FEDORA_BASE_IMAGE" =~ "${OS_RELEASE_ID}-cloud-base-${OS_RELEASE_VER}" ]]; then
- DISABLE_UPDATES_TESTING=1
-else
- DISABLE_UPDATES_TESTING=0
-fi
-
-bash $PACKER_BASE/fedora_packaging.sh
-# Load installed environment right now (happens automatically in a new process)
-source /usr/share/automation/environment
-
-echo "Enabling cgroup management from containers"
-ooe.sh sudo setsebool container_manage_cgroup true
-
-# Ensure there are no disruptive periodic services enabled by default in image
-systemd_banish
-
-rh_finalize
-
-echo "SUCCESS!"
diff --git a/contrib/cirrus/podbot.py b/contrib/cirrus/podbot.py
deleted file mode 100755
index 9ca4915a7..000000000
--- a/contrib/cirrus/podbot.py
+++ /dev/null
@@ -1,105 +0,0 @@
-#!/usr/bin/env python3
-
-# Simple and dumb script to send a message to the #podman IRC channel on frenode
-# Based on example from: https://pythonspot.com/building-an-irc-bot/
-
-import os
-import time
-import random
-import errno
-import socket
-import sys
-
-class IRC:
-
- response_timeout = 30 # seconds
- irc = socket.socket()
-
- def __init__(self, server, nickname, channel):
- self.server = server
- self.nickname = nickname
- self.channel = channel
- self.irc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-
- def _send(self, cmdstr):
- self.irc.send(bytes(cmdstr + '\r\n', 'utf-8'))
-
- def message(self, msg):
- data = 'PRIVMSG {0} :{1}\r\n'.format(self.channel, msg)
- print(data)
- self._send(data)
-
- @staticmethod
- def fix_newlines(bufr):
- return bufr.replace('\\r\\n', '\n')
-
- def _required_response(self, needle, haystack):
- start = time.time()
- end = start + self.response_timeout
- while time.time() < end:
- if haystack.find(needle) != -1:
- return (False, haystack)
- time.sleep(0.1)
- try:
- haystack += str(self.irc.recv(4096, socket.MSG_DONTWAIT))
- except socket.error as serr:
- if serr.errno == errno.EWOULDBLOCK:
- continue
- raise # can't handle this
- return (True, haystack) # Error
-
- def connect(self, username, password):
- # This is ugly as sin, but seems to be a working send/expect sequence
-
- print("connecting to: {0}".format(self.server))
- self.irc.connect((self.server, 6667)) #connects to the server
- self._send("USER {0} {0} {0} :I am {0}".format(self.nickname))
- self._send("NICK {0}".format(self.nickname))
-
- err, haystack = self._required_response('End of /MOTD command.'
- ''.format(self.nickname), "")
- if err:
- print(self.fix_newlines(haystack))
- print("Error connecting to {0}".format(self.server))
- return True
-
- print("Logging in as {0}".format(username))
- self._send("PRIVMSG NickServ :IDENTIFY {0} {1}".format(username, password))
- err, _ = self._required_response("You are now identified for", "")
- if err:
- print("Error logging in to {0} as {1}".format(self.server, username))
- return True
-
- print("Joining {0}".format(self.channel))
- self._send("JOIN {0}".format(self.channel))
- err, haystack = self._required_response("{0} {1} :End of /NAMES list."
- "".format(self.nickname, self.channel),
- haystack)
- print(self.fix_newlines(haystack))
- if err:
- print("Error joining {0}".format(self.channel))
- return True
- return False
-
- def quit(self):
- print("Quitting")
- self._send("QUIT :my work is done here")
- self.irc.close()
-
-
-if len(sys.argv) < 3:
- print("Error: Must pass desired nick and message as parameters")
-else:
- for try_again in (True,False):
- irc = IRC("irc.freenode.net", sys.argv[1], "#podman")
- err = irc.connect(*os.environ.get('IRCID', 'Big Bug').split(" ", 2))
- if err and try_again:
- print("Trying again in 5 seconds...")
- time.sleep(5)
- continue
- elif err:
- break
- irc.message(" ".join(sys.argv[2:]))
- time.sleep(5.0) # avoid join/quit spam
- irc.quit()
- break
diff --git a/contrib/cirrus/setup_environment.sh b/contrib/cirrus/setup_environment.sh
index 855866477..b406d7b5c 100755
--- a/contrib/cirrus/setup_environment.sh
+++ b/contrib/cirrus/setup_environment.sh
@@ -33,9 +33,6 @@ done
# Sometimes environment setup needs to vary between distros
# Note: This should only be used for environment variables, and temporary workarounds.
-# Anything externally dependent, should be made fixed-in-time by adding to
-# contrib/cirrus/packer/*_setup.sh to be incorporated into VM cache-images
-# (see docs).
cd "${GOSRC}/"
case "${OS_RELEASE_ID}" in
ubuntu)
@@ -44,8 +41,6 @@ case "${OS_RELEASE_ID}" in
# All SELinux distros need this for systemd-in-a-container
setsebool container_manage_cgroup true
- workaround_bfq_bug
-
if [[ "$ADD_SECOND_PARTITION" == "true" ]]; then
bash "$SCRIPT_BASE/add_second_partition.sh"
fi
@@ -67,14 +62,14 @@ source "$SCRIPT_BASE/lib.sh"
case "$CG_FS_TYPE" in
tmpfs)
warn "Forcing testing with runc instead of crun"
- # On ubuntu, the default runc is usually not new enough.
- if [[ "${OS_RELEASE_ID}" == "ubuntu" ]]; then
- X=$(echo "export OCI_RUNTIME=/usr/lib/cri-o-runc/sbin/runc" | \
- tee -a /etc/environment) && eval "$X" && echo "$X"
- else
- X=$(echo "export OCI_RUNTIME=/usr/bin/runc" | \
- tee -a /etc/environment) && eval "$X" && echo "$X"
- fi
+ # On ubuntu, the default runc is usually not new enough.
+ if [[ "$OS_RELEASE_ID" == "ubuntu" ]]; then
+ X=$(echo "export OCI_RUNTIME=/usr/lib/cri-o-runc/sbin/runc" | \
+ tee -a /etc/environment) && eval "$X" && echo "$X"
+ else
+ X=$(echo "export OCI_RUNTIME=/usr/bin/runc" | \
+ tee -a /etc/environment) && eval "$X" && echo "$X"
+ fi
;;
cgroup2fs)
# This is necessary since we've built/installed from source, which uses runc as the default.
diff --git a/contrib/cirrus/success.sh b/contrib/cirrus/success.sh
deleted file mode 100755
index 8783f6b81..000000000
--- a/contrib/cirrus/success.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-
-source $(dirname $0)/lib.sh
-
-req_env_var CIRRUS_BRANCH CIRRUS_REPO_FULL_NAME CIRRUS_BASE_SHA CIRRUS_CHANGE_IN_REPO CIRRUS_CHANGE_MESSAGE
-
-cd $CIRRUS_WORKING_DIR
-
-if [[ "$CIRRUS_BRANCH" =~ "pull" ]]
-then
- echo "Retrieving latest HEADS and tags"
- git fetch --all --tags
- echo "Finding commit authors for PR $CIRRUS_PR"
- unset NICKS
- if [[ -r "$AUTHOR_NICKS_FILEPATH" ]]
- then
- SHARANGE="${CIRRUS_BASE_SHA}..${CIRRUS_CHANGE_IN_REPO}"
- EXCLUDE_RE='merge-robot'
- EMAILCSET='[:alnum:]-+_@.'
- AUTHOR_NICKS=$(egrep -v '(^[[:space:]]*$)|(^[[:space:]]*#)' "$AUTHOR_NICKS_FILEPATH" | sort -u)
- # Depending on branch-state, it's possible SHARANGE could be _WAY_ too big
- MAX_NICKS=10
- # newline separated
- GITLOG="git log --format='%ae'"
- COMMIT_AUTHORS=$($GITLOG $SHARANGE || $GITLOG -1 HEAD | \
- tr --delete --complement "$EMAILCSET[:space:]" | \
- egrep -v "$EXCLUDE_RE" | \
- sort -u | \
- tail -$MAX_NICKS)
-
- for c_email in $COMMIT_AUTHORS
- do
- c_email=$(echo "$c_email" | tr --delete --complement "$EMAILCSET")
- echo -e "\tExamining $c_email"
- NICK=$(echo "$AUTHOR_NICKS" | grep -m 1 "$c_email" | \
- awk --field-separator ',' '{print $2}' | tr -d '[[:blank:]]')
- if [[ -n "$NICK" ]]
- then
- echo -e "\t\tFound $c_email -> $NICK in $(basename $AUTHOR_NICKS_FILEPATH)"
- else
- echo -e "\t\tNot found in $(basename $AUTHOR_NICKS_FILEPATH), using e-mail username."
- NICK=$(echo "$c_email" | cut -d '@' -f 1)
- fi
- if ! echo "$NICKS" | grep -q "$NICK"
- then
- echo -e "\tUsing nick $NICK"
- NICKS="${NICKS:+$NICKS, }$NICK"
- else
- echo -e "\tNot re-adding duplicate nick $NICK"
- fi
- done
- fi
-
- unset MENTION_PREFIX
- [[ -z "$NICKS" ]] || \
- MENTION_PREFIX="$NICKS: "
-
- URL="https://github.com/$CIRRUS_REPO_FULL_NAME/pull/$CIRRUS_PR"
- PR_SUBJECT=$(echo "$CIRRUS_CHANGE_MESSAGE" | head -1)
- ircmsg "${MENTION_PREFIX}Cirrus-CI testing successful for PR '$PR_SUBJECT': $URL"
-else
- URL="https://cirrus-ci.com/github/containers/libpod/$CIRRUS_BRANCH"
- ircmsg "Cirrus-CI testing branch $(basename $CIRRUS_BRANCH) successful: $URL"
-fi