diff options
Diffstat (limited to 'contrib/cirrus/README.md')
| -rw-r--r-- | contrib/cirrus/README.md | 182 |
1 files changed, 0 insertions, 182 deletions
diff --git a/contrib/cirrus/README.md b/contrib/cirrus/README.md index 977762293..f66560cc8 100644 --- a/contrib/cirrus/README.md +++ b/contrib/cirrus/README.md @@ -76,95 +76,6 @@ exercising cgroups v2 with Podman integration tests. Also depends on having `SPECIALMODE` set to 'cgroupv2` -### ``test_build_cache_images_task`` Task - -Modifying the contents of cache-images is tested by making changes to -one or more of the ``./contrib/cirrus/packer/*_setup.sh`` files. Then -in the PR description, add the magic string: ``[CI:IMG]`` - -***N/B: Steps below are performed by automation*** - -1. ``setup_environment.sh``: Same as for other tasks. - -2. ``build_vm_images.sh``: Utilize [the packer tool](http://packer.io/docs/) - to produce new VM images. Create a new VM from each base-image, connect - to them with ``ssh``, and perform the steps as defined by the - ``$PACKER_BASE/libpod_images.yml`` file: - - 1. On a base-image VM, as root, copy the current state of the repository - into ``/tmp/libpod``. - 2. Execute distribution-specific scripts to prepare the image for - use. For example, ``fedora_setup.sh``. - 3. If successful, shut down each VM and record the names, and dates - into a json manifest file. - 4. Move the manifest file, into a google storage bucket object. - This is a retained as a secondary method for tracking/auditing - creation of VM images, should it ever be needed. - -### ``verify_test_built_images`` Task - -Only runs following successful ``test_build_cache_images_task`` task. Uses -images following the standard naming format; ***however, only runs a limited -sub-set of automated tests***. Validating newly built images fully, requires -updating ``.cirrus.yml``. - -***N/B: Steps below are performed by automation*** - -1. Using the just build VM images, launch VMs and wait for them to boot. - -2. Execute the `setup_environment.sh` as in the `testing` task. - -2. Execute the `integration_test.sh` as in the `testing` task. - - -***Manual Steps:*** Assuming the automated steps pass, then -you'll find the new image names displayed at the end of the -`test_build_cache_images`. For example: - - -``` -...cut... - -[+0747s] ==> Builds finished. The artifacts of successful builds are: -[+0747s] --> ubuntu-18: A disk image was created: ubuntu-18-libpod-5664838702858240 -[+0747s] --> fedora-29: A disk image was created: fedora-29-libpod-5664838702858240 -[+0747s] --> fedora-30: A disk image was created: fedora-30-libpod-5664838702858240 -[+0747s] --> ubuntu-19: A disk image was created: ubuntu-19-libpod-5664838702858240 -``` - -Notice the suffix on all the image names comes from the env. var. set in -*.cirrus.yml*: `BUILT_IMAGE_SUFFIX: "-${CIRRUS_REPO_NAME}-${CIRRUS_BUILD_ID}"`. -Edit `.cirrus.yml`, in the top-level `env` section, update the suffix variable -used at runtime to launch VMs for testing: - - -```yaml -env: - ...cut... - #### - #### Cache-image names to test with (double-quotes around names are critical) - ### - _BUILT_IMAGE_SUFFIX: "libpod-5664838702858240" - FEDORA_CACHE_IMAGE_NAME: "fedora-30-${_BUILT_IMAGE_SUFFIX}" - PRIOR_FEDORA_CACHE_IMAGE_NAME: "fedora-29-${_BUILT_IMAGE_SUFFIX}" - ...cut... -``` - -***NOTES:*** -* If re-using the same PR with new images in `.cirrus.yml`, - take care to also *update the PR description* to remove - the magic ``[CI:IMG]`` string. Keeping it and - `--force` pushing would needlessly cause Cirrus-CI to build - and test images again. -* In the future, if you need to review the log from the build that produced - the referenced image: - - * Note the Build ID from the image name (for example `5664838702858240`). - * Go to that build in the Cirrus-CI WebUI, using the build ID in the URL. - (For example `https://cirrus-ci.com/build/5664838702858240`. - * Choose the *test_build_cache_images* task. - * Open the *build_vm_images* script section. - ### `docs` Task Builds swagger API documentation YAML and uploads to google storage (an online @@ -226,99 +137,6 @@ gsutil cors set /path/to/file.json gs://libpod-master-releases file. Therefore, if it is not functioning or misconfigured, a person must have altered it or changes were made to the referring site (e.g. `docs.podman.io`). -## Base-images - -Base-images are VM disk-images specially prepared for executing as GCE VMs. -In particular, they run services on startup similar in purpose/function -as the standard 'cloud-init' services. - -* The google services are required for full support of ssh-key management - and GCE OAuth capabilities. Google provides native images in GCE - with services pre-installed, for many platforms. For example, - RHEL, CentOS, and Ubuntu. - -* Google does ***not*** provide any images for Fedora (as of 5/2019), nor do - they provide a base-image prepared to run packer for creating other images - in the ``test_build_vm_images`` Task (above). - -* Base images do not need to be produced often, but doing so completely - manually would be time-consuming and error-prone. Therefore a special - semi-automatic *Makefile* target is provided to assist with producing - all the base-images: ``libpod_base_images`` - -To produce new base-images, including an `image-builder-image` (used by -the ``cache_images`` Task) some input parameters are required: - -* ``GCP_PROJECT_ID``: The complete GCP project ID string e.g. foobar-12345 - identifying where the images will be stored. - -* ``GOOGLE_APPLICATION_CREDENTIALS``: A *JSON* file containing - credentials for a GCE service account. This can be [a service - account](https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually) - or [end-user - credentials](https://cloud.google.com/docs/authentication/end-user#creating_your_client_credentials) - -* Optionally, CSV's may be specified to ``PACKER_BUILDS`` - to limit the base-images produced. For example, - ``PACKER_BUILDS=fedora,image-builder-image``. - -If there is no existing 'image-builder-image' within GCE, a new -one may be bootstrapped by creating a CentOS 7 VM with support for -nested-virtualization, and with elevated cloud privileges (to access -GCE, from within the GCE VM). For example: - -``` -$ alias pgcloud='sudo podman run -it --rm -e AS_ID=$UID - -e AS_USER=$USER -v $HOME:$HOME:z quay.io/cevich/gcloud_centos:latest' - -$ URL=https://www.googleapis.com/auth -$ SCOPES=$URL/userinfo.email,$URL/compute,$URL/devstorage.full_control - -# The --min-cpu-platform is critical for nested-virt. -$ pgcloud compute instances create $USER-image-builder \ - --image-family centos-7 \ - --boot-disk-size "200GB" \ - --min-cpu-platform "Intel Haswell" \ - --machine-type n1-standard-2 \ - --scopes $SCOPES -``` - -Then from that VM, execute the -``contrib/cirrus/packer/image-builder-image_base_setup.sh`` script. -Shutdown the VM, and convert it into a new image-builder-image. - -Building new base images is done by first creating a VM from an -image-builder-image and copying the credentials json file to it. - -``` -$ hack/get_ci_vm.sh image-builder-image-1541772081 -...in another terminal... -$ pgcloud compute scp /path/to/gac.json $USER-image-builder-image-1541772081:. -``` - -Then, on the VM, change to the ``packer`` sub-directory, and build the images: - -``` -$ cd libpod/contrib/cirrus/packer -$ make libpod_base_images GCP_PROJECT_ID=<VALUE> \ - GOOGLE_APPLICATION_CREDENTIALS=/path/to/gac.json \ - PACKER_BUILDS=<OPTIONAL> -``` - -Assuming this is successful (hence the semi-automatic part), packer will -produce a ``packer-manifest.json`` output file. This contains the base-image -names suitable for updating in ``.cirrus.yml``, `env` keys ``*_BASE_IMAGE``. - -On failure, it should be possible to determine the problem from the packer -output. Sometimes that means setting `PACKER_LOG=1` and troubleshooting -the nested virt calls. It's also possible to observe the (nested) qemu-kvm -console output. Simply set the ``TTYDEV`` parameter, for example: - -``` -$ make libpod_base_images ... TTYDEV=$(tty) - ... -``` - ## `$SPECIALMODE` Some tasks alter their behavior based on this value. A summary of supported |