Merge pull request #1385 from ypu/systemtest

Add system test with ginkgo

1 files changed, 55 insertions, 4 deletions

diff --git a/test/README.md b/test/README.md
index a068bb4f5..2a9a4d4b1 100644
--- a/test/README.md
+++ b/test/README.md
@@ -1,8 +1,33 @@
 ![PODMAN logo](../logo/podman-logo-source.svg)
-# Integration Tests
+# Test utils
+Test utils provide common functions and structs for testing. It includes two structs:
+* `PodmanTest`: Handle the *podman* command and other global resources like temporary
+directory. It provides basic methods, like checking podman image and pod status. Test
+suites should create their owner test *struct* as a composite of `PodmanTest`, and their
+owner PodmanMakeOptions().
+
+* `PodmanSession`: Store execution session data and related *methods*. Such like get command
+output and so on. It can be used directly in the test suite, only embed it to your owner
+session struct if you need expend it.
+
+## Unittest for test/utils
+To ensure neither *tests* nor *utils* break, There are unit-tests for each *functions* and
+*structs* in `test/utils`. When you adding functions or structs to this *package*, please
+update both unit-tests for it and this documentation.
+
+### Run unit test for test/utils
+Run unit test for test/utils.
+
+```
+make localunit
+```
+
+## Structure of the test utils and test suites
+The test *utils* package is at the same level of test suites. Each test suites also have their
+owner common functions and structs stored in `libpod_suite_test.go`.
 
-Our primary means of performing integration testing for libpod is with the
-[Ginkgo](https://github.com/onsi/ginkgo) BDD testing framework. This allows
+# Ginkgo test framework
+[Ginkgo](https://github.com/onsi/ginkgo) is a BDD testing framework. This allows
 us to use native Golang to perform our tests and there is a strong affiliation
 between Ginkgo and the Go test framework.
 
@@ -32,8 +57,16 @@ The gomega sources can be simply installed with the command:
 GOPATH=~/go go get github.com/onsi/gomega/...
 ```
 
-### Running the integration tests
+# Integration Tests
+Test suite for integration test for podman command line. It has its own structs:
+* `PodmanTestIntegration`: Integration test *struct* as a composite of `PodmanTest`. It
+set up the global options for *podman* command to ignore the environment influence from
+different test system.
+
+* `PodmanSessionIntegration`: This *struct* has it own *methods* for checking command
+output with given format JSON by using *structs* defined in inspect package.
 
+## Running the integration tests
 You can run the entire suite of integration tests with the following command:
 
 ```
@@ -67,3 +100,21 @@ make shell
 ```
 
 This will run a container and give you a shell and you can follow the instructions above.
+
+# System test
+System tests are used for testing the *podman* CLI in the context of a complete system. It
+requires that *podman*, all dependencies, and configurations are in place.  The intention of
+system testing is to match as closely as possible with real-world user/developer use-cases
+and environments. The orchestration of the environments and tests is left to external
+tooling.
+
+* `PodmanTestSystem`: System test *struct* as a composite of `PodmanTest`. It will not add any
+options to the command by default. When you run system test, you can set GLOBALOPTIONS,
+PODMAN_SUBCMD_OPTIONS or PODMAN_BINARY in ENV to run the test suite for different test matrices.
+
+## Run system test
+You can run the test with following command:
+
+```
+make localsystem
+```