This test suite exercises a full Cloud Foundry deployment using the golang cf CLI and curl. It is restricted to testing user-facing features as a user interacting with the system via the CLI.

For example, one test pushes an app with cf push, hits an endpoint on the app with curl that causes it to crash, and asserts that we eventually see a crash event registered in gcf events.

Tests that will NOT be introduced here are ones which could be tested at the component level, such as basic CRUD of an object in the Cloud Controller. These tests belong with that component.

NOTE: Because we want to parallize execution, tests should be written in such a way as to be runable individually. This means that tests should not depend on state in other tests, and should not modify the CF state in such a way as to impact other tests.

Set up your golang development environment, per golang.org.

You will probably also need the following SCM programs in order to go get source code:

• git
• mercurial
• bazaar

See Go CLI for instructions on installing the go version of cf.

Make sure that curl is installed on your system.

Make sure that the go version of cf is accessible in your $PATH, and that it is either renamed to gcf, or that there is a symlink from gcf to the location of cf.

Check out a copy of cf-acceptance-tests and make sure that it is added to your $GOPATH. The recommended way to do this is to run go get github.com/cloudfoundry/cf-acceptance-tests. You will receive a warning "no buildable Go source files"; this can be ignored as there is no compilable go code in the package.

All go dependencies required by CATs are vendored in cf-acceptance-tests/Godeps. The test script itself, bin/test, ensures that the vendored dependencies are available when executing the tests by prepending this directory to $GOPATH.

To run the CF Acceptance tests, you will need:

• a running CF instance
• credentials for an Admin user
• an environment variable $CONFIG which points to a .json file that contains the application domain

The following script will configure these prerequisites for a bosh-lite installation. Replace credentials and URLs as appropriate for your environment.

#! /bin/bash

cat > integration_config.json <<EOF
{
  "api": "api.10.244.0.34.xip.io",
  "admin_user": "admin",
  "admin_password": "admin",
  "apps_domain": "10.244.0.34.xip.io"
}
EOF
export CONFIG=$PWD/integration_config.json

If you are running the tests with version newer than 6.0.2-0bba99f of the Go CLI against bosh-lite or any other environment using self-signed certificates, add

  "skip_ssl_validation": true

to your integration_config.json as well.

The tests in one_push_many_restarts_test.go operate on an app that is supposed to persist between runs of the CF Acceptance tests. If these tests are run, they will create an org, space, and quota and push the app to this space. The test config will provide default names for these entities, but to configure them, add the following key-value pairs to integration_config.json:

  "persistent_app_host": "myapp",
  "persistent_app_space": "myspace",
  "persistent_app_org": "myorg",
  "persistent_app_quota_name": "myquota",

To execute the tests, run:

./bin/test

Internally the bin/test script runs tests using ginkgo.

Arguments, such as -focus=, -nodes=, etc., that are passed to the script are sent to ginkgo

For example, to execute tests in parallel across four processes one would run:

./bin/test -nodes=4

Be careful with this number, as it's effectively "how many apps to push at once", as nearly every example pushes an app.

To see verbose output from cf, set CF_VERBOSE_OUTPUT to true before running the tests.

export CF_VERBOSE_OUTPUT=true

Set CF_TRACE_BASENAME and the trace output from cf will be captured in files named ${CF_TRACE_BASENAME}${Ginkgo Node Id}.txt.

export CF_TRACE_BASENAME=cf_trace_

The following files may be created:

cf_trace_1.txt
cf_trace_2.txt
...

If a test fails, look for the node id is the test output:

=== RUN TestLifecycle

Running Suite: Application Lifecycle
====================================
Random Seed: 1389376383
Parallel test node 2/10. Assigned 14 of 137 specs.

The cf trace output for the tests in these specs will be found in cf_trace_2.txt

CATs use godep to manage go dependencies.

All go packages required to run CATs are vendored into the cf-acceptance-tests/Godeps directory.

When making changes to the test suite that bring in additional go packages, you should use the workflow described in the Add or Update a Dependency section of the godep documentation.