buildbot master #1

  • By charmers
  • Latest version (#1)
  • precise
  • Stable
  • Edge

Description

The BuildBot is a system to automate the compile/test cycle required by
most software projects to validate code changes. By automatically
rebuilding and testing the tree each time something has changed, build
problems are pinpointed quickly, before other developers are
inconvenienced by the failure. The guilty developer can be identified
and harassed without human intervention. By running the builds on a
variety of platforms, developers who do not have the facilities to
test their changes everywhere before checkin will at least know
shortly afterwards whether they have broken the build or not. Warning
counts, lint checks, image size, compile time, and other build
parameters can be tracked over time, are more visible, and are
therefore easier to improve.


Usage

This charm depends on the buildbot-slave charm.

It expects all configuration to be provided initially at deploy time,
using the --config option.

You can provide a master.cfg file to use inline with your config file,
or you can provide a full branch that includes a master.cfg and any
other desired support files. If you provide a branch, be aware that
this charm expects to control the .tac file.

In both cases, you need to modify your master.cfg file to be used by
the charm.

  • The BuildmasterConfig's "slaves" key should usually be an empty
    list. This list will be populated by slaves dynamically as they
    join. You may be able to also define slaves that are hosted
    elsewhere (not in juju). This has not been tested and may need
    some adjustments to work.

  • The "builders" key should contain instances of
    buildbot.config.BuilderConfig, not dicts. The "slavenames"
    argument should be a list of a single empty string. The empty
    string will be removed and replaced with the names of slaves that
    dynamically declare that they want to participate in those builds.

Examples are below.

Demo of a single master.cfg file

This uses the standard buildbot Pyflakes example.

Start with the buildbot-slave and buildbot-master charm in your charm
repository.

juju bootstrap
juju deploy --repository=./charms local:buildbot-master --config=./charms/oneiric/buildbot-master/examples/pyflakes.yaml
juju deploy --repository=./charms local:buildbot-slave
juju set buildbot-slave builders=runtests
juju add-relation buildbot-slave buildbot-master

Look in the pyflakes.yaml file to see how the master.cfg was modified
as described above.

Demo of a master.cfg in a branch with supporting files

This is a pretty odd example in several ways, but you can also run a
master and slave that run Launchpad's tests, like so:

juju bootstrap
juju deploy --config=./charms/oneiric/buildbot-master/examples/lpbuildbot.yaml --repository=./charms local:buildbot-master
juju deploy --config=./charms/oneiric/buildbot-slave/examples/lpbuildbot.yaml --repository=./charms local:buildbot-slave
juju add-relation buildbot-slave buildbot-master

Note that starting up the buildbot-slave for Launchpad takes a very,
very long time (over two hours) and requires a large EC2 instance to
avoid out of memory errors.

Running the charm tests

1) Establish a charm repository if you do not already have one. A charm
repository is a directory with subdirectories for each Ubuntu version being
used. Inside those per-Ubuntu-version directories are the charm
directories. For example, to make a charm repository for this charm under
Oneiric follow these steps:

a) mkdir -p ~/juju-charms/oneiric
b) ln -s `pwd` ~/juju-charms/oneiric/buildbot-master

2) Copy the juju_wrapper file into some place earlier in your PATH than the
real juju executable, naming it "juju". Edit the CHARM_TEST_REPO variable
therein to reflect the location of the charm repo from step 1.

3) Bootstrap the juju environment if not already done:

juju bootstrap

4) Run the tests: RESOLVE_TEST_CHARMS=1 tests/buildbot-master.test

XXX bac 2012-02-23: The tests do not run locally using precise. Set
default-series: oneiric to get them to pass.

Running the charm helper tests

Just run "python hooks/tests.py".

Configuration

extra-packages
(string) A space-separated list of packages to be installed. The repositories to use in getting these packages should have been set prior to setting this value. Calling multiple times will install the newly specified packages while leaving the previous ones installed.
config-user
(string) The user for access to a restricted URL.
buildbot-pkg
(string) The package name, possibly with versioning information, to be installed for buildbot. Example values are 'buildbot', 'buildbot/lucid', or 'buildbot=0.7.9'.
buildbot
extra-repository
(string) The full line to be inserted into an apt sources.list for a repository. If called multiple times the new repository will be added but ones added previously will not be removed. An example would be: deb http://us.archive.ubuntu.com/ubuntu/ lucid main universe
bucket-name
(string) The bucket used to store buildbot history. If not provided a default based on the access-key will be used.
config-file
(string) A master.cfg file. Use of this configuration is mutually exclusive with the use of config-transport and config-url.
save-history-now
(string) Configuration hack to fire off the saving of the buildbot master history. Normally this would be done in the stop hook but due to Bug 872264 that hook is not firing properly. The value of the setting is not important but it must change between invocations or the event will not be recogized. Monotonically increasing integer values would be a good choice. Or a time string.
config-url
(string) The location the buildbot master configuration is to be fetched. It must be compatible with the `config-transport`.
access-key
(string) Access key for EC2.
config-private-key
(string) The private key for `config-user`.
secret-key
(string) Secret key for EC2.
config-transport
(string) The command transport for fetching the configuration directory from the `config-url`. Must be one of [bzr, rsync]. If adding a new supported protocol, ensure the program is installed in the `start` hook and that it is properly handled in the `config-changed` hook.
installdir
(string) The directory where the Buildbot master will be installed.
/var/lib/buildbot/masters/master