scalebot jenkins #9

  • By dannf
  • Latest version (#9)
  • xenial
  • Stable

Description

Automates jobs in a ScaleBot Lab.


What is ScaleBot?

ScaleBot is a framework for running scheduled jobs against a cloud. It is built upon the Jenkins CI and Juju modeling tools.

QuickStart (using Amazon Web Services)

Clone a local copy of the sample ScaleBot configuration repository:

$ git clone git://git.launchpad.net/~scalebot-team/+git/scalebot-config

Make a copy of the aws credentials template, and add your account keys:

$ cd scalebot-config
$ cp labs/aws/credentials.yaml.template labs/aws/credentials.yaml
$ vi labs/aws/credentials.yaml

Deploy the ScaleBot bundle. In this example, we'll deploy to local LXD containers:

$ juju bootstrap localhost
Creating Juju controller "localhost-localhost" on localhost/localhost
Looking for packaged Juju agent version 2.2.4 for amd64
To configure your system to better support LXD containers, please see: https://github.com/lxc/lxd/blob/master/doc/production-setup.md
Launching controller instance(s) on localhost/localhost...
 - juju-689661-0 (arch=amd64)
Fetching Juju GUI 2.9.2
Waiting for address
Attempting to connect to 10.98.200.35:22
Bootstrap agent now started
Contacting Juju controller at 10.98.200.35 to verify accessibility...
Bootstrap complete, "localhost-localhost" controller now available.
Controller machines are in the "controller" model.
Initial model "default" added.
$ juju deploy bundle.yaml
Deploying charm "cs:xenial/jenkins-4"
Deploying charm "cs:~ce-hyperscale/scalebot-jenkins-3"
Related "jenkins:extension" and "scalebot:extension"
Deploy of bundle completed.

Once deployment has completed, use juju status to find the IP of your scalebot jenkins server. You can connect to it at http://ip-of-server:8080. The sample configuration includes an example "Hello World" job. Building this job will deploy a node in the Lab cloud and run echo "Hello World" on it.

Architecture

ScaleBot is deployed from a Juju charm that builds a running instance of Jenkins CI, initialized with a git repository that provides job definitions and a target cloud to run those jobs against. We refer to this target cloud as a "lab". ScaleBot will use the target cloud ("lab") configuration to automatically setup an internal Juju controller which Jenkins jobs can use to deploy systems and workloads as necessary within the lab.

alt text

Of course, since scalebot-jenkins is a charm itself, it also is deployed in a cloud. This can just be a LXD cloud though, as the controller and Jenkins server can easily run in containers. The important thing is that the jenkins server in the ScaleBot cloud is on a network that can access the Lab cloud.

alt text

Configuration Repository

The scalebot-jenkins charm accepts configuration that tells ScaleBot where to fetch Jenkins job and Lab Cloud definitions. This data must be in a git repository with the following layout:

jobs/<jobfoo>.yaml
jobs/<jobbar>.yaml
...
jobs/<jobbaz>.yaml

labs/<labname>/<labname>.yaml
labs/<labname>/clouds.yaml
labs/<labname>/credentials.yaml
labs/<labname>/model-defaults.yaml

Jobs must be in the YAML format used by Jenkins Job Builder, as that is the tool used to load them into Jenkins. The clouds.yaml, credentials.yaml, and model-defaults.yaml files are passed directly to Juju when bootsrapping the Lab controller, so see Juju documentation for details on their contents.

You may also want to store code in this repository and call it from your jobs.

Job Environment

ScaleBot will automatically create an empty model prior to building a jenkins job. This model will have a unique name, based on the $BUILD_ID. Jobs can assume that this model exists and is in-focus prior to executing, so your build code can just begin deploying charms immediately on startup.

Since it is expected that builds will deploy systems using Juju, the python bindings for juju are pre-installed on the system.

Environment Variables

In addition to the environment variables set by Jenkins, the following variable is also available:

$SCALEBOT_REPO will be set to the path of the locally cloned copy of the Scalebot configuration repository.

Configuration

scalebot_juju_bootstrap_constraints
(string) Bootstrap constraints for Juju controller
scalebot_config_branch
(string) The branch of $scalebot_repo to checkout
scalebot_juju_model_defaults
(string) Juju model defaults configuration YAML
scalebot_config_repo
(string) URI to a Scalebot git repository (e.g. git+ssh://dannf@git.launchpad.net/hyperscale)
scalebot_juju_credentials
(string) Credentials YAML for Juju cloud defined in scalebot_juju_clouds
package_status
(string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
install
extra_packages
(string) Space separated list of extra deb packages to install.
scalebot_juju_clouds
(string) Juju cloud config YAML for target cloud
scalebot_private_ssh_key
(string) A null-passphrase SSH private key with access to $scalebot_repo
install_keys
(string) List of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). The keys should be the full ASCII armoured GPG public keys. While GPG key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for PPA sources where the package signing key is securely retrieved from Launchpad.
install_sources
(string) List of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). Each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding Personal Package Archives, or a distribution component to enable.