block storage broker #8

Block Storage Broker Charm

This charm intends to handle all credentials and interactions with cloud block
storage services, such as nova and EC2 clouds, so that other charms don't need
to have visibility to those credentials or commands necessary to create, attach
or detach cloud storage volumes. Volume consumers need only request volume size and provide their instance-id.
This charm runs as a standalone unit but could easily share resources with a
more computationally intensive service.

Block storage broker provides one interface.

- volume-request: The volume consumer, your charm or the storage 
  subordinate charm, will specify the instance-id and an optional size,
  volume-label or volume-id via relation-set calls.

  When creating a new volume, this charm's default_volume_size configuration
  setting will be used if no size is provided via the relation. A
  volume label of the format "<your_juju_unit_name> unit volume" will be
  used if no volume-label is provided via the relation data.
  When reattaching an existing volumes to an instance, the relation data
  volume-id will be used if set, and as a fallback option, any volume
  matching the relation volume-label will be attached to the instance.

  When the volume is attached, the block-storage-broker charm will publish
  block-device-path via the relation data to announce the
  device path that is available for mount on the instance-id.

Usage

This charm is intended for use with the storage subordinate charm
which will allow the principal units to request and automatically mount volumes.
The command set below will relate your charm to storage and storage to the
block-storage-broker and allocate an 11 Gig nova volume for each postgresql
unit.

$ cat >block-storage-bundle.cfg <<END
common:
    services:
        postgresql:
            charm: cs:precise/postgresql
            constraints: mem=2048
        storage:
            charm: cs:precise/storage
            options:
                volume_size: 11
                provider: block-storage-broker
                root: /srv/data
doit-openstack:
    inherits: common
    series: precise
    services:
        block-storage-broker:
            charm: cs:precise/block-storage-broker
            options:
                provider: nova
                key: <YOUR_OS_USERNAME>
                endpoint: <YOUR_OS_AUTH_URL>
                region: <YOUR_OS_REGION>
                secret: <YOUR_OS_PASSWORD>
                tenant: <YOUR_OS_TENANT_NAME>
    relations:
        - ["postgresql", "storage"]
        - ["storage", "block-storage-broker"]
doit-ec2:
    inherits: common
    series: precise
    services:
        block-storage-broker:
            charm: cs:precise/block-storage-broker
            options:
                provider: ec2
                key: <YOUR_EC2_ACCESS_KEY>
                endpoint: <YOUR_EC2_URL>
                secret: <YOUR_EC2_SECRET_KEY>
    relations:
        - ["postgresql", "storage"]
        - ["storage", "block-storage-broker"]
END

# To deploy and relate volumes using your openstack credentials
$ juju-deployer -c block-storage-bundle.cfg doit-openstack

# To deploy and relate volumes using your ec2 credentials
$ juju-deployer -c block-storage-bundle.cfg doit-ec2

Sample Data Relation

The Block Storage Broker charm is designed to make your life easier when
requesting volume attachments to your instances in Openstack and EC2 clouds.
As a charm author, implementing the data relation is very straightforward.

Sample Metadata:

requires:
    block-storage-broker:
        interface: volume-request

Sample Joined Hook:

#!/bin/bash

# Grab your instance id
METADATA_URL=http://169.254.169.254/openstack/2012-08-10/meta_data.json
INSTANCE_ID=`curl $METADATA_URL | jsonpipe | grep uuid | awk '{ print $2 }'`

relation-set size=11
relation-set instance-id=$INSTANCE_ID
relation-set volume-label="My volume label $JUJU_UNIT_NAME"
# If you are attempting to mount an existing known volume-id 123-123-123
relation-set volume-id=123-123-123

Sample Changed Hook:

#!/bin/bash
device_path=`relation-get block-device-path`
if [ -z "$device_path" ] ; then
    juju-log "wait for related service to start"
    exit 0
fi
service my_service stop
mount $device_path /somewhere
change_my_service_to_use_mount /somewhere
service my_service start

Known Issues

Since juju may not set target availability zones when deploying units per
feature bug lp:1183831, block-storage-broker charm will avoid trying to attach
volumes that exist in a different availability zone than the instance which
is requesting the volume. Instead of trying to copy volumes from other zones
into the existing instance's zone, block-storage-broker will create a new
volume and attach it to the instance. This way the admin can manually copy
needed files from other region volumes.

TODO

• Investigate Block Storage Broker discovery of instance-id if no instance-id provided
• Allow config option to automatically destroy a volume upon relation departed/broken