druid config #59

Description

This charm is responsible for
providing common configuration options to a Druid Cluster. It is a lightweight
charm that provides configuration files to each node in your cluster.
For this reason, it is not necessary to scale out this charm as you scale out
your cluster. There should only be one unit of this charm per cluster, and all
Druid nodes (Druid Overlord, Druid Coordinator, Druid Middlemanager, Druid
Historical and Druid Broker) must be directly related to this charm.


Overview

Welcome to the Druid Config charm by Spicule. This charm is responsible for
providing common configuration options to a Druid Cluster. It is a lightweight
charm that provides configuration files to each node in your cluster.
For this reason, it is not necessary to scale out this charm as you scale out
your cluster. There should only be one unit of this charm per cluster, and all
Druid nodes (Druid Overlord, Druid Coordinator, Druid Middlemanager, Druid
Historical and Druid Broker) must be directly related to this charm.

Usage

To deploy this charm from the command line, enter the following:

juju deploy cs:~spiculecharms/druid-config --series xenial

That's it! This charm requires only the smallest instance of whatever Cloud
platform you are using, and by applying no hardware constraints, Juju will
automatically request the smallest instance available.

Once you have deployed your Druid Config charm, you must add a relation to each
of your Druid nodes. This is simple to do from the command line:

juju add-relation druid-config druid-<druid-node-name>

MySQL and Zookeeper

In addition to relating Config to your Druid nodes, you must also relate
it to a MySQL charm for metadata storage, as well as a Zookeeper charm
for cluster coordination. Deploy these charms and add relations to them,
and Config will automatically configure your cluster to point to these
servers.

Storage Options

Druid Config is responsible for configuring your Druid Cluster's storage
settings. How you set this up depends on your choice of storage solution.
Druid currently supports three main storage options:

  • Local storage
  • S3 Buckets
  • HDFS

Local storage is enabled by default on deployment. To utilise local storage,
simply spin up your Druid Config charm and relate it to your Druid nodes.

To configure your cluster to utilise an S3 bucket from the command line on
deployment, enter the following:

juju deploy cs:~spiculecharms/druid-config --series xenial --config s3-storage="True" --config s3-bucket="<s3-bucket>" --config s3-accessKey="<s3-accessKey>" --config s3-secretKey="<s3-secretKey>"

To configure your cluster to utilise HDFS, simply add a relation to the
Hadoop Plugin charm, which you can find in our handy Anssr Hadoop Bundle,
and Druid Config will handle the rest for you!

Druid Config is also responsive to changes in these configuration settings,
allowing the user to change their storage solution as needed. If you spin
up a cluster using local storage at first, you can modify the configuration
of Config to use S3 instead, and the Druid Nodes will re-configure and
restart automatically. Likewise, if you wish to change to HDFS, simply
spin up your Anssr Hadoop Bundle,
relate the Hadoop Plugin to Config and your cluster will be re-configured
to utilise your Hadoop cluster.

Known Limitations and Issues

Currently, the only known limitations and issues are some extreme use cases
which will not apply to most users. For example, Juju will not let users
remove the Hadoop-Config relation, so switching between HDFS clusters is
not currently possible.

If you find any further issues with this charm, please do let us know!
You can find our contact details below.

Contact Information

Thank you for using the Druid Config charm - we hope you find it useful! For
more information, here are our contact details:

Spicule: http://www.spicule.co.uk
Anssr: http://anssr.io
Email: info@spicule.co.uk

For assistance with Juju, please refer to the Juju Discourse.

Druid

For more information about Druid, please refer to the Druid website.

Configuration

s3-bucket
(string) S3 Bucket
s3-secretKey
(string) S3 Secret Key
s3-storage
(boolean) Enable S3 Storage
s3-accessKey
(string) S3 Access Key