elasticsearch #1

Description

Elasticsearch is a search server based on Lucene. It provides a
distributed, multitenant-capable full-text search engine with a
RESTful web interface and schema-free JSON documents. This package
contains the infrastructure needed to an ElasticSearch node.


Disclaimer

This ElasticSearch charm is for 99% based on the charm of James Beedy.
What we have changed is the extra_bindings settings and which IP the charm now
distribute over its relation. It only uses the machine's private IP. We noticed
problems while deploying the ELK stack on GCE. It uses the FAN IP addresses to
pass over the client relation, however the client (Kibana in this case) couldn't
reach ElasticSearch.

Overview

Elasticsearch is a flexible and powerful open source, distributed, real-time
search and analytics engine. Architected from the ground up for use in
distributed environments where reliability and scalability are must haves,
Elasticsearch gives you the ability to move easily beyond simple full-text
search. Through its robust set of APIs and query DSLs, plus clients for the
most popular programming languages, Elasticsearch delivers on the near
limitless promises of search technology.

Excerpt from elasticsearch.org

Usage

This charm can be used to deploy Elasticsearch in any way that is suppored by upstream Elasticsearch, and then some.

The charm configuration exposes an option called node-type, which is used by the charm to know what type of
Elasticsearch node to configure. In order to orchestrate this charm to facilitate more complex deployments,
you must configure the node-type on a per application basis.

The options for the node-type config can be explained as follows:

  • all - The node will assume all roles of Elasticsearch, there will be no difference in configuration from one node to the next.

  • master - The node will assume the 'master' node-type. Master nodes will wait for the number of peers to be >= the charm configuration
    option minn-master-count (this defaults to 1) before bootstrapping the cluster.

  • coordinator - The node will assume the 'coordinator' node-type. Coordinator nodes will wait until they have a relation to the master before
    joining the cluster.

  • data - The node will assume the 'data' node-type. Data nodes will wait until they have a relation to the master before
    joining the cluster.

  • ingest - The node will assume the 'ingest' node-type. Ingest nodes will wait until they have a relation to the master before
    joining the cluster.

Juju Storage

This charm supports Juju storage (as of Juju 2.3).

To deploy this charm using Juju storage (most common for data nodes)

juju deploy elasticsearch --storage data=ebs,10G

Following deployment, we can see the attached volume, and that is being used for Elasticsearch data:

$ df -h | grep elasticsearch
/dev/xvdf1      9.8G   23M  9.2G   1% /srv/elasticsearch-data

$ ls -la /srv/elasticsearch-data
total 28
drwxr-xr-x 4 elasticsearch elasticsearch  4096 Oct 30 21:53 .
drwxr-xr-x 3 root          root           4096 Oct 30 21:49 ..
drwx------ 2 elasticsearch elasticsearch 16384 Oct 30 21:49 lost+found
drwxr-xr-x 3 elasticsearch elasticsearch  4096 Oct 30 21:53 nodes

Basic (node-type='all')

Deploying this charm with the defaults will get you Elasticsearch installed from the elastic.co apt sources, and an all-in-one node-type, where
each unit is every node type.

For example:

juju deploy elasticsearch -n 3

The above command would deploy 3 elasticsearch nodes, of which all are master, data, ingest, and coordinator.
This functionality mirrors that of the legacy Juju Elasticsearch charm, you can deploy and scale a cluster without worrying about node-types
because every node is all node-types.

Desparate Node Types

The extended functionality of this charm lends itself to the configuration of Elasticsearch clusters with non-uniform node-types.

For example:

# config.yaml
es-master:
  node-type: "master"
es-data:
  node-type: "data"
es-coordinator:
  node-type: "coordinator"
es-ingest:
  node-type: "ingest"
# Deploy 3 units of each node-type
juju deploy elasticsearch es-master -n 3 --config config.yaml

juju deploy elasticsearch es-data -n 3 --config config.yaml

juju deploy elasticsearch es-ingest -n 3 --config config.yaml

juju deploy elasticsearch es-coordinator -n 3 --config config.yaml

# Make the relations between the components of the cluster
# and the master node

juju relate es-master:provide-master es-data:request-master

juju relate es-master:provide-master es-ingest:request-master

juju relate es-master:provide-master es-coordinator:request-master

Managed Configuration MGMT

This charm manages three files primarily, they are:
/etc/default/elasticsearch
/etc/elasticsearch/elasticsearch.yml
* /etc/elasticsearch/discovery-file/unicast_hosts.txt

If you wish to make a customization to any of the aforementioned files, please
do so through the charm configuration options.

Only file based discovery is currently supported (pull requests welcome!).

Copyright

  • AGPLv3 (see copyright file in this directory)

Contact Information

Elasticsearch

Configuration

node-type
(string) Type of elasticsearch node. Choices are 'data', 'master', 'ingest', 'coordinating', 'all'.
all
firewall_enabled
(boolean) this option will enable or disable the ufw. By default this is set to 'false' since the firewall is never enabled when a machine or container is created with juju. Set to 'true' to enable firewall. Once enabled rules will be added when another charm has a relation with elasticsearch. This charm will be able to access elasticsearch on port 9200, while all the other traffice to that port will be blocked. Enabling the firewall will also allow port 22 from anywhere.
custom-config
(string) String representation of custom elasticsearch.yml configs
package_status
(string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
install
extra_packages
(string) Space separated list of extra deb packages to install.
cluster-name
(string) This sets the elasticsearch cluster name. This name is then added to the elasticsearch.yml file
elasticsearch
min-master-count
(int) Minimum number of master nodes
1
install_keys
(string) list of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). the keys should be the full ascii armoured gpg public keys. while gpg key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for ppa sources where the package signing key is securely retrieved from launchpad.
- | -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.14 (GNU/Linux) mQENBFI3HsoBCADXDtbNJnxbPqB1vDNtCsqhe49vFYsZN9IOZsZXgp7aHjh6CJBD A+bGFOwyhbd7at35jQjWAw1O3cfYsKAmFy+Ar3LHCMkV3oZspJACTIgCrwnkic/9 CUliQe324qvObU2QRtP4Fl0zWcfb/S8UYzWXWIFuJqMvE9MaRY1bwUBvzoqavLGZ j3SF1SPO+TB5QrHkrQHBsmX+Jda6d4Ylt8/t6CvMwgQNlrlzIO9WT+YN6zS+sqHd 1YK/aY5qhoLNhp9G/HxhcSVCkLq8SStj1ZZ1S9juBPoXV1ZWNbxFNGwOh/NYGldD 2kmBf3YgCqeLzHahsAEpvAm8TBa7Q9W21C8vABEBAAG0RUVsYXN0aWNzZWFyY2gg KEVsYXN0aWNzZWFyY2ggU2lnbmluZyBLZXkpIDxkZXZfb3BzQGVsYXN0aWNzZWFy Y2gub3JnPokBOAQTAQIAIgUCUjceygIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgEC F4AACgkQ0n1mbNiOQrRzjAgAlTUQ1mgo3nK6BGXbj4XAJvuZDG0HILiUt+pPnz75 nsf0NWhqR4yGFlmpuctgCmTD+HzYtV9fp9qW/bwVuJCNtKXk3sdzYABY+Yl0Cez/ 7C2GuGCOlbn0luCNT9BxJnh4mC9h/cKI3y5jvZ7wavwe41teqG14V+EoFSn3NPKm TxcDTFrV7SmVPxCBcQze00cJhprKxkuZMPPVqpBS+JfDQtzUQD/LSFfhHj9eD+Xe 8d7sw+XvxB2aN4gnTlRzjL1nTRp0h2/IOGkqYfIG9rWmSLNlxhB2t+c0RsjdGM4/ eRlPWylFbVMc5pmDpItrkWSnzBfkmXL3vO2X3WvwmSFiQbkBDQRSNx7KAQgA5JUl zcMW5/cuyZR8alSacKqhSbvoSqqbzHKcUQZmlzNMKGTABFG1yRx9r+wa/fvqP6OT RzRDvVS/cycws8YX7Ddum7x8uI95b9ye1/Xy5noPEm8cD+hplnpU+PBQZJ5XJ2I+ 1l9Nixx47wPGXeClLqcdn0ayd+v+Rwf3/XUJrvccG2YZUiQ4jWZkoxsA07xx7Bj+ Lt8/FKG7sHRFvePFU0ZS6JFx9GJqjSBbHRRkam+4emW3uWgVfZxuwcUCn1ayNgRt KiFv9jQrg2TIWEvzYx9tywTCxc+FFMWAlbCzi+m4WD+QUWWfDQ009U/WM0ks0Kww EwSk/UDuToxGnKU2dQARAQABiQEfBBgBAgAJBQJSNx7KAhsMAAoJENJ9ZmzYjkK0 c3MIAIE9hAR20mqJWLcsxLtrRs6uNF1VrpB+4n/55QU7oxA1iVBO6IFu4qgsF12J TavnJ5MLaETlggXY+zDef9syTPXoQctpzcaNVDmedwo1SiL03uMoblOvWpMR/Y0j 6rm7IgrMWUDXDPvoPGjMl2q1iTeyHkMZEyUJ8SKsaHh4jV9wp9KmC8C+9CwMukL7 vM5w8cgvJoAwsp3Fn59AxWthN3XJYcnMfStkIuWgR7U2r+a210W6vnUxU4oN0PmM cursYPyeV0NX/KQeUeNMwGTFB6QHS/anRaGQewijkrYYoTNtfllxIu9XYmiBERQ/ qPDlGRlOgVTd9xUfHFkzB52c70E= =92oX -----END PGP PUBLIC KEY BLOCK-----
install_sources
(string) list of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding personal package archives, or a distribution component to enable.
- 'deb https://artifacts.elastic.co/packages/5.x/apt stable main'
java-opts
(string) Java Opts