elasticsearch #4

  • By paulcz
  • Latest version (#4)
  • precise
  • Stable
  • Edge

Description

ElasticSearch is a distributed RESTful search engine built for the cloud. Features include:

  • Distributed and Highly Available Search Engine.
  • Each index is fully sharded with a configurable number of shards.
  • Each shard can have one or more replicas.
  • Read / Search operations performed on either one of the replica shard.
  • Multi Tenant with Multi Types.
  • Support for more than one index.
  • Support for more than one type per index.
  • Index level configuration (number of shards, index storage, …).
  • Various set of APIs
  • HTTP RESTful API
  • Native Java API.
  • All APIs perform automatic node operation rerouting.
  • Document oriented
  • No need for upfront schema definition.
  • Schema can be defined per type for customization of the indexing process.
  • Reliable, Asynchronous Write Behind for long term persistency.
  • (Near) Real Time Search.
  • Built on top of Lucene
  • Each shard is a fully functional Lucene index
  • All the power of Lucene easily exposed through simple configuration / plugins.
  • Per operation consistency
  • Single document level operations are atomic, consistent, isolated and durable.
  • Open Source under Apache 2 License.
  • Original charm by Luis Arias luis@balsamiq.com


To deploy elasticsearch locally you just need to:

juju bootstrap
juju deploy elasticsearch
you can add more units ...  LXC can deal with multicast
juju add-unit elasticsearch

To deploy on ec2 you need to specify a config file with your AWS access key and secret key which are used by
elasticsearch for discovery. See the elastisearch on ec2 tutorial for more information:

see http://www.elasticsearch.org/tutorials/2011/08/22/elasticsearch-on-ec2.html

For instance in an elasticsearch.yaml config file you would have:

elasticsearch:
  access-key: Your_AWS_Access_Key
  secret-key: Your_AWS_Secret_Key

You can also set a region config parameter in this file if the us-east region is not the one you want to use for
discovery. See the elasticache aws plugin documentation for more information:

https://github.com/elasticsearch/elasticsearch-cloud-aws

all that failing I have built an experimental config item 'zenmasters' which should enable unicast clustering.

Once the unit has started you can test if everything's working by using curl as below to do a health check
which should give you a similar json response.

$ curl -XGET 'http://<ec2 dns or ip of a node>:9200/_cluster/health?pretty=true'

{
  "cluster_name" : "es-demo",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 5,
  "active_shards" : 10,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

the download files for the current version are included in the package, but they can be downloaded by removing the files, or changing the config to point to a different version.

there seems to be something a bit buggy with the service wrapper on the first unit ... if it's not start/stopping properly, get in there and hard kill any java processes then start it again using the service wrapper. seems to work fine after doing that.

will relation join with logstash and kibana charms via the cluster and rest relations.

use http://ip.addr:9200/_plugin/head to see a good status view of the cluster.

see logstash-indexer charm's README.md file for usage examples.

Configuration

zenmasters
(string) comma seperated list of masters. if you run more than one ES nodes and your platform doesn't support multicast discovery, set this to at least the address of the first node. default 'false' tells it not to do anything. Once you have a master up you can use juju set elasticsearch zenmasters="10.0.3.180" ... but try and get multicast or ec2 discovery working first ;).
false
bootstrap-class
(string) Class used by elasticsearch service wrapper, should be org.elasticsearch.bootstrap.ElasticSearch for versions less than or equal to 0.18.7
org.elasticsearch.bootstrap.ElasticSearchF
ec2-secret-key
(string) Secret key for EC2.
checksum
(string) MD5 checksum of elasticsearch download.
1f3b8ea5468554d47a3862aba2c36176
ec2-region
(string) This sets the region that will be used for discovery on ec2
us-east-1
downloadurl
(string) URL to download elasticsearch from.
https://download.elasticsearch.org/elasticsearch/elasticsearch
version
(string) This sets the elasticsearch version number that will be used. If you're going to plug in logstash, make sure versions match.
0.90.5
ec2
(boolean) If set elasticsearch will use ec2 discovery for clustering
cluster-name
(string) This sets the elasticsearch cluster name, which is used to discover and auto-join other nodes. don not change this if you want elasticsearch and kibana to work.
elasticsearch
checksumwrapper
(string) MD5 checksum of elasticsearch service wrapper download.
450fcee5225cc4837e8b61eb64bcc2d2
ec2-access-key
(string) Access key for EC2.