mesos slave #1

  • By jonnybgod
  • Latest version (#1)
  • trusty
  • Stable
  • Edge

Description

Apache Mesos abstracts CPU, memory, storage, and othercompute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

What is Mesos?

A distributed systems kernel

Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with API’s for resource management and scheduling across entire datacenter and cloud environments.

Mesos Features

  • Scalability to 10,000s of nodes
  • Fault-tolerant replicated master and slaves using ZooKeeper
  • Support for Docker containers
  • Native isolation between tasks with Linux Containers
  • Multi-resource scheduling (memory, CPU, disk, and ports)
  • Java, Python and C++ APIs for developing new parallel applications
  • Web UI for viewing cluster state

Overview

This charm install Mesos based on Mesosphere's packages and instructions. Refer to: - Mesosphere mesos cluster instalation instructions

Charm features:

  • Runs mesos slave
  • Option to install docker
  • Automatically configures mesos-dns if available from master

Usage

juju deploy mesos-slave
juju juju add-relation mesos-master mesos-slave

For full description of the options refer to:

This Charm is in beta and not all mesos-slave options are available at the moment. Let me know if you require any option or feel free to contribute.

Known Limitations and Issues

Local Provider Blockers

The Docker Charm will not work out of the box on the
local provider. LXC containers are goverend by a
very strict App Armor
policy that prevents accidental
misuses of privilege inside the container. Thus running the mesos-slave Charm with docker containerizer
inside the local provider is not a supported deployment method
.

TODO

  • Missing mesos slave options
  • Tests

Configuration

credential
(string) The credentials are the username and password used to access a secured Mesos master. A single line with the 'principal' and 'secret' separated by whitespace. For example: 'mesos rocks'
resource_monitoring_interval
(string) Periodic time interval for monitoring executor resource usage (e.g., 10secs, 1min, etc) (default: 1secs)
default_role
(string) Resources, for example, CPU, can be constrained by roles. The --resources flag allows control over resources (for example: cpu(prod):3, which reserves 3 CPU for the prod role). If a resource is detected but is **not** specified in the resources flag, then it will be assigned this default_role. The default value allows all roles to have access to this resource.
isolation
(string) There are a number of types of isolators for each type of resource which can be different from platform to platform. A linux platform has cgroups which can provide CPU and memory isolation. This flag always for the configuration of a set of isolations the slave will use. (default: posix/cpu,posix/mem).
hostname
(string) The hostname the slave should report. If left unset, system hostname will be used.
install_docker
(boolean) Should Docker be installed?
True
attributes
(string) 'rack:2;U:1'. This would be a way of indicating that this node is in rack 2 and is U 1. The attributes are arbitrary and can be thought of as ways of tagging a node. By default there are no attributes.
containerizers
(string) Comma separated list of containerizer implementations to compose in order to provide containerization. Available options are 'mesos', 'external', and 'docker' (on Linux). The order the containerizers are specified is the order they are tried (--containerizers=mesos). (default: mesos)
docker,mesos
executor_registration_timeout
(string) Amount of time to wait for an executor to register with the slave before considering it hung and shutting it down.
5mins
logging_level
(string) Log message at or above this level; possible values: 'INFO', 'WARNING', 'ERROR'. (default: INFO).
resources
(string) Total consumable resources per slave, in the form 'name(role):value;name(role):value...'. This value can be set to limit resources per role, or to overstate the number of resources that are available to the slave.