oozie #16

Description

Oozie is a workflow scheduler system to manage Apache Hadoop jobs.


Overview

Oozie is a workflow scheduler system to manage Apache Hadoop jobs.

Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.

Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time (frequency) and data availability.

Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and Distcp) as well as system specific jobs (such as Java programs and shell scripts).

Oozie is a scalable, reliable and extensible system.

This charm deploys version 4.3.0 from Apache Bigtop.

Deploying

This charm requires Juju 2.0 or greater. If Juju is not yet set up, please
follow the getting-started instructions prior to deploying this charm.

This charm is intended to be deployed via one of the apache bigtop bundles.
For example:

juju deploy hadoop-processing

This will deploy an Apache Bigtop Hadoop cluster. More information about this
deployment can be found in the bundle readme.

Now add Oozie and relate it to the cluster via the hadoop-plugin:

juju deploy ~spiculecharms/oozie
juju add-relation oozie plugin

Network-Restricted Environments

Charms can be deployed in environments with limited network access. To deploy
in this environment, configure a Juju model with appropriate proxy and/or
mirror options. See Configuring Models for more information.

Verifying

Status

Apache Bigtop charms provide extended status reporting to indicate when they
are ready:

juju status

This is particularly useful when combined with watch to track the on-going
progress of the deployment:

watch -n 2 juju status

The message column will provide information about a given unit's state.
This charm is ready for use once the status message indicates that it is
ready.

Using

Once the deployment has been verified, Apache Oozie will be available to execute
Oozie workflows with your data.

Issues

Apache Bigtop tracks issues using JIRA (Apache account required). File an
issue for this charm at:

https://issues.apache.org/jira/secure/CreateIssue!default.jspa

Ensure Bigtop is selected as the project. Typically, charm issues are filed
in the deployment component with the latest stable release selected as the
affected version. Any uncertain fields may be left blank.

Contact Information

Resources

Configuration

bigtop_version
(string) Apache Bigtop release version. The default, '1.2.1' will use the current GA release, Bigtop 1.2.1, for all hiera data, puppet recipes, and installable packages. Set this to 'master' to use the latest upstream bits.
1.2.1
cuda-version
(string) Version of the cuda-repo deb to install. Valid options can be found at: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64
9.1.85-1
install-cuda
(boolean) Install the CUDA binaries if capable hardware is present (True by default). Set to False to disable CUDA installation regardless of capable hardware.
True