python django #5

Description

This charm will install Django. It can also install your Django
project and his dependencies from either a template or from a
version control system.
It can also link your project to a database and sync the schemas.
This charm also come with a Fabric fabfile to interact with the
deployement in a cloud aware manner.

Python-django Charm

Authors:

What is Django?

Django is a high-level web application framework that loosely follows
the model-view-controller (MVC) design pattern. Python's equivalent to
Ruby on Rails, Django lets you build complex data-driven websites quickly
and easily - Django focuses on automating as much as possible and
adhering to the "Don't Repeat Yourself" (DRY) principle. Django
additionally emphasizes reusability and "pluggability" of components;
many generic third-party "applications" are available to enhance
projects or to simply reduce development time even further.

Notable features include:

  • An object-relational mapper (ORM)
  • Automatic admin interface
  • Elegant URL dispatcher
  • Form serialization and validation system
  • Templating system
  • Lightweight, standalone web server for development and testing
  • Internationalization support
  • Testing framework and client

The charm

This charm will install Django. It can also install your Django
project and its dependencies from either a template or from a
version control system.

It can also link your project to a database and sync the schemas.
This charm also comes with a Fabric fabfile to interact with the
deployment in a cloud-aware manner.

Quick start

Simply:

juju bootstrap
juju deploy python-django

juju deploy postgresql
juju add-relation python-django postgresql:db

juju deploy gunicorn
juju add-relation python-django gunicorn
juju expose python-django

In a couple of minutes, your new (vanilla) Django site should be ready at
the public address of gunicorn. You can find it in the output of the
juju status command.

This is roughtly equivalent to the Creating a project
step in Django's tutorial.

Example: Deploying using a site template

  1. Setup your Django specific parameters in mydjangosite.yaml like this one:

    mydjangosite:
        project_template_url: https://github.com/xenith/django-base-template/zipball/master
        project_template_extension: py,md,rst
    
  2. Deployment with Gunicorn:

    juju bootstrap
    juju deploy --config mydjangosite.yaml mydjangosite
    
    juju deploy postgresql
    juju add-relation mydjangosite postgresql:db
    
    juju deploy gunicorn
    juju add-relation mydjangosite gunicorn
    juju expose mydjangosite
    

Example: Deploying using a code repository

  1. Setup your Django specific parameters in mydjangosite.yaml like this one:

    mydjangosite:
        vcs: bzr
        repos_url: lp:~patrick-hetu/my_site
    
  2. Deployment with Gunicorn:

    juju bootstrap
    juju deploy --config mydjangosite.yaml python-django mydjangosite
    
    juju deploy postgresql
    juju add-relation mydjangosite postgresql:db
    
    juju deploy gunicorn
    juju add-relation mydjangosite gunicorn
    juju expose mydjangosite
    
  3. Your new Django site should be accessible at the public address of
    Gunicorn. To find it, look for it in the output of the juju status command.

Project layout and code injection

Continuing from the previous example, your web site should be on the Django node at:

/srv/mydjangosite/

As you can see there the charm has injected some code at the end of your settings.py
file (or created it if it was not there) to be able to import what's in the
juju_settings/ directory.

It's recommended that you make your vcs ignore database and secret files or
any files that have information that you don't want to publish.

Complex configuration example: dpaste

mydpastesite:
  django_version: ''
  django_south: True
  django_south_version: ''
  vcs: 'git'
  repos_url: 'https://github.com/bartTC/dpaste.git'
  repos_branch: '2.6'
  application_path: 'dpaste'
  django_settings: 'dpaste.settings'
  settings_injection_path: 'settings/__init__.py'
  urls_injection_path: 'urls/__init__.py'
  requirements_pip_files: 'requirements.txt'
  additional_distro_packages: "python-imaging,python-tz,python-dev,build-essential,libpq-dev,libmysqlclient-dev,libxml2-dev,libxslt1-dev"

Upgrade the charm

This charm allow you to upgrade your deployment using Juju's upgrade-charm
command. This command will:

  • upgrade Django
  • upgrade additional pip packages
  • upgrade additional Debian packages
  • upgrade using requirements files in your project

Management with Fabric

Fabric is a Python (2.5 or higher) library and command-line tool for
streamlining the use of SSH for application deployment or system administration tasks.

It provides a basic suite of operations for executing
local or remote shell commands (normally or via sudo) and uploading/downloading
files, as well as auxiliary functionality such as prompting the running user
for input, or aborting execution.

This charm includes a Fabric script that use Juju's information to perform various
tasks.

For a list of tasks type this command after bootstraping your Juju environment:

fab -l

For example, with a python-django service deployed you can run commands on all of its units:

fab -R python-django pull
[10.0.0.2] Executing task 'pull'
[10.0.0.2] run: bzr pull lp:~my_name/django_code/my_site
...
[10.0.0.2] run: invoke-rc.d gunicorn restart
...

Or you can also run commands on a single unit:

fab -R python-django/0 manage:createsuperuser
...
[10.0.0.2] out: Username (leave blank to use 'ubuntu'):

Limitation:

  • You can only execute tasks for one role at a time.
    But it can be a service or unit.

If you want to extend the fabfile check out fabtools.

Ansible

See: https://github.com/cmars/juju-ansible

Security

Note that if you're using a requirement.txt file the packages will
be downloaded with pip and it doesn't do any cryptographic
verification of its downloads.

Writing application charm

To create an application subordinate charm that can be related to this charm you need
to at least define an interface named directory-path in your metadate.yaml file
like this:

[...]
requires:
  python-django:
    interface: directory-path
    scope: container
    optional: true

When you add a relation between your charm and the python-django charm,
you will be able to get those relation variables from the hook:

  • settings_dir_path
  • urls_dir_path
  • django_admin_cmd
  • install_root

Now your charm will be informed about where it needs to add new settings
and urls files and how to run additionnal Django commands.
The Django charm reloads Gunicorn after the relation to catch the changes.

Changelog

X: Not yet released

  • Python3 compatibility
  • Support for Django 1.7
  • Support for virtualenv
  • Tests
  • More vcs options
  • More pip options

6: Notable changes:

  • The charm now use charm-helpers for hooks
  • The chams now use ansible for installation
  • Mysql and Redis and rabbitmq (celery) support was added
  • The charm can install South and do migrations
  • The charm include a Fabric and Ansible scripts to interact with Juju

Note that I still need to think about the role of Fabric, Ansible, Salt, charm-helpers, etc

Configuration changes:

  • python_path now accept multiple path separated by commas
  • requirements_pip_files now also accept urls
  • requirements_pip_files and requirements_apt_files are empty by default
  • settings_secret_key_path was renamed settings_secret_key_name
  • settings_database_path was renamed settings_database_name

New options:

  • unit_config
  • settings_amqp_name
  • celery_always_eager
  • django_south
  • django_south_version
  • django_debug
  • django_allowed_host
  • django_extra_settings
  • settings_injection_path
  • url_injection_path
  • pip_extra_args

Backwards incompatible changes:

  • Some default options changed so be sure to checkout your configuration files.
  • The use of Ansible change a bit how things are done now be sure to test.

3: Notable changes:

  • Rewrite the charm using python instead of BASH scripts
  • Django projects now need no modification to work with the charm
  • Use the django-admin startproject command with configurable arguments if no repos is specified
  • Juju's generated settings and urls files are now added in a juju_settings and a juju_urls directories by default
  • New MongoDB relation (server side is yet to be done)
  • New upgrade hook that upgrade pip and debian packages
  • Expose ports is now handle by the charm

Configuration changes:

  • default user and group is now ubuntu
  • new install_root option
  • new django_version option
  • new additional_pip_packages option
  • new repos_branch,repos_username,repos_password options
  • new project_name, project_template_extension, project_template_url options
  • new urls_dir_name and settings_dir_name options
  • new project_template_url and project_template_extension options
  • database, uploads, static, secret and cache settings locations are now configurable
  • extra_deb_pkg was renamed additional_distro_packages
  • requirements was renamed requirements_pip_files and now support multiple files
  • if python_path is empty set as install_root

Backwards incompatible changes:

  • swift support was moved to a subordinate charm
  • postgresql relation hook was rename pgsql instead of db

2: Notable changes:

  • You can configure all wsgi (Gunicorn) settings via the config.yaml file
  • Juju compatible Fabric fabfile.py is included for PAAS commands
  • Swift storage backend is now optional

Backwards incompatible changes:

  • Use split settings and urls
  • Permissons are now based on WSGI's user and group instead of just being www-data
  • media and static files are now in new directories ./uploads and ./static/
  • Deprecated configuration variables: site_domain, site_username, site_password, site_admin_email

1: Notable changes:

  • Initial release

Inspiration

Configuration

install_root
(string) The root directory to checkout to.
/srv/
wsgi_access_logfile
(string) The Access log file to write to.
repos_username
(string) The vcs user name. Note: *Subversion only* settings. For other vcs use the repos_url for auth.
wsgi_workers
(int) The number of worker process for handling requests. 0 for count(cpu) + 1
django_allowed_hosts
(string) A space separated list for settings.ALLOWED_HOSTS in django. Default value will be the hostname, fully-qualified name, and public IP.
wsgi_worker_class
(string) Socket protocol. Can be: http (default), uwsgi, fastcgi or workers type. Can be: sync (default), eventlet, gevent, tornado
django_extra_settings
(string) Allows setting up extra settings.* values for Django. Acceptable values are limited to comma delimited key-value pairs like: SETTING_X=foo, SETTING_Y=bar
requirements_apt_files
(string) Comma separated relative paths to requirement files. Note that the charm won't manually upgrade packages defined in this file. Leave the variable to an empty string if you don't want the feature.
additional_distro_packages
(string) Comma separated extra packages to install.
python-imaging,python-docutils,python-tz
urls_injection_path
(string) The place where the code injection path will be append. This is relative to the urls_dir_name path define earlier.
urls.py
wsgi_backlog
(int) The maximum number of pending connections.
2048
project_template_extension
(string) When Django copies the project template files, it also renders certain files through the template engine: the files whose extensions match the --extension option (py by default) and the files whose names are passed with the --name option.
port
(int) Port the application will be listenning.
8080
wsgi_user
(string) Switch worker processes to run as this user. User id (as an int) or the name.
www-data
wsgi_timeout
(int) Timeout of a request in seconds.
30
wsgi_extra
(string) Space separated extra settings. For example: --debug
celery_always_eager
(string) If True, all tasks will be executed locally by blocking until the task returns.
False
wsgi_keep_alive
(int) Keep alive time in seconds.
2
repos_branch
(string) The repo branch to pull out from. If empty, it will pull out the default branch or trunk (such as origin/master with git). Note that this setting only applies to git. This option is not supported for hg. For svn and bzr, specify the branch name as part of the URL.
django_south
(boolean) Enable the use of south migrations
django_debug
(boolean) Enable disable settings.DEBUG for django
project_template_url
(string) If not repository url is found, the charm will create a new project. This option is the --template argument value for the startproject command to use a custom project template. Django will also accept URLs (http, https, ftp) to compressed archives with the app template files, downloading and extracting them on the fly. For more informations see: https://docs.djangoproject.com/en/dev/ref/django-admin/#startproject-projectname-destination
settings_amqp_name
(string) The place where the amqp configuration will be appended or written.
20-amqp.py
settings_database_name
(string) The place where the database configuration will be appended or written. Set the variable to an empty string if you don't want the feature.
20-engine-%(engine_name)s.py
application_path
(string) The relative path to install_root where the manage.py script is located.
pip_extra_args
(string) Extra arguments passed to pip.
wsgi_log_file
(string) The log file to write to. If empty the logs would be handle by upstart.
wsgi_group
(string) Switch worker process to run as this group. A valid group id (as an int) or the name.
www-data
wsgi_log_level
(string) The granularity of Error log outputs.
info
settings_dir_name
(string) The name of the directory where generated settings will be written. Set the variable to an empty string if you don't want the feature.
juju_settings
repos_password
(string) The vcs password. Note: *Subversion only* settings. For other vcs use the repos_url for auth.
wsgi_umask
(string) A bit mask for the file mode on files written.
0002
wsgi_timestamp
(string) The variable to modify to trigger reloads.
django_south_version
(string) Version or origin from which to install. May be one of the following: distro (default), ppa:somecustom/ppa, a deb url sources entry or a valid pip line like 'South' or 'South==0.8.4' or a reposiroty url (without the -e). Leaving it empty if you don't want the charm to install South.
distro
settings_secret_key_name
(string) The place where the secret key configuration will be appended or written. Set the variable to an empty string if you don't want the feature.
10-secret.py
wsgi_wsgi_file
(string) The name of the WSGI application.
wsgi
python_path
(string) Set additionnals, colon separated, PYTHONPATH to the project.
django_settings
(string) The Python path to your Django settings module. Leave it empty if your settings file is at the root of your repos.
vcs
(string) The vcs software to use. Only hg, git, bzr, and svn are currently supported.
wsgi_access_logformat
(string) The Access log format. Don't forget to escape all quotes and round brackets.
urls_dir_name
(string) The name of the directory where generated url will be written. Set the variable to an empty string if you don't want the feature.
juju_urls
celery_amqp_vhost
(string) Set a specific vhost for amqp relation. By default it's the unit_name.
django_version
(string) Version or origin from which to install. May be one of the following: distro (default), ppa:somecustom/ppa, a deb url sources entry or a valid pip line like 'Django' or 'Django==1.5' or a reposiroty url (without the -e). Leaving it empty if you don't want the charm to install Django.
distro
requirements_pip_files
(string) Comma separated relative paths or urls to a requirement files. Note that the charm won't manually upgrade packages defined in this file. Leave the variable to an empty string if you don't want the feature.
unit-config
(string) base64 encoded string to hold configuration information for the unit. The contents will be written to a file named <install_root>/<unit>/unit_config where <unit> is the location the branch is extracted to.
site_secret_key
(string) The web site secret key. Leave empty will generate one. NOTE: You **NEED** to set this in a multi-units architecture or you will have some trouble.
listen_ip
(string) IP adresses to listen on. By default we listen on all of them.
0.0.0.0
settings_injection_path
(string) The place where the code injection path will be append. This is relative to the settings_dir_name path define earlier.
settings.py
wsgi_max_requests
(int) The maximum number of requests a worker will process before restarting.
repos_url
(string) The vcs url to checkout.
wsgi_worker_connections
(int) The maximum number of simultaneous clients.
128
additional_pip_packages
(string) Comma separated extra packages to install.