1 #Elasticsearch Puppet module
3 [![Build Status](https://travis-ci.org/elasticsearch/puppet-elasticsearch.png?branch=master)](https://travis-ci.org/elasticsearch/puppet-elasticsearch)
7 1. [Overview](#overview)
8 2. [Module description - What the module does and why it is useful](#module-description)
9 3. [Setup - The basics of getting started with Elasticsearch](#setup)
10 * [The module manages the following](#the-module-manages-the-following)
11 * [Requirements](#requirements)
12 4. [Usage - Configuration options and additional functionality](#usage)
13 5. [Advanced features - Extra information on advanced usage](#advanced-features)
14 6. [Limitations - OS compatibility, etc.](#limitations)
15 7. [Development - Guide for contributing to the module](#development)
16 8. [Support - When you need help with this module](#support)
22 This module manages Elasticsearch (http://www.elasticsearch.org/overview/elasticsearch/)
26 The elasticsearch module sets up Elasticsearch instances and can manage plugins and templates.
28 This module has been tested against ES 1.0 and up.
32 ###The module manages the following
34 * Elasticsearch repository files.
35 * Elasticsearch package.
36 * Elasticsearch configuration file.
37 * Elasticsearch service.
38 * Elasticsearch plugins.
39 * Elasticsearch templates.
43 * The [stdlib](https://forge.puppetlabs.com/puppetlabs/stdlib) Puppet library.
46 #### Repository management
47 When using the repository management you will need the following dependency modules:
49 * Debian/Ubuntu: [Puppetlabs/apt](http://forge.puppetlabs.com/puppetlabs/apt)
50 * OpenSuSE: [Darin/zypprepo](https://forge.puppetlabs.com/darin/zypprepo)
56 ####Install a specific version
59 class { 'elasticsearch':
64 Note: This will only work when using the repository.
66 ####Automatic upgrade of the software ( default set to false )
68 class { 'elasticsearch':
73 ####Removal/decommissioning
75 class { 'elasticsearch':
80 ####Install everything but disable service(s) afterwards
82 class { 'elasticsearch':
89 This module works with the concept of instances. For service to start you need to specify at least one instance.
93 elasticsearch::instance { 'es-01': }
96 This will set up its own data directory and set the node name to `$hostname-$instance_name`
100 Instance specific options can be given:
103 elasticsearch::instance { 'es-01':
104 config => { }, # Configuration hash
105 init_defaults => { }, # Init defaults hash
106 datadir => [ ], # Data directory
110 See [Advanced features](#advanced-features) for more information
114 Install [a variety of plugins](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-plugins.html#known-plugins):
116 ####From official repository
118 elasticsearch::plugin{'lmenezes/elasticsearch-kopf':
119 module_dir => 'kopf',
120 instances => 'instance_name'
125 elasticsearch::plugin{ 'elasticsearch-jetty':
126 module_dir => 'jetty',
127 url => 'https://oss-es-plugins.s3.amazonaws.com/elasticsearch-jetty/elasticsearch-jetty-1.2.1.zip',
128 instances => 'instance_name'
134 You can also use a proxy if required by setting the `proxy_host` and `proxy_port` options:
136 elasticsearch::plugin { 'lmenezes/elasticsearch-kopf',
137 module_dir => 'kopf',
138 instances => 'instance_name',
139 proxy_host => 'proxy.host.com',
144 #####Plugin name could be:
145 * `elasticsearch/plugin/version` for official elasticsearch plugins (download from download.elasticsearch.org)
146 * `groupId/artifactId/version` for community plugins (download from maven central or oss sonatype)
147 * `username/repository` for site plugins (download from github master)
151 #### Add a new template using a file
153 This will install and/or replace the template in Elasticsearch:
156 elasticsearch::template { 'templatename':
157 file => 'puppet:///path/to/template.json'
161 #### Add a new template using content
163 This will install and/or replace the template in Elasticsearch:
166 elasticsearch::template { 'templatename':
167 content => '{"template":"*","settings":{"number_of_replicas":0}}'
171 #### Delete a template
174 elasticsearch::template { 'templatename':
181 By default it uses localhost:9200 as host. you can change this with the `host` and `port` variables
184 elasticsearch::template { 'templatename':
185 host => $::ipaddress,
190 ###Bindings / Clients
192 Install a variety of [clients/bindings](http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html):
197 elasticsearch::python { 'rawes': }
202 elasticsearch::ruby { 'elasticsearch': }
205 ###Connection Validator
207 This module offers a way to make sure an instance has been started and is up and running before
208 doing a next action. This is done via the use of the `es_instance_conn_validator` resource.
210 es_instance_conn_validator { 'myinstance' :
211 server => 'es.example.com',
216 A common use would be for example :
220 require => Es_Instance_Conn_Validator['myinstance'],
224 ###Package installation
226 There are 2 different ways of installing the software
230 This option allows you to use an existing repository for package installation.
231 The `repo_version` corresponds with the major version of Elasticsearch.
234 class { 'elasticsearch':
236 repo_version => '1.4',
240 ####Remote package source
242 When a repository is not available or preferred you can install the packages from a remote source:
246 class { 'elasticsearch':
247 package_url => 'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.deb'
253 class { 'elasticsearch':
254 package_url => 'puppet:///path/to/elasticsearch-1.4.2.deb'
260 class { 'elasticsearch':
261 package_url => 'file:/path/to/elasticsearch-1.4.2.deb'
267 Most sites will manage Java separately; however, this module can attempt to install Java as well.
268 This is done by using the [puppetlabs-java](https://forge.puppetlabs.com/puppetlabs/java) module.
271 class { 'elasticsearch':
276 Specify a particular Java package/version to be installed:
279 class { 'elasticsearch':
280 java_install => true,
281 java_package => 'packagename'
285 ###Service management
287 Currently only the basic SysV-style [init](https://en.wikipedia.org/wiki/Init) and [Systemd](http://en.wikipedia.org/wiki/Systemd) service providers are supported, but other systems could be implemented as necessary (pull requests welcome).
292 The *defaults* file (`/etc/defaults/elasticsearch` or `/etc/sysconfig/elasticsearch`) for the Elasticsearch service can be populated as necessary. This can either be a static file resource or a simple key value-style [hash](http://docs.puppetlabs.com/puppet/latest/reference/lang_datatypes.html#hashes) object, the latter being particularly well-suited to pulling out of a data source such as Hiera.
296 class { 'elasticsearch':
297 init_defaults_file => 'puppet:///path/to/defaults'
300 #####hash representation
303 'ES_USER' => 'elasticsearch',
304 'ES_GROUP' => 'elasticsearch',
307 class { 'elasticsearch':
308 init_defaults => $config_hash
312 Note: `init_defaults` hash can be passed to the main class and to the instance.
319 There are 4 different ways of setting data directories for Elasticsearch.
320 In every case the required configuration options are placed in the `elasticsearch.yml` file.
325 `/usr/share/elasticsearch/data/$instance_name`
327 Which provides a data directory per instance.
330 ####Single global data directory
333 class { 'elasticsearch':
334 datadir => '/var/lib/elasticsearch-data'
337 Creates the following for each instance:
339 `/var/lib/elasticsearch-data/$instance_name`
341 ####Multiple Global data directories
344 class { 'elasticsearch':
345 datadir => [ '/var/lib/es-data1', '/var/lib/es-data2']
348 Creates the following for each instance:
349 `/var/lib/es-data1/$instance_name`
351 `/var/lib/es-data2/$instance_name`
354 ####Single instance data directory
357 class { 'elasticsearch': }
359 elasticsearch::instance { 'es-01':
360 datadir => '/var/lib/es-data-es01'
363 Creates the following for this instance:
364 `/var/lib/es-data-es01`
366 ####Multiple instance data directories
369 class { 'elasticsearch': }
371 elasticsearch::instance { 'es-01':
372 datadir => ['/var/lib/es-data1-es01', '/var/lib/es-data2-es01']
375 Creates the following for this instance:
376 `/var/lib/es-data1-es01`
378 `/var/lib/es-data2-es01`
381 ###Main and instance configurations
383 The `config` option in both the main class and the instances can be configured to work together.
385 The options in the `instance` config hash will merged with the ones from the main class and override any duplicates.
390 class { 'elasticsearch':
391 config => { 'cluster.name' => 'clustername' }
394 elasticsearch::instance { 'es-01':
395 config => { 'node.name' => 'nodename' }
397 elasticsearch::instance { 'es-02':
398 config => { 'node.name' => 'nodename2' }
403 This example merges the `cluster.name` together with the `node.name` option.
407 When duplicate options are provided, the option in the instance config overrides the ones from the main class.
410 class { 'elasticsearch':
411 config => { 'cluster.name' => 'clustername' }
414 elasticsearch::instance { 'es-01':
415 config => { 'node.name' => 'nodename', 'cluster.name' => 'otherclustername' }
418 elasticsearch::instance { 'es-02':
419 config => { 'node.name' => 'nodename2' }
423 This will set the cluster name to `otherclustername` for the instance `es-01` but will keep it to `clustername` for instance `es-02`
425 ####Configuration writeup
427 The `config` hash can be written in 2 different ways:
429 ##### Full hash writeup
431 Instead of writing the full hash representation:
433 class { 'elasticsearch':
436 'name' => 'ClusterName',
440 'attributes' => 'rack'
448 ##### Short hash writeup
450 class { 'elasticsearch':
453 'name' => 'ClusterName',
454 'routing.allocation.awareness.attributes' => 'rack'
463 This module has been built on and tested against Puppet 3.2 and higher.
465 The module has been tested on:
469 * Ubuntu 12.04, 14.04
472 Other distro's that have been reported to work:
478 Testing on other platforms has been light and cannot be guaranteed.
485 Need help? Join us in [#elasticsearch](https://webchat.freenode.net?channels=%23elasticsearch) on Freenode IRC or subscribe to the [elasticsearch@googlegroups.com](https://groups.google.com/forum/#!forum/elasticsearch) mailing list.