How to install ElasticSeach plugins using docker compose

Question

I have a docker-compose.yml file with an elastic search image:

elasticsearch:
  image: elasticsearch
  ports:
    - "9200:9200"
  container_name: custom_elasticsearch_1

If I want to install additional plugins like the HQ interface or the attachment-mapper I have to do a manual installation with the following commands:

$ docker exec custom_elasticsearch_1 plugin install royrusso/elasticsearch-HQ
$ docker exec custom_elasticsearch_1 plugin install mapper-attachments

Is there a way to install them automatically when I run the docker-compose up command?

score 17 · Accepted Answer · edited Oct 08 '21 at 07:06

17

Here is a blog post by Elastic pertaining to exactly that! You need to use a Dockerfile which executes commands to extend an image. Your Dockerfile will look something like this:

FROM custom_elasticsearch_1

RUN elasticsearch-plugin install royrusso/elasticsearch-HQ

edited Oct 08 '21 at 07:06

Jolta

2,620
1
29
42

answered Sep 25 '16 at 20:50

fylie

1,675
1
10
14

6

Mmm so there is no way to achieve that without adding another Dockerfile? I would like to do it inside the docker-compose – iamdeit Sep 25 '16 at 20:57
3

Docker compose uses the Dockerfile if you add the build command to docker-compose.yml. [Here](https://blog.codeship.com/orchestrate-containers-for-development-with-docker-compose/) is an example of use of a Dockerfile with Compose. But, [here](https://coderwall.com/p/ouemxq/install-plugins-on-elasticsearch-with-docker-compose) is an example of someone installing plugins without a Dockerfile. – fylie Sep 25 '16 at 22:36
I see, apparently an additional file is always required. – iamdeit Sep 25 '16 at 22:50
Thanks, @fylie. Your answer probably saved me hours today. – missingfaktor Jan 15 '18 at 14:47
You're welcome @missingfaktor! Glad my answer helped :) – fylie Jan 16 '18 at 18:49
Thanks @fylie, I ended using the [second link](https://coderwall.com/p/ouemxq/install-plugins-on-elasticsearch-with-docker-compose) in your comment – JP Lew Dec 19 '18 at 23:41
1

this does not work with latest Docker version as of 2021. The `RUN plugin install ` throws error saying `plugin` in not found! The following works: `RUN elasticsearch-plugin install analysis-icu` Please update the answer as this has been accepted as answer by the question owner. – Asrar May 22 '21 at 00:19
@Asrar, done. (The command has nothing to do with Docker, it's changed by Elasticsearch.) – Jolta Oct 08 '21 at 07:07

score 15 · Answer 2 · answered Oct 01 '19 at 21:30

15

Inspired by @NickPridorozhko's answer, but updated and tested with elasticsearch^7.0.0 (with docker stack / swarm), example with analysis-icu:

elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
  user: elasticsearch
  command: >
    /bin/sh -c "./bin/elasticsearch-plugin list | grep -q analysis-icu 
    || ./bin/elasticsearch-plugin install analysis-icu; 
    /usr/local/bin/docker-entrypoint.sh"
  ...

The main difference are the updated commands for ^7.0.0, and the use of the docker entrypoint instead of ./bin/elasticsearch (in a stack's context, you'd get an error related to a limit of spawnable processes).

answered Oct 01 '19 at 21:30

Maen

10,603
3
45
71

this gives me an error "WARNING: plugin requires additional permissions " when i use it to install repoistory-s3 – Kay Nov 09 '20 at 10:35
2

@Kay That doesn't seem to be related. See [this github issue](https://github.com/elastic/cloud-on-k8s/issues/2220) and this [other answer](https://stackoverflow.com/questions/57406534/ingest-attachment-needs-more-rights). – Maen Nov 09 '20 at 10:48
1

For Opensearch, this works as well, with `/usr/share/opensearch/bin/opensearch-plugin` and `/usr/share/opensearch/opensearch-docker-entrypoint.sh` – mgaert May 06 '22 at 08:53

score 6 · Answer 3 · answered Feb 19 '19 at 11:15

This works for me. Install plugin before and then continue with starting the elasticsearch.

elasticsearch:
  image: elasticsearch
  command:
    - sh
    - -c
    - "plugin list | grep -q plugin_name || plugin install plugin_name;
       /docker-entrypoint.sh elasticsearch"

score 5 · Answer 4 · answered Sep 04 '20 at 10:23

5

The ingest-attachment plugin requires additional permissions and prompts the user during the installation. I used the yes command :

elasticsearch:
  image: elasticsearch:6.8.12
  command: >
    /bin/sh -c "./bin/elasticsearch-plugin list | grep -q ingest-attachment 
    || yes | ./bin/elasticsearch-plugin install --silent ingest-attachment; 
    /usr/local/bin/docker-entrypoint.sh eswrapper"

answered Sep 04 '20 at 10:23

David Le Borgne

801
2
9
23

tried on docker compose can confirm this works – Oliver Dec 30 '21 at 15:26
1

This should be the accepted answer. Worked great for me on ES 7.x and does not require messing around with a custom `Dockerfile`. – Matt Jones Sep 16 '22 at 14:00
Should we create a new docker image of Elasticsearch with this we are trying to add the azure repository plugin – yasin mohammed Oct 06 '22 at 10:58

score 1 · Answer 5 · answered Sep 23 '19 at 14:09

If you're using the ELK stack from sebp/elk

You need to setup your Dockerfile like

FROM sebp/elk

ENV ES_HOME /opt/elasticsearch
WORKDIR ${ES_HOME}

RUN yes | CONF_DIR=/etc/elasticsearch gosu elasticsearch bin/elasticsearch-plugin \
    install -b mapper-attachments

As seen on https://elk-docker.readthedocs.io/#installing-elasticsearch-plugins.

It should also work just for Elastic Search only as well.

score 0 · Answer 6 · answered May 22 '21 at 00:31

An example with Elasticsearch v6.8.15. For simplicity we will use a docker-compose.yml and a Dockerfile.

The content of Dockerfile:

FROM docker.elastic.co/elasticsearch/elasticsearch:6.8.15

RUN elasticsearch-plugin install analysis-icu
RUN elasticsearch-plugin install analysis-phonetic

And the content docker-compose.yml:

version: '2.2'
services:
  elasticsearch:
    #image: docker.elastic.co/elasticsearch/elasticsearch:6.8.15
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - http.cors.allow-headers=X-Requested-With,Content-Type,Content-Length,Authorization
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/data
      - esplugins1:/usr/share/elasticsearch/plugins
    ports:
      - 9268:9200
    networks:
      - esnet
  elasticsearch2:
    #image: docker.elastic.co/elasticsearch/elasticsearch:6.8.15\
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: elasticsearch2
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - http.cors.enabled=true
      - http.cors.allow-origin=*
      - http.cors.allow-headers=X-Requested-With,Content-Type,Content-Length,Authorization
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "discovery.zen.ping.unicast.hosts=elasticsearch"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata2:/usr/share/elasticsearch/data
      - esplugins2:/usr/share/elasticsearch/plugins
    networks:
      - esnet

volumes:
  esdata1:
    driver: local
  esdata2:
    driver: local
  esplugins1:
    driver: local
  esplugins2:
    driver: local

networks:
  esnet:

This is the default Elasticsearch 6.8.15 docker-compose.yml file from Elasticsearch website itself https://www.elastic.co/guide/en/elasticsearch/reference/6.8/docker.html#docker-cli-run-prod-mode. And I added two named data volumes, esplugins1 and esplugins2, for two of these nodes. So these plugins can be persisted between docker-compose down.

Note, if you ever run docker-compose down -v then these volumes will be removed!

I commented out the image line and moved that image to Dockerfile. And then using RUN command added the elasticsearch-plugin install command. This elasticsearch-plugin command is natively available in the elasticsearch container. And you can check this once you are in the container shell.

Dashrath Mundkar · Answer 7 · 2022-06-17T19:23:55.500

0

Just for somebody who is using elasticsearch version starting from 7 and want to install plugin through the dockerfile then use the --batch flag

FROM elasticsearch:7.16.2
RUN bin/elasticsearch-plugin install repository-azure --batch

edited Jun 17 '22 at 19:23

answered Jun 17 '22 at 13:20

Dashrath Mundkar

7,956
2
28
42

How to install ElasticSeach plugins using docker compose

7 Answers7