1

We are planning to store our (Spring-Boot) application logs to ElasticSearch. I am aware of ELK stack, which uses FileBeat + LogStash to collect and process the logs.

What is desired: Have an appender in logback.xml to directly send the logs to ElasticSearch. The very basic idea is of having an appender like File-Appenders with the difference of target for storing logs being ElasticSearch. At the same time, we want to do it in asynchronous manner. FYI, we are using slf4j with logback implementations for logging.

More specifically: We want to remove the intermediators:: Logstash or Beats as they will need more infra and may bring unwanted overhead. And having the process of sending logs to ElasticSearch in asynchronous way would be really great (so that application does not suffer latency due to logging).

What I have already tried: Send Spring Boot logs directly to LogStash. But it seems of not much use, since it internally uses file appenders and the logs are then sent to LogStash.

Is there any such appenders available? Or maybe there is some workaround.

Harish Kumar Saini
  • 362
  • 3
  • 20
  • When you have multiple machines and multiple indexes for different app and environment logs, we may need logstash, else you can use filebeat as a shipping tool to Elasticsearch. – Jinna Balu Aug 27 '21 at 19:19
  • @JinnaBalu, even for a single app and single index in elastic, Is it possible to send logs to Elastic (app -> elastic) without writing logs -> to log file -> filebeat ->elastic. Is there is no other way but to use logstash/filebeat? – Harish Kumar Saini Aug 28 '21 at 14:50
  • 1
    but that will effect the performance of the application. Let springboot generate a logfile and forget, otherwise we give an extra thread to connect and process the log. – Jinna Balu Aug 28 '21 at 17:51

0 Answers0