FLX_BACKEND_ELASTIC_008 Alert

Description

A critical Alert is raised when the logstash service is down.

Severity

This alert is flagged as Warning.

Customer Impact

This alert indicates that the logstash service which is meant to populate the logs to the Elastic search service is down. Logs will stop flowing to Elasticsearch, hence no logs will be available in kibana.

Operational Remediation Process

Login to the server which is hosting the logstash services. Usually its the monitoring server in the environment.

$ ssh SERVER

Note the health state of the affected service.

$ systemctl status logstash

Get the logs of the services.

$ cd /var/log/logstash
$ cat *

You can give it try to restart the logstash service and wait for it to become active. Chances are that after the logstash restart, the log events can be processed.

$ systemctl restart logstash

The logs of the services will give more insight into the actual issue. We need to troubleshoot further with specified errors in the services.

Note: Sometime logstash will take time to restart.