FLX_APP_FLEX_006 Alert

Description

This alert is raised when there is one or more FLEX services are down.

Severity

This alert is flagged as Critical.

Customer Impact

With failed flex services in the flex node, the functionality of flex UI will be affected. The flex application comprises many microservices interconnected with each other. Hence it's crucial to have healthy flex services running across all nodes.

Depending on the failed service, the flex application will be impacted.

  • best case: If you get a failed transfer service, in this case, the user won't be able to use web upload to upload assets.
  • worst case: If you get a failed master or job service, this can hugely impact the overall performance of the flex application.

Operational Remediation Process

Log in to the servers which are hosting the services which are down.

$ ssh SERVER

Note the health state of the affected service.

$ docker ps | grep <service name>

Get the logs of the services.

$ docker logs <container_name>

You can give try to restart one service at a time and wait for it to become healthy. Check the logs again and verify if any error repeats.

$ docker compose stop <service name>
$ docker compose start <service name>

If the error repeats even after restart, The issue should be escalated to the customer support team, else monitor the service for a while.