Monitoring
Dalet Flex comes bundled with different tools related to monitoring and observability. All run on the Monitor instance and allows you to understand system's behavior and helps with troubleshooting.
Elastic stack collects system and application logs from every instance and consists of:
- ElasticSearch, the logs indexing and querier engine.
- Logstash, the logs collector, filtering and processing engine.
- Kibana, the WebUI, used to analyze logs through various dashboards.
Victoria Metrics stack is used to collect various system and application metrics from every instance. Each Dalet Flex server instance features a built-in Grafana Alloy agent, which periodically collect metrics from various exporters under Prometheus format and pushes them to centralized Victoria Metrics instance. All collected metrics are exposed to Grafana under PromQL format.
Grafana is industry-standard metrics visualization engine. The Grafana instance provided with Dalet Flex comes bundled with dozens of ready-to-be-used dashboards, allowing to track system's behavior over time.
AlertManager is directly associated to Prometheus metrics to provide incident management and alerting toolset. Based on known KPIs (e.g. excessive CPU / Memory usage over a long period of time, failed jobs ratio ...), rules are being processed, leading to alerting notifications to one of the various backends (Email, PagerDuty, Slack, MS Teams, WebHooks) to your Operations team.
Dalet Flex observability stack relies on per-node agents, used to collect logs and metrics and shipping them to:
- Flex internal Monitor instance
- Dalet Control Tower, when possible (recommended usage).
- Customer third-party observability stack, when required.