ElastAlert: The Archived Alerting Framework That Defined Elasticsearch Monitoring

Hook

Before Elastic Watcher existed, a small team at Yelp built an alerting system so elegant that it accumulated 8,000 GitHub stars—and then they archived it. Here's why it mattered, and what replaced it.

Context

In the early 2010s, Elasticsearch emerged as the go-to solution for log aggregation and search, but it had a glaring omission: no native alerting. Companies were ingesting millions of log entries daily, yet had no straightforward way to be notified when error rates spiked, servers went silent, or unusual patterns emerged. The alternatives were bleak—write custom scripts that constantly polled Elasticsearch, build entire alerting platforms from scratch, or simply miss critical issues until users complained.

Yelp faced this exact problem at scale. With thousands of services generating logs, they needed a solution that could define complex alerting rules without requiring engineers to write Python for every new alert. In 2014, they open-sourced ElastAlert, a framework that transformed alerting from a programming exercise into a configuration problem. By expressing rules in simple YAML files, developers could monitor frequency spikes, detect flatlines, track cardinality changes, and route alerts to Slack, PagerDuty, or email—all without touching application code. The project became one of the most-starred Elasticsearch tools on GitHub, battle-tested in production by Yelp and adopted by thousands of organizations. Then, in 2020, Yelp archived it, leaving users scrambling to understand what happened and where to migrate.

Technical Insight

ElastAlert's architecture is deceptively simple: it's a Python daemon that runs a continuous loop, executing Elasticsearch queries at defined intervals and evaluating results against pluggable rule types. The genius lies in its abstraction layers. Instead of embedding alerting logic in application code, you write YAML configurations that declare what to monitor and when to alert.

Here's a practical example—detecting when error logs spike beyond normal levels:

name: Production Error Spike
type: spike
index: logs-*
timestamp_field: '@timestamp'

# Query only error-level logs
filter:
- term:
    level: "ERROR"

# Alert if errors increase 3x compared to previous hour
threshold_ref: 3
timeframe:
  hours: 1
spike_height: 3
spike_type: "up"

# Send to Slack when triggered
alert:
- slack:
    slack_webhook_url: "https://hooks.slack.com/services/YOUR/WEBHOOK"
    slack_username_override: "ElastAlert"
    slack_emoji_override: ":warning:"

Under the hood, ElastAlert loads this configuration and instantiates a SpikeRule object. Every minute (configurable via run_every), it queries Elasticsearch for documents matching the filter within the specified timeframe, compares the current count against the reference window, and triggers the Slack alerter if the threshold is breached. The modular design separates concerns beautifully—rule types handle matching logic while alert types manage notification delivery.

The framework includes eight built-in rule types that cover most monitoring scenarios: frequency (count exceeds threshold), spike (sudden increase/decrease), flatline (no documents received), new_term (first appearance of a field value), cardinality (unique value count changes), metric_aggregation (numeric field thresholds), percentage_match (ratio of matching documents), and any (simple query match). Each rule type is a Python class inheriting from RuleType, making custom rules straightforward to implement.

ElastAlert's alert enhancement system is particularly clever. You can aggregate fields, attach Kibana dashboard links, and include arbitrary metadata in notifications:

aggregation:
  hours: 1

# Group alerts by host to reduce noise
aggregation_key: "host.keyword"

# Include top error messages in alert
top_count_keys:
  - "error.message"
top_count_number: 5

# Link directly to Kibana for investigation
kibana_url: "https://kibana.company.com"
use_kibana4_dashboard: "logs-dashboard"

This configuration batches alerts by host over one hour, preventing alert storms when a single server fails. The alert includes the five most common error messages and a direct link to the relevant Kibana dashboard with pre-filtered timestamps. This level of polish made ElastAlert production-ready out of the box.

The polling architecture trades some real-time responsiveness for operational simplicity. ElastAlert maintains state in Elasticsearch itself (or optionally in local files), tracking the last queried timestamp per rule to ensure no data gaps. When a query runs, it fetches documents since the last successful run, processes them through the rule logic, and updates the state. If ElastAlert crashes and restarts, it resumes from the last checkpoint. This design proved remarkably robust in Yelp's production environment, handling service restarts, network hiccups, and Elasticsearch cluster maintenance without losing alerts or double-firing.

Gotcha

The elephant in the room: ElastAlert is officially archived and unmaintained. Yelp moved on, leaving the original repository frozen in 2020. This isn't just a minor inconvenience—it means no compatibility updates for modern Elasticsearch versions (8.x+), no security patches, and no bug fixes. The Elasticsearch query DSL and APIs have evolved, and newer Elasticsearch versions may reject queries that ElastAlert generates. If you're running Elasticsearch 7.x or earlier, it might still function, but you're building on a foundation that will only become more brittle over time.

Even when it was actively maintained, ElastAlert had architectural limitations. The polling-based design introduces inherent latency—if you run queries every minute, you could miss a ten-second spike that resolved itself. Heavy aggregations across large time windows can generate significant Elasticsearch cluster load, potentially impacting production query performance. I've seen poorly configured spike rules with wide timeframes effectively DDoS their own Elasticsearch clusters during high-traffic periods. You need to carefully tune buffer_time, run_every, and query complexity to balance alert freshness against cluster impact.

Alert deduplication also requires manual configuration. Without proper aggregation_key and realert settings, you'll experience alert fatigue—hundreds of Slack messages for the same underlying issue. The documentation helps, but getting the balance right between noise reduction and missing critical alerts takes iteration and production experience.

Verdict

Use if: You're maintaining a legacy system already running ElastAlert on Elasticsearch 7.x and cannot allocate migration resources immediately—it still works in frozen environments. Better yet, migrate to ElastAlert2 (the community fork at jertel/elastalert2) if you want the same YAML-based approach with active development, modern Elasticsearch support, and ongoing security updates. ElastAlert2 maintains near-complete compatibility with original configurations, making migration straightforward.

Skip if: You're starting fresh, running Elasticsearch 8.x+, or need long-term support. Use Elastic's native Alerting/Watcher feature instead if you have a paid license—it's deeply integrated and officially supported. For open-source solutions, ElastAlert2 is the direct successor. If you're already using Grafana for visualization, its built-in alerting can query Elasticsearch and provides a unified interface. Don't deploy the archived YelpArchive/elastalert repository in new projects—you're inheriting technical debt from day one.