Cybersecurity & Information Security

Elastic Security: Creating Detection Rules and Investigation Guides

6 min readPublished: August 5, 2026

Professional visual illustration on Detection Rule in Elastic Security in the field of SIEM and detection

Quick answer

A good Detection Rule in Elastic Security starts with the behavior to detect and the available data, not with choosing a random language. Select an appropriate Rule type, validate ECS and fields, write a Query, define Schedule and Lookback, Risk and Severity, Suppression and Exceptions, and attach an Investigation Guide that leads the analyst through Triage, Analysis, and Response.

Elastic Security includes a Detection engine that runs Rules on Elasticsearch data and generates Alerts when conditions are met. It supports several Rule types, including Custom query, Event correlation using EQL, Threshold, Indicator match, New terms, ES|QL, and Machine learning. Each type solves a different problem.

A Rule that returns Results is not necessarily a useful Detection. To turn a Query into an operational product, you need a Data contract, Schedule, Triage context, Risk, Exceptions, Owner, a testing process, and an Investigation Guide. The analyst who receives an Alert needs to understand within minutes what was detected, which fields are important, what a likely False Positive is, and what to look for next.

Step 1: Define Use Case and Choose Rule Type

Start with a behavioral statement: “Detect a certain type of Process executed in an unexpected context on user workstations.” Then define what constitutes a match, what Context is required, and what is a legitimate explanation. Only then choose the Rule type.

Question	Suitable Rule type	Example
Matching values or Boolean conditions	Custom query	process.name and command_line
Sequence of events by time and entity	EQL	Process start followed by Network connection
Number of events exceeds a threshold	Threshold	Many failures by user or source.ip
Aggregation, calculation, or derived fields	ES\|QL	STATS by host then WHERE on count
IOC vs. event	Indicator match	destination.ip vs. threat index
Value appearing for the first time	New terms	Rare Process on Host
Behavioral deviation without a rigid Pattern	Machine learning	Anomaly above Threshold

An incorrect choice creates a complicated Query or unstable alerts. If event order is essential, EQL is more natural than ES|QL. If aggregation is needed, ES|QL or Threshold are more suitable. Custom query is good for direct field matching.

Step 2: Ensure ECS and Data Readiness

Elastic Common Schema — ECS — defines common names and structures, such as event.category, event.type, host.name, user.name, process.name, process.command_line, and process.parent.name. A Rule that relies on fields not consistently mapped will work for one Integration and fail for another.

Open Sample events and check: Is event.category process? Does event.type include start? Is process.command_line collected or hidden? Does process.entity_id exist? Is @timestamp the event time or ingestion time? Document Index patterns, Integrations, Version, and Required fields. The list of Required fields in the interface is user information and does not correct actual mapping.

Step 3: Write a Focused Query

In a lab example, we want to detect the creation of a Process named lab-admin-tool.exe when the Parent is not the expected management software. The name is fictional. In a Custom query, you can write a simple KQL:

event.category:process and event.type:start and process.name:"lab-admin-tool.exe" and not process.parent.name:"approved-manager.exe"

The Query indicates an anomaly but does not prove maliciousness. You need to check Signer, Hash, Path, User, Host, Parent command line, and frequency. Before creating a Rule, run it in Discover or Timeline on various ranges. Check for Null fields, Variants in name, or unreliable Sources.

If the behavior requires a sequence — for example, Process followed by Network connection from the same process.entity_id — switch to EQL. If you want to summarize how many Hosts executed the tool or calculate Count by Parent, ES|QL might be suitable.

Step 4: Schedule and Lookback

A Rule includes a Query, Schedule, and Actions. Interval determines how often it runs. Lookback extends the search window to cover late-arriving Events. Overlapping windows can create duplicate Alerts if there is no appropriate Deduplication or Suppression. Too short a window misses late data.

Check Ingestion delay by source. Endpoint events may arrive almost in real-time, while a Cloud or Batch source may be delayed. Define Run every and Additional look-back based on measurement. After changing a Pipeline or Integration, remeasure.

Step 5: Severity, Risk, and MITRE

Severity describes the match's criticality according to the Use Case; Risk score allows numerical ranking. Do not use High for every Rule. An anomalous Process on a Lab host differs from the same Process on a Domain Controller. You can use Risk score override when a reliable field provides Context, but you must check Missing values and Range.

MITRE ATT&CK mapping should match the behavior the Rule detects, not the full attack scenario you imagine. A Rule that detects Process execution does not necessarily prove Persistence or Exfiltration.

Step 6: Suppression and Exceptions

Alert suppression groups recurring matches by fields to reduce volume. It is not a substitute for a correct Query. Suppression by host.name can hide developments if the same Host generates several different Behaviors. Choose fields that represent the investigation unit, such as host.id and process.hash, and define a Window based on Baseline.

Exceptions exclude known matches. Create a narrow Exception: Signed Hash, Path, a specific Parent, and a defined Host group, instead of excluding process.name across the entire organization. Add a Comment, Owner, and Review date. When possible, prefer a managed Exception list over a Query with dozens of NOT clauses.

Step 7: Write an Investigation Guide

An Investigation Guide is a Markdown document attached to the Rule and appears alongside an Alert. According to Elastic, a good guide is structured around Triage, Analysis, and Response, starts with context rather than a list of Commands, and refers to Alert fields, Timeline queries, and Osquery where appropriate.

Part	What to include	Example
Context	What the rule detects and why it's important	Unexpected Process outside management tools
Triage	Quick checks and False Positives	Signer, Path, Parent, Host group
Analysis	Timeline and supplementary searches	Network, User logons, file creation
Response	Steps if confirmed	Escalation, isolation with approval, collection
Closure	Disposition conditions	Approved software, test, compromise

You can refer to dynamic fields such as host.name or user.name and add Timeline buttons when the version and license support it. Keep the guide short and scannable. The analyst works under pressure; long, unstructured paragraphs will remain unread.

Step 8: Validation and Tuning

Run Preview or a historical Query and mark True, False, and Benign Positive examples.
Test the Rule with a simulated Positive sample and a Negative sample. Ensure it fails when a field is missing and does not generate a false match.
Initially, run without a risky Response. Measure Alert volume, investigation times, and Context quality.
Check Rule execution status, gaps, permissions, and API key. Rules run with the permissions of the last user who edited them.
Tune Query, Schedule, Suppression, and Exceptions separately to know what fixed the problem.
Perform a Regression test after updating Integration, ECS mapping, or Elastic version.

Checklist

Rule type is suitable for the question.
Indices, Data view, ECS, and Required fields checked.
Query returns a clear unit of investigation.
Schedule and Lookback cover Delay without excessive duplication.
Severity, Risk, and MITRE align with behavior.
Suppression and Exceptions are narrow and documented.
Investigation Guide includes Triage, Analysis, and Response.
Owner, Review date, Metrics, and Regression test exist.

Common Mistakes

Choosing EQL, KQL, or ES|QL based on preference rather than the question.
Copying a Prebuilt Rule without checking Data requirements.
Assuming ECS is mapped because the field appears in some events.
Increasing Lookback without understanding duplications.
Creating a broad Exception on Process name.
Activating automatic Response before Validation.
Writing an Investigation Guide that only says “check if malicious.”

Summary and CTA

Choose a fictional Process in the lab and build a complete Rule: Data contract, Query, Schedule, Risk, one Exception, Investigation Guide, and Test cases. Then let another analyst investigate an Alert without verbal explanation. If the guide and fields are insufficient, improve the Rule before adding more logic.

FAQ

What Rule type is suitable for a single Process?

Mostly, a Custom query is sufficient if it's about field conditions. If a time sequence is needed, EQL is more suitable; if aggregation is needed, consider ES|QL or Threshold.

Do Required fields guarantee that the fields exist?

No. The list is user documentation. You must check actual Mapping and Sample events.

What is the difference between Suppression and Exception?

Suppression aggregates recurring Alerts; Exception prevents Alert creation when defined conditions are met.

Can I edit the Investigation Guide of a Prebuilt Rule?

The ability depends on the license and version. Sometimes you need to duplicate the Rule and then edit the copy.

Why did a Rule stop working after editing?

Rules use the permissions and API key created for the last user who edited them. A change by a user without read permissions can affect execution.