Lucene Query Tutorial for SOC Analysts (Grafana Explore)

Audience

Security Operations Center (SOC) analysts working with OpenSearch / Wazuh Indexer backend storage and SOCFortress SIEM and visualizations in Grafana to investigate and hunt cybersecurity events.
The SOCFortress backend to store logs, events and alerts is based on the Wazuh Indexedr / OpenSearch. OpenSearch is an open-source, enterprise-grade search and observability suite.

1. Introduction to Lucene in OpenSearch

Lucene is the core search engine that powers OpenSearch. While OpenSearch supports both Lucene Query Syntax and its own Query DSL (JSON-based), Lucene syntax is used in the SIEM stack tools deployed by SOCFortress, Graylog and Grafana (Explore menu).

2. Basic Lucene Syntax

Match all:

*

Note: Using Wildcards is NOT a good practice when running queries due to the resources consumed in OpenSearch.

🔹 Match field value:

rule_group2:sysmon

🔹 AND / OR / NOT logic:

rule_group2:sysmon AND rule_group3:sysmon_event1
rule_group2:sysmon AND (rule_group3:sysmon_event1 OR rule_group3:sysmon_event1)

🔹 Grouping:

rule_group2:sysmon AND (rule_group3:sysmon_event1 OR rule_group3:sysmon_event1) AND agent_name:windows_server1

3. Practice queries

1. Failed logins by user “administrator” in the past hour

data_win_system_eventID:4625:"4625" AND data_win_eventdata_targetUserName:"administrator" AND timestamp:[now-1h TO now]

• Use wildcards sparingly — they slow down searches.
• Use quotations for exact phrases, e.g., "C:\Windows\System32\cmd.exe".
• Group queries using parentheses to avoid logic errors.
• Save common queries in Grafana Explorer (later in this tutorial).
• Combine Lucene with dashboards and visualizations as provided by the SOCFortress team.

5. Lucene queries in Graylog

Lucene queries in Graylog can be executed via its “Search” menu:

Or selecting any of the Streams available in the “Streams” menu.

A big advcantage of using queries in Graylog is its predictive search, also known as autocomplete or search-as-you-type.

This feature in Graylog suggests relevant search terms as you type in the search bar. It provides real-time suggestions based on initial keystrokes and the fields strored in the backend:

6. Lucene queries in Grafana (Explore)

Use Explore to query, collect, and analyze data for detailed real-time data analysis. 

With Explore it's possible to:

• Create visualizations to integrate into your dashboards.
• Create queries using mixed data sources.
• Create multiple queries within a single interface.
• Understand the shape of your data across various data sources.
• Perform real time data exploration and analysis.

Explore will be accessed from the main, left menu in Grafana:

After that, we need to select the Datasource to load data from (FORTINET) in the example below:

By default, Explore will plot events / unit of time for the selected datasource:

If we intend to use Explore for log/event analysis, we'll change the Metric from “Count” to “Logs” and click “Run Query”:

NOTE: Explore will use “Last 1 hour”; simply change as desired.

After that, logs will be displayed:

Note that the logs displayed will indicate the severity of the events registered, as claddified by the Fortigate firewall (in this example).

Since OpenSearch is a text-based database, we can, for example, type “8.8.8.8” in the query and search for ANY appearance of this string in all logs collected:

In the logs we can also select to display the field “action” and all matches for 8.8.8.8 and the firewall action will be shown in this case: