Indexing Strategy in Logmanager 4

Indexing directly affects how fast analysts can search and filter data. In Logmanager 4, the indexing model was redesigned to improve performance in larger environments. This is useful because it reduces the amount of irrelevant data scanned during searches and creates a better foundation for future retention and backup features.

Prefer watching?

Check the related video walkthrough:

1. What an Index Is

In practical terms, an index is a collection of documents optimized for search operations. Each incoming log event becomes a document with fields such as timestamps, IP addresses, usernames, event categories, and parser outputs.

2. How Earlier Versions Worked

In earlier releases, logs were typically stored in daily indexes. All events generated within a 24-hour window were grouped together. This model was simple, but when environments started producing hundreds of gigabytes per day, searches had to scan a very large dataset.

3. What Changed in Logmanager 4

Logmanager 4 introduces per-parser indexing. Instead of putting everything from one day into the same place, logs are grouped based on the parser that processed them. For example, Windows logs handled by the Windows parser are stored together in a dedicated index.

4. Why This Is Useful

When an analyst searches for Windows logs, the platform can go directly to the relevant index instead of scanning through unrelated network, web, or firewall events. This improves search speed and keeps the architecture more scalable.

5. Future Improvements

The new architecture also enables features planned for future releases, such as granular retention policies and the ability to back up specific indexes instead of the entire dataset.