“Half the time the toilet’s out of reach – the other half it’s out of order”.

–Arthur C. Clarke, Report on Planet Three and Other Speculations

Sounds familiar, doesn’t it? Arthur C. Clarke humorously hits the nail on the head about life’s balancing acts. The balance between user freedom and resource utilization is key. In the world of data management, we often face a similar conundrum: making sure resources are available and in good working order when we need them the most. This is where Cribl Search steps in with its neat new feature, Usage Groups. It’s like having a smart thermostat for your data management system – it ensures you’re using your resources effectively without burning through your Cloud credits. Let’s dive in and see how Usage Groups can make your life easier and your data management smarter.

What Are Usage Groups?

Usage Groups in Cribl Search allow administrators to set specific limits on search usage for different users or usage groups. This functionality is crucial for managing resource consumption and ensuring users don’t exhaust the allocated credits, thus maintaining optimal system performance. With Usage Groups, admins can ensure that each user operates within a specified resource boundary, fostering a more balanced and efficient use of the Cribl Search credits.

Default Groups

Out of the box, Cribl Search offers two primary groups:

System: This group sets system-level limits applicable to all searches.

Default: This is for all ad hoc searches not covered by other groups.

In addition to that, let’s see how we can create custom usage groups to serve our unique requirements better.

Creating Custom Usage Groups

Creating a new Usage Group is straightforward. Navigate to Settings > Search Settings > Usage Groups and click Add Usage Group. Here, you can name your group, define its limits, and enable it. Assigning users to these groups is just as easy, ensuring everyone has the right level of access and resource allocation.

Setting Up Custom Usage Groups

Let’s say you have different teams within your organization, each with unique search requirements. You aim to ensure efficient use of Cribl Search without compromising the team’s ability to extract valuable insights. We can create different usage groups for different teams, but before we do that, let’s see what settings are available for us and what they do:

Earliest Relative Time Range

What It Does: Sets how far back in time a user can search. For example, settings like 30d or 1y allow searches up to 30 days or one year back, respectively. You can also specify time in seconds by entering a numeric value without a time unit.

User Concurrent Ad Hoc Search Limit

What It Does: Determines the maximum number of ad hoc searches a single user can perform simultaneously. This helps manage the system load, by preventing an individual user from running too many simultaneous searches.

Overall Concurrent Search Limit

What It Does: Caps the total number of concurrent searches that can be conducted across the entire organization. This is crucial for ensuring that the system remains stable and responsive by avoiding an overload of simultaneous search queries.

Executors Limit

What It Does: Specifies the maximum number of executors that can be dispatched for a single search. This limit is important for controlling the computational resources allocated to each search, ensuring efficient use of system resources.

Running Time Limit

What It Does: Sets the maximum duration, in seconds, that a search is allowed to run. This limit prevents searches from running indefinitely, which can tie up resources and affect system performance.

Time Range Limit

What It Does: Defines the maximum time range for a search query. For instance, a limit of 3 days (3d) means a search can’t span more than a 3-day period. This helps in focusing the searches and managing the amount of data being processed.

User Concurrent Scheduled Search Limit

What It Does: Determines the maximum number of scheduled searches a user can have running at the same time. This is vital for balancing the system’s load, especially for searches set to run automatically at specific times.

Results Limit

What It Does: Controls the maximum number of events (data points) that can be returned in a search result. This is important for ensuring that search results are manageable and pertinent.

Byte Limit

What It Does: Puts a cap on the maximum number of bytes that can be read in a single search. This limit is important for managing data throughput and maintaining efficient use of storage and network resources.

Tailoring Usage Groups for Specific Teams

Now that we’re familiar with the available settings in Cribl Search’s Usage Groups, let’s visualize how these settings can be applied to distinct personas or teams:

Scenario 1: The Incident Response Teams – Tiered Approach

Challenge: Incident Response Teams often operate in a tiered structure. The lower-tier team members (the Initial Responders) handle initial assessments, while higher-tier members (Hunters) engage in deep and comprehensive investigations.

Tier 1 Members – Hunters, Conducting Deep-Dive Investigations

• Earliest Relative Time Range: Expanded to 1 year for comprehensive historical context in investigations.
• Running Time Limit: Extended to 28,800 seconds (8 hours), granting the flexibility for extensive, overnight analysis sessions.
• User Concurrent Ad Hoc Search Limit: Increased to 4, allowing for a broad scope of concurrent deep-dive investigations.
• Time Range Limit: Broadened to 30 days to trace the development of incidents over a month.
• Results Limit: Elevated to 200,000 events to ensure a detailed dataset is available for intricate pattern analysis.

Tier 2 Members – The Initial Responders

• Earliest Relative Time Range: Set to 7 days to prioritize recent incidents.
• Running Time Limit: Capped at 300 seconds for swift query returns, facilitating rapid initial response.
• User Concurrent Ad Hoc Search Limit: Limited to 3 to allow for essential multitasking without overwhelming new incidents with complex queries.
• Time Range Limit: Restricted to 24 hours, focusing the team on the most critical, immediate events.
• Results Limit: Set to 1,000 events to provide a quick overview without data overload, ideal for initial sorting and triage.

Scenario 2: Tool/System Admin – The Analytics Product Experts

Challenge: Responsible for maintaining various analytics tools, the Sys Admin requires a broad but controlled access to data, focusing more on system health and less on specific data queries.

• Overall Concurrent Search Limit: Set to 4 to accommodate multiple tool maintenance tasks.
• Executors Limit: Increased to 100, providing the necessary processing power for system-wide diagnostics and maintenance.
• Byte Limit: Set to 10 GB, giving admins the capacity to perform extensive data operations and manage large-scale onboarding.
• User Concurrent Scheduled Search Limit: Positioned at 2 to maintain regular, automated system checks while still reserving bandwidth for unplanned, critical tasks.
• Results Limit: Flexible; adjusted as needed, based on the admin’s role in supporting both IRT members and Hunters with data access.

The Impact of Custom Usage Groups

• Tailored Efficiency: Each persona works within a search environment optimized for specific tasks.
• Resource Optimization: By setting appropriate limits, you align resource consumption with each team’s needs, ensuring a balanced system use.
• Enhanced Productivity: Customized settings empower each team to perform their roles effectively without encountering unnecessary system bottlenecks.
• Controlled Operational Costs: Efficient resource management translates to cost savings, particularly in cloud environments.

Wrap-up: Empowering Teams with Smart Resource Allocation

In essence, Cribl Search’s Usage Groups feature empowers teams with needed tools while maintaining efficient resource management. It ensures that each team has the search capabilities they need to succeed, all under a well-managed, resource-conscious framework.

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.