What might you accomplish if you could easily search your data lakes without paying to move the data first? The most likely outcome is that you address a critical security incident quicker than ever, save your organization millions of dollars, get a promotion, and then go down in history as the best-looking, most talented analyst to have searched a storage bucket.
If not that, you’ll at least be able to consistently make better-informed security decisions, uncover actionable insights, improve outcomes, and find competitive advantages hidden within your data lake. Either way, you’re in good shape.
Connect Your Cribl Search Environment to Google Cloud Storage
Cribl Search is making futures brighter with native support for platforms like Amazon S3, Amazon Security Lake, Azure Blob, and Google Cloud Storage — enabling seamless data analysis right at its source. You can target specific data, create new datasets in minutes, and start searches with just one click.
Here’s how you can set up your Cribl Search environment to connect to your Google Cloud Storage (GCS) in just a few minutes. Check out this video or read below to get set up and start searching your data.
Start Creating a Google Cloud Storage Dataset Provider
The dataset provider tells Cribl Search where to query and what access credentials to use.
To set this up, navigate to Data → Dataset Providers in your Cribl Search dashboard. Click Create Provider in the top right and select Google Cloud Storage as your dataset provider type. Give the dataset provider a unique ID and description.
Down below, you’ll see a field to put in the authentication key from the GCS side. First, you’ll have to create a service account and key on the GCS side — the account will also need access with storage admin role privileges to the storage bucket you’d like to search.
Set Up Google Cloud Storage Service Account and Key
Move Into the GCP console to set up your Google Cloud Storage service account and key.
First, you’ll need to create a service account that Cribl Search will use to access your GCS resources. Select IAM & Admin from the left navigation menu. Click Service Accounts and then Create Service Account. Give the service account a name and click Done at the bottom. Find the account you just created in the list and copy the email address of that service account for later.
Next, you’ll pivot to the GCS bucket to grant the service account access to your data. Select Cloud Storage from the left navigation menu and select your bucket from the list. Click Permissions → Grant Access and paste your service account email address into the New Principals field. Use the dropdown to assign the Storage Admin role and click Save.
The last item on the GCP side is to create a key for your service account. Navigate back to service accounts, click the one you created, then select the KEYS → ADD KEY → Create new key. Select JSON for the key type and click Create. You’ll see a download for your key in your downloads folder.
Finish Setting Up Your Dataset Provider
Now, you can pivot back to Cribl Search to finish setting up your dataset provider. Drag and drop the downloaded key into the Service Account Credentials field, click Save, and your dataset provider will be ready to use.
Set Up Your Google Cloud Services Dataset in Cribl Search
Next, you’ll set up your dataset so that Cribl Search knows exactly which data to search. Select Datasets from the left side menu and click Add Dataset. in the top right. Give it a name and description, then select the provider you created.
When you select a dataset provider, a couple of extra fields will pop up:
Bucket Path refers to any prefix within your bucket that you want to search. Here, I added my bucket name, an archive prefix, and some tokenized, time-based prefixes. Feel free to change this depending on what your partitions look like.
Path Filter accepts any Javascript expressions for path filters you want to add.Once you click Save, your data set should be ready to use. Start searching by navigating back to the homepage and starting a search there, or leverage the Search Action Button at the end of the row for your data set.
Check out our documentation for more detailed info on how to set up Cribl Search to query Google Cloud Storage buckets, or visit our YouTube Channel to learn what else you can do with Cribl Search — including How to Search Azure Blob Storage Containers and How to Search Amazon S3 Buckets.
