Cribl

Source

Amazon S3 / S3 Compatible Stores

Secure, durable, highly scalable object storage

Amazon Simple Storage Service (S3) offers storage of any amount of data, at any time, from anywhere on the web. S3 is accessible via a web services interface, a management console, SDKs for several languages and frameworks, and several APIs.

Read our Solution Brief

How to get data flowing

This is a built-in integration between Cribl Stream and the Amazon S3 APIs. Stream pulls data from S3 buckets using event notifications through Amazon SQS. Stream’s S3 Destination can be adapted to send data to services for which Stream currently has no preconfigured Destination.

S3 as Source and Stream as a destination

  • Configure your systems to store outbound log data in AWS S3 buckets.

  • Configure LogStream to read data from S3 via Sources > Amazon S3.

  • Supply your SQS queue. IAM roles or manual keys are both supported.

  • Stream will start fetching data as SQS messages become available.

S3 as Destination and Stream as a source

  • Configure Stream to send data to S3 via Destinations > Amazon S3.

  • Configure bucket name, AWS Region, staging location, data format, key and file name prefixes, optional partitioning expression, and other details.

  • Authenticate via keys or IAM role.

  • Stream will start sending data as it becomes available.

View Documentation

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on-Sandboxes for those interested in how companies globally leverage our products for their data challenges.