Seamless Integrations for your Data
Get telemetry from any source to your favorite destinations.
Quick Reference Guide
Analyze large-scale datasets in Amazon S3 using standard SQL
Destination
Amazon Athena is an interactive query service that facilitates analyzing large-scale datasets in Amazon S3, using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
This integration is facilitated through Cribl Stream’s S3 Destination.
Configure Stream to send data to S3 via Destinations > Amazon S3.
Configure bucket name, AWS Region, staging location, data format, key and file name prefixes, optional partitioning expression, and other details.
Authenticate via keys or IAM role.
Stream will start sending data to S3 as it becomes available.
In Athena, define the schema and start querying using standard SQL.
A fast, worldwide content delivery network (CDN)
Source
Amazon Cloudfront is a fast content delivery network (CDN) service that securely delivers data, videos, applications, and APIs worldwide with low latency, high transfer speeds, and a developer-friendly environment.
This integration is facilitated through Cribl Stream’s S3 Source and/or Kinesis Source.
Standard logs (access logs)
Enable access logging in Cloudfront.
Specify an Amazon S3 bucket for log storage, granting appropriate permissions to write to the bucket.
Configure Stream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
Real-time logs
Enable real-time logs in Cloudfront.
Configure the desired sampling rate, fields, and path patterns.
Specify a Kinesis data stream, by its Amazon Resource Name (ARN).
Specify the Kinesis stream’s number of shards and IAM role.
Configure Stream to read data from Kinesis via Sources > Amazon Kinesis.
Specify the stream name, AWS Region, record data format, authentication details, and optional parameters.
Stream will receive data as it becomes available.
Monitor and store log files from AWS and other resources
Source
The Amazon CloudWatch Logs service offers centralized, highly scalable monitoring and storage of log files from Amazon EC2 instances, AWS CloudTrail, Amazon Route 53, and other sources.
Data ingestion from Amazon CloudWatch Logs is facilitated through Cribl Stream’S S3, Kinesis, and/or Kinesis Firehose Sources.
CloudWatch Logs as source and Stream as a destination
Configure CloudWatch Logs to export log events to AWS S3 buckets.
Specify an Amazon S3 bucket for log storage, granting appropriate permissions to write to the bucket.
Configure Stream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue.
IAM roles or manual keys are both supported.
LogStream will start fetching data as SQS messages become available.
CloudWatch Logs as Destination and Stream as a source
Configure Stream to send data to CloudWatch Logs via Destinations > Amazon CloudWatch Logs.
Specify the log group, log stream, AWS Region, and optional parameters.
Authenticate via keys or IAM role.
Stream will start sending data as it becomes available.
Monitor and store log files from AWS and other resources
Destination
The Amazon CloudWatch Logs service offers centralized, highly scalable monitoring and storage of log files from Amazon EC2 instances, AWS CloudTrail, Amazon Route 53, and other sources.
For data export to CloudWatch Logs, there is a built-in integration between Cribl Stream and CloudWatch Logs.
CloudWatch Logs as Destination and Stream as a source
Configure Stream to send data to CloudWatch Logs via Destinations > Amazon CloudWatch Logs.
Specify the log group, log stream, AWS Region, and optional parameters.
Authenticate via keys or IAM role.
Stream will start sending data as it becomes available.
Automatically distribute incoming application traffic across multiple targets
Source
Amazon’s Elastic Load Balancing services manage traffic and capacity for deployments on AWS. These services automatically distribute incoming application traffic, scaling resources to handle traffic demands. Application Load Balancers (ALB) route HTTP/HTTPS traffic. Network Load Balancers (NLB) and Classic Load Balancers (CLB) route TCP traffic.
These integrations are facilitated through Cribl Stream’s S3 Source.
Enable access logging for your load balancer.
Specify an Amazon S3 bucket for log storage, and (as needed) grant Elastic Load Balancing permission to write to the bucket.
Configure Stream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
Load streaming data into data stores, data lakes, and analytics tools
Source
Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to data lakes, data stores, and analytics services.
This is a built-in integration between Cribl Stream and Amazon Firehose.
Configure LogStream to read data from Firehose via Sources > Amazon Firehose.
Specify the IP address, port, authorization tokens, and any TLS credentials to use when connecting to Amazon Firehose.
Stream will start fetching data as the Firehose stream becomes available.
Scalable, durable, real-time data streaming
Destination
Amazon Kinesis Data Streams (KDS) is a scalable, durable, real-time data streaming service.
This is a built-in integration between Cribl Stream and Amazon Kinesis Data Streams.
Amazon KDS as Source and Stream as a destination
Create a Kinesis data stream using the AWS Management Console, CLI. or KDS API
Configure Stream to read data from Amazon KDS via Sources > Amazon Kinesis.
Specify the stream name, region, and optional shard segmentation details. IAM roles or manual keys are both supported.
Stream will start fetching data as KDS streams become available.
Amazon KDS as Destination and Stream as a source
Configure Stream to send data to Amazon KDS via Destinations > Amazon Kinesis.
Specify the stream name, region, and optional details.
Authenticate via IAM role or keys.
Stream will start sending data as it becomes available.
Scalable, durable, real-time data streaming
Source
Amazon Kinesis Data Streams (KDS) is a scalable, durable, real-time data streaming service.
This is a built-in integration between Cribl Stream and Amazon Kinesis Data Streams.
Amazon KDS as Source and Stream as a destination
Create a Kinesis data stream using the AWS Management Console, CLI. or KDS API
Configure Stream to read data from Amazon KDS via Sources > Amazon Kinesis.
Specify the stream name, region, and optional shard segmentation details. IAM roles or manual keys are both supported.
Stream will start fetching data as KDS streams become available.
Amazon KDS as Destination and Stream as a source
Configure Stream to send data to Amazon KDS via Destinations > Amazon Kinesis.
Specify the stream name, region, and optional details.
Authenticate via IAM role or keys.
Stream will start sending data as it becomes available.
Highly available, scalable, IPv6-compliant DNS routing
Source
Amazon Route 53 is a highly available, scalable, IPv6-compliant Domain Name System (DNS) web service. It provides reliable routing to Internet applications and infrastructure running in or outside AWS.
This integration is facilitated through Cribl Stream’s S3 Source.
In Route 53, configure query logging to send logs to CloudWatch Logs.
Configure CloudWatch to export Route 53 logs to AWS S3 buckets.
Configure Stream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
Secure, durable, highly scalable object storage
Source
Amazon Simple Storage Service (S3) offers storage of any amount of data, at any time, from anywhere on the web. S3 is accessible via a web services interface, a management console, SDKs for several languages and frameworks, and several APIs.
This is a built-in integration between Cribl Stream and the Amazon S3 APIs. Stream pulls data from S3 buckets using event notifications through Amazon SQS. Stream’s S3 Destination can be adapted to send data to services for which Stream currently has no preconfigured Destination.
S3 as Source and Stream as a destination
Configure your systems to store outbound log data in AWS S3 buckets.
Configure LogStream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
S3 as Destination and Stream as a source
Configure Stream to send data to S3 via Destinations > Amazon S3.
Configure bucket name, AWS Region, staging location, data format, key and file name prefixes, optional partitioning expression, and other details.
Authenticate via keys or IAM role.
Stream will start sending data as it becomes available.
Managed messaging service for AWS, HTTP, email, and mobile receivers
Source
Amazon Simple Notification Service (SNS) is a fully managed messaging service for application-to-application (A2A) and application-to-person (A2P) communication.
This integration is facilitated through the Cribl Stream SQS Source.
Configure SNS to publish messages to an Amazon SQS queue.
Configure Stream to read data from SQS via Sources > Amazon SQS.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
Fully managed message queuing service for distributed systems and microservices
Destination
Fully managed message queuing service for distributed systems and microservices
This is a built-in integration between Cribl Stream and Amazon SQS.
Amazon SQS as Destination and Stream as a source
Configure Stream to send data to Amazon KDS via Destinations > Amazon Kinesis.
Specify the stream name, region, and optional details.
Authenticate via IAM role or keys.
Stream will start sending data as it becomes available.