Page 2 of 9
Fully managed message queuing service for distributed systems and microservices
Source
Fully managed message queuing service for distributed systems and microservices
This is a built-in integration between Cribl LogStream and Amazon SQS.
Amazon SQS as Source and Stream as a destination
Create a Kinesis data stream using AWS’ Management Console, CLI, or KDS API.
Configure Stream to read data from Amazon KDS via Sources > Amazon Kinesis.
Specify the stream name, region, and optional shard segmentation details.
IAM roles or manual keys are both supported for authentication.
Stream will start fetching data as KDS streams become available.
Collect, aggregate, and route log data
Source
Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating, and moving large amounts of log data from many different sources to a centralized data store.
This is an integration facilitated through Cribl Stream’s HTTP/S Source.
Configure a Flume HTTP Sink for Cribl.
Configure Stream to ingest Flume data via Sources > HTTP.
Specify the address, port, optional authentication and TLS settings, and optional throttling and other parameters.
Stream will start receiving data as it becomes available.
Monitor and audit activity on your AWS account
Source
AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. This event history simplifies security analysis, resource change tracking, and troubleshooting. In addition, you can use CloudTrail to detect unusual activity in your AWS accounts. These capabilities help simplify operational analysis and troubleshooting.
This integration is facilitated through the Cribl Stream’s S3 Source.
On AWS side, configure CloudTrail to deliver events to your S3 bucket, Configure your S3 bucket to send event notifications to a SQS queue
Find the integration in Stream througj Sources > Amazon S3
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available
Simple, scalable, fully managed network file system
Destination
Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. It is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.
This integration is facilitated through Cribl Stream’s Filesystem Destination.
Configure Stream to send data to AWS EFS via Destinations > Filesystem.
Specify the output and staging locations, partitioning expression, compression and backpressure behavior, and optional parameters.
Stream will start sending data as it becomes available.
Event-driven, serverless computing
Source
AWS Lambda’s serverless compute service enables running code without provisioning or managing servers or runtimes. Upload code as a ZIP file or container image, and Lambda automatically allocates execution power based on incoming requests, events, and traffic. You pay only for the computing time you consume.
This integration is facilitated through Cribl Stream’s S3 Source and/or Kinesis Source. (Both paths rely on Lambda’s default logging to Amazon CloudWatch Logs.)
Lambda to Stream via S3
Configure CloudWatch Logs to export incoming log events from Lambda to AWS S3 buckets.
Specify an Amazon S3 bucket for log storage, granting appropriate permissions to write to the bucket.
Configure Stream to read data from S3 via Sources > Amazon S3.
Supply your SQS queue. IAM roles or manual keys are both supported.
Stream will start fetching data as SQS messages become available.
Create, and apply, appropriate IAM roles and permissions policies in both Cloudwatch Logs and Amazon Kinesis Data Firehose.
Create a destination Kinesis Firehose delivery stream, configured with the appropriate role and bucket ARNs.
Create a CloudWatch Logs subscription filter to send incoming log events to your Amazon
Kinesis Data Firehose delivery stream.
Configure Stream to read data from Kinesis via Sources > Amazon Kinesis.
Specify the stream name, AWS Region, record data format, authentication details, and optional parameters.
Stream will receive data as it becomes available.
Monitor requests to protect web applications and APIs against common exploits
Source
The AWS Web Application Firewall helps protect web applications and APIs against common web exploits by monitoring the HTTP and HTTPS requests forwarded to other Amazon services. You can use predefined Managed Rules (which are regularly updated to block new threats) and/or write your own security rules. Pricing is based on how many rules you deploy, and on your volume of incoming web requests.
This integration is facilitated through Cribl Stream’s Amazon Kinesis Firehose Source.
Configure AWS WAF to send logs from your web ACL (access control list) to an Amazon Kinesis Data Firehose.
Configure Stream to read data from Firehose via Sources > Amazon Firehose.
Specify the IP address, port, authorization tokens, and any TLS credentials to use when connecting to Amazon Firehose.
Stream will start fetching data as the Firehose stream becomes available.
Axiom captures all your event data for every need
Destination
Axiom is a log management and analytics solution that reduces the cost and management overhead of logging as much data as you want.
With Axiom, organizations no longer need to choose between their data and their costs. Axiom has been built from the ground-up to allow for highly efficient data ingestion and storage, and then a zero-to-infinite query scaling that allows you to query all your data, all the time.
Organizations use Axiom for continuous monitoring and observability, as well as an event store for running analytics and deriving insights from all their event data.
Axiom consists of a datastore and a user-experience that work in tandem to provide a completely unique log-management and analytics experience.
You can send logs from Cribl Stream to Axiom using HTTP or Syslog destination.
Cost-optimized, cloud-oriented object storage for unstructured data
Destination
Azure Blob Storage is Microsoft’s cloud-oriented object storage solution, optimized for storing massive amounts of unstructured data (text, binaries, etc.). Features include cost-optimized tiered storage, and integration with Microsoft’s Azure Data Lake Storage Gen2.
This is a built-in integration through the Cribl Stream Azure Blob Storage Source and Destination.
Azure Blob Storage as Source and Stream as a destination
Configure Azure Event Grid to send queue notifications when new blobs are added to a storage account.
Configure Stream to listen for Blob Storage data via Sources > Azure Blob Storage.
Specify the queue, connection string, Event Breaker settings, and optional parameters.
Stream will start fetching data as it becomes available.
Azure Blob Storage as Destination and LogStream as a source
Configure LogStream to send data to Azure Event Hubs via Destinations > Azure Blob Storage.
Specify the Azure container name, blob prefix, staging location, partitioning expression, backpressure behavior, and optional parameters.
LogStream will start sending data as it becomes available.
Analyze big data streams with a fast and powerful data exploration service to reduce time to insights by sending to Azure Data Explorer (ADX).
Destination
Azure Data Explorer (ADX) is a fully managed, high-performance, big data analytics platform that makes it easy to analyze high volumes of data in near real time. The Azure Data Explorer toolbox gives you an end-to-end solution for data ingestion, query, visualization, and management. Using Cribl allows routing all of your Cribl sources to ADX for further analysis.
Follow detailed instructions for Stream or Edge.
Configure App Registration in Azure Authentication via Client Secret or Certificate
Configure Stream / Edge to send data to ADX via Destinations > Azure > Azure Data Explorer
Configure sources to send data to your new destination either via Routes or QuickConnect.
Simple, secure, and scalable real-time ingestion of streaming data
Source
Azure Event Hubs offer big data streaming, along with a fully managed data ingestion service that’s simple, trusted, and scalable. Event Hubs can receive and process millions of events per second, and can supply data to any real-time analytics provider or batching/storage adapter. Other features include simple construction of dynamic data pipelines; geo-replication and disaster recovery; integration with Apache Kafka clients and applications; and usage-based pricing.
This is a built-in integration through the Cribl Stream Azure Event Hubs Source and/or Destination.
Azure Event Hubs as Source and Stream as a destination
Configure Stream to listen for Event Hubs data via Sources > Azure Event Hubs.
Specify the Event Hubs brokers, hub name, and optional SASL authentication parameters.
Stream will start fetching data as it becomes available.
Simple, secure, and scalable real-time ingestion of streaming data
Destination
Azure Event Hubs offer big data streaming, along with a fully managed data ingestion service that’s simple, trusted, and scalable. Event Hubs can receive and process millions of events per second, and can supply data to any real-time analytics provider or batching/storage adapter. Other features include simple construction of dynamic data pipelines; geo-replication and disaster recovery; integration with Apache Kafka clients and applications; and usage-based pricing.
This is a built-in integration through the Cribl Stream Azure Event Hubs Source and/or Destination.
Azure Event Hubs as Destination and Stream as a source
Configure Stream to send data to Azure Event Hubs via Destinations > Azure Event Hubs.
Specify the Event Hubs brokers, hub name, and other parameters.
Stream will start sending data as it becomes available.
Collect and organize log and performance data on Azure resources
Destination
Azure Monitor Logs collect and organize log and performance data from monitored resources, such as platform logs from Azure services, log and performance data from virtual machines’ agents, and usage and performance data from applications.
This is a built-in integration through the Cribl Stream Azure Monitor Logs Destination.
Configure Stream to send data to Azure Event Hubs via Destinations > Azure Monitor Logs.
Specify the Azure Log Analytics Workspace ID, Workspace Key, Log Type, backpressure behavior, and optional parameters.
Stream will start sending data as it becomes available.