From the first commit to the LogStream codebase, we intended to launch LogStream as a service. Early on, there was even a debate as to whether to launch the SaaS version first. The conventional wisdom is to build all new products in the cloud, and the market is certainly trending in that direction. However, just because something works in one market does not mean it’ll translate to another. In logging, for high scale customers, they needed to be able to process the data as near to where it originates as possible in order to solve their #1 problem: cost. In order to best serve our customers, we decided to go to market first with a downloadable product.

We have seen tremendous adoption of LogStream, by customers such as Autodesk, TransUnion, Blue Voyant, and more. Half of our customers today are running LogStream in cloud services inside their VPCs. Autodesk uses LogStream to easily route machine data to one of multiple SaaS services. TransUnion gets massive savings by enriching data and filtering based on smart context. Blue Voyant centralizes enrichment, reshaping and filtering logic for their customers in one central pipeline, eliminating massive operational complexity. Our largest customers are processing hundreds of terabytes of data per day, where processing the data near where it originates avoids potentially millions of dollars in network egress charges.

LogStream is the easiest distributed system to set up we’ve ever used, but we’re not resting there. Today, we’re making it even easier, with the beta release of LogStream Cloud. LogStream Cloud is the easiest way to get data into your favorite SaaS services. LogStream Cloud is a fully hosted offering of LogStream as a service, available with affordable on-demand pricing so you pay for only what you consume.

For the first 25 beta customers, we’re offering LogStream Cloud free for 6 months, until April 15th, 2021. LogStream Cloud is currently for customers processing under 5 terabytes of data per day. Our ideal prospects are looking for a no-code experience for easy onboarding and reshaping of large volumes of machine data before sending on to one or more data stores like Splunk Cloud, Elastic Cloud, Snowflake, AWS Elasticsearch Service, AWS S3, Confluent Cloud, and many others. To learn more about LogStream Cloud and to learn more about pricing, read on.

Why LogStream Cloud?

LogStream Cloud helps customers affordably onboard data to their favorite cloud services. LogStream Cloud makes it easier than ever to get data from APIs, Event Streaming, or existing agents into a cloud solution, with just the right data in the right shape. LogStream Cloud is accessible with an easy-to-use no-code experience for transforming even complex, ugly nested data structures common in machine data while enriching and filtering down to the most valuable subset of the stream.

LogStream pays for itself by making it easy to filter out noise from chatty logs. LogStream customers report being able to filter out 80%+ of events from sources like AWS CloudTrail, or easily converting logs like AWS CloudFront access logs to metrics for dashboarding rather than ingesting full fidelity logs to their log solution. Customers also use LogStream to dynamically sample Kubernetes logs and bring in a good representative sample of what are usually very repetitive logs.

LogStream Cloud also gives customers control over their data and makes it easy to store full fidelity copies in cheap object storage for easy replay later. LogStream allows customers to reshape and filter with confidence they can always get back to the original raw data if they so need. With an on-demand pricing model, running large batch jobs only requires paying for the resources consumed.

LogStream Cloud supports all the different sources and destinations you need. There are a ton of use cases for LogStream Cloud: 

• Accept a webhook, enrich against lookup tables, and dump the data in Snowflake
• Collect from Office 365 activity APIs and send to Splunk Cloud
• Replay raw log data from a S3 bucket, parse and map to Elastic Common Schema, and send to Elastic Cloud instance for an investigation

And many more! LogStream Cloud makes it easy and affordable to integrate any API, Event Stream, Log Shipper, or Metrics Agent to any destination supported by LogStream. Now with LogStream Cloud, we provide a managed service priced purely on consumption.

Pricing

LogStream Cloud is priced on a consumption model. There are 5 dimensions that drive price for LogStream Cloud: Base, GB In, GB Out, Egress Passthrough, and S3 Out. Walking through these:

• Base: $1/Hr.
  • Covers master node and the base infrastructure for the cluster.
• GB In: $.05/GB
  • Raw data, metered by the Gigabyte, coming into LogStream
• GB Out: $.10/GB
  • Raw data, metered by the Gigabyte, leaving LogStream
• Egress Passthrough:
  • Charges incurred by the cloud provider for egressing data
  • Charged as bytes on the wire, so compression advised
  • Provisioning LogStream Cloud in the same region as the destination is advisable
• S3: Free
  • Data written to S3 is at no charge.

What this means in practice is that bringing data in, but filtering it out, is only charged for the ingest. Data sent to S3 is free, making it very cost effective to create a full archive of all data. Egress is passed through rather than hidden, as customer configurations can drive wildly different egress charges from AWS.

We want LogStream Cloud to be easy to buy. Our goal with our pricing is to make it a very low commitment to try out your first use case, and the product will scale with you as you grow. 

Early Adopter Program

We’re making LogStream Cloud available in limited release today. Our early adopter program is opening today and will run until April 15th for 25 customers. LogStream Cloud may become generally available before that date. For the duration of our early adopter program, LogStream will be free, with capacity limited to below 5TB/Day. 

We are looking for prospects which meet the following criteria:

• At least one production use case in mind
• In AWS, preferably in regions us-east-1 or us-west-2.
• Below 5TB/Day of Ingest

If you think you fit that, admittedly very wide, profile, then we’d love to speak with you! Fill out our signup form and we’ll get back to you. Additional information about LogStream Cloud is available on our website.

Wrapping Up

Cribl is a customers-first company. LogStream Cloud is changing how we are able to engage with our customers, with lower commitment and consumption pricing which is directly aligned to usage. LogStream Cloud is the easiest and most cost effective way to onboard data to your SaaS logging tools, metric databases, and cloud data warehouses. 

We’re excited to see the new use cases our customers are going to come up with. We’d love to hear your feedback. Please join our community slack and drop us a line, or simply send us an email at hello@cribl.io. 

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a generous free usage plan across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started. We also offer a hands-on Sandbox for those interested in how companies globally leverage our products for their data challenges.