Log Analytics tools are one of the most expensive categories of observability and security tooling, and costs are continuing to increase dramatically. According to an IDC-Seagate Report in 2018, global data stores are increasing at 61% per year. This is accompanied by an explosion of log data being collected and stored. With costs at large customers already in the 10s of millions each year, increasing at double-digits year over year means material costs in the millions.
Log Analytics system costs are a function of the volume of data they are ingesting and storing. Historically, the amount of infrastructure required for a log analysis system scaled linearly with the amount of data stored. Newer innovations separate storage and compute in these systems which allows for longer retention with less cost, but in every architecture, more data requires more compute to process the data at query time. Quite simply, more data, more cost.