How to flatten nested events

In this quick How To post we’ll cover how to flatten a nested event, either entirely or just parts of it. We’ll cover two flavors of flattening:

1. Creating top level fields with fully qualified names, e.g {foo: {bar: 42}} -> {foo_bar: 42}

2. Promoting a nested object’s field to top level, e.g. {foo: {bar: 42}} -> {bar: 42}

Let’s assume that we’re working with an event that looks like this (internal LogStream log) and we want to flatten or promote the contents of opts

Flattening

We recommend using the flatten function which is purpose built for this use-case. It allows you to specify an optional prefix as well as what delimiter to use when creating fully qualified field names. For many destination systems it is challenging to use fields containing a . thus the default delimiter is set to _

Promoting

We recommend using the Eval function in combination with JavaScript’s Object.assign function to copy fields from the nested object to the top level object, then finally remove the nested object. In this case we’re referencing a special field called __e which references the entire event.  We’re effectively copying fields from opts to the top level, then deleting opts field. You will notice that this method does not recursively promote nested objects (notice metricStorage), you’d have to manually do that, using another statement in the eval.

 If you’ve enjoyed reading this far please take Cribl LogStream for a spin and join our community Slack  where lot’s more cool use-cases like this come up frequently.