Cribl Git

Leveraging Git for Cribl Stream Config: A Backup and Tracking Solution

August 3, 2023

Having your Cribl Stream instance connected to a remote git repo is a great way to have a backup of the cribl config. It also allows for easy tracking and viewing of all Cribl Stream config changes for improved accountability and auditing. Our Goal: Get Cribl configured with a remote Git repo and also configured with git signed commits. Git signed commits are a way of using cryptography to digitally add a signature to git commits. The receiver of the data can verify that the signature is authentic, and therefore must’ve come from the signatory and not another imposter. Our Requirements:

  • Git 2.34 or later which has signature verification
  • OpenSSH 8.2p1 or later (which has the SSH-keygen -Y sign) ability
  • SSH key with ED25519 recommended

Our Setup: Cribl Stream setup in distributed mode with enterprise license. Tested on Cribl version 4.1.3

You will need shell access to the Cribl leader node in order to complete the full setup with Git signed commits.

On the leader – as Cribl user create an SSH keypair using ssh-keygen

Add the SSH key into Github via the web interface (Settings → SSH/GPG keys → Authentication keys → keytype authentication key)

Add again the same key into GitHub via a web interface (settings → SSH/GPG keys → authentication keys – but choose signing key as the keytype.

Create a new private repo in Github via the web interface. Ensure your repo is private as there are sensitive items which are stored in this repo.

Next, configure Cribl Stream to use the new repository.

In Cribl Stream – global settings → Git settings → Remote → Fill out the remote URL and set auth type to SSH and add the SSH private key. In the below example, criblbren is the org and dbtest.git is the repository.

On the leader as cribl user run the following command:

ssh-keyscan -H github.com >> ~/.ssh/known_hosts

Note: The first time when setting up in Cribl you may need to set ssh strict host key checking toggle to be off. After successfully configuring be sure to re-enable the toggle on again.

In Crib GUI make a test configuration change and commit and then push to the remote Git repo. Changes will show up in Github after the push. At this point, the commits will be unsigned but this test confirms connectivity from the Cribl leader to Github. The Cribl .git configuration will be located in the directory on the leader where you installed Cribl for example /opt/cribl/.git/config.

Now as the cribl user on the leader host, edit the cribl install .git/config file and adjust name/email and add signing key settings. Set the signing key to the path to the ssh key you created. Also, set/add the commit stanza and set gpgsign = true. This tells git to sign your commits automatically and avoids having to use the -S flag when committing. You’ll also want to add the GPG stanza to have git use SSH for the commit signing.

Note: In the user section below name and email are your github.com account login/email.

cribl@dbtest:/opt/cribl/.git$ cat config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[user]
name = criblben
email = bmarcus@cribl.io
signingkey = /home/cribl/.ssh/id_ed25519.pub
[remote "origin"]
url = git@github.com:criblben/dbtest.git
fetch = +refs/heads/:refs/remotes/origin/
[branch "master"]
remote = origin
merge = refs/heads/master
[commit]
gpgsign = true

[gpg]
format = ssh

As a Cribl user on the leader in the Cribl install location,  run the commands git status and git push. Make a change in Cribl GUI – and commit within Cribl GUI on the leader. Then push to the remote repo in Cribl GUI. The change will show as signed in Github. The green “verified” label will show up. Clicking the green label will show the user and fingerprint key details.

There are other methods of using Github signed commits using GPG keys and S/MIME keys but I found the ssh keys easiest to work with. This tutorial was done with GitHub and some variations in approach may be applied to other systems such as Gitlab, Bitbucket, etc.

References/Helpful Links


 

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a generous free usage plan across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started. We also offer a hands-on Sandbox for those interested in how companies globally leverage our products for their data challenges.

.
Blog
Feature Image

Cribl Stream: Up To 47x More Efficient vs OpenTelemetry Collector

Read More
.
Blog
Feature Image

12 Ways We Sleighed Innovation This Year

Read More
.
Blog
Feature Image

Scaling Observability on a Budget with Cribl for State, Local, and Education

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?