June 3, 2022
Git integration has always been at the foundation of Stream. In the fall 2021 release of Cribl Stream (both on-prem software and Cloud), our Enterprise users have a received set of APIs to separate the development and deployment of Stream. Stream GitOps connects with your favorite git based versioning platforms and leverages their PR, approve/reject, and CI/CD workflows to push production-ready changes from a development branch into a main branch or release. We know how important it is to ensure your production Steam environment only receives updates and changes that have met your internal processes and reviews.
In this blog post, we’ll walk through the steps to configure Cribl Stream integration with a GitHub repository. These steps enable a continuous integration/continuous delivery between a Cribl development environment and a Cribl production environment. A great background to the subject is described in a previous blog post
Note – Before we start: Make sure you have Cribl Enterprise License in both the production and development instances of Cribl
ssh-keygen -t ed25519 -C "firstname.lastname@example.org"
/root/.ssh/id_ed25519(private – will go to Cribl Stream) and id_ed25519.pub (public – will go to GitHub)
ssh-keyscan -H github.com >> ~/.ssh/known_hosts
On the GitHub UI -> Cribl repository, you will see the Cribl Stream configurations in your ‘ master ‘ branch
An alternative to SSH authentication is to use Username / Password (Token) authentication.
First, generate GitHub Token. In your GitHub Account -> Settings -> Developer Settings -> Personal Access Token -> Generate a new Token
Second, In your Cribl Stream production environment, go to Settings > Git Settings
In your Cribl Stream Production instance, reload your Leader node UI. On the Git Setting -> General -> Branch, you should be able to see 3 options (dev, master, production) branches.
Select ‘production‘ as the branch
Select ‘Push‘ for the GitOps workflow. After you save your changes, your Cribl production environment becomes a Read Only mode
In the Cribl Stream Development environment, add a new Route to test our end to end:
In the Cribl Stream Development environment, Select ‘ Commit ‘ and ‘ Git Push ‘. This will send the configurations to GitHub dev branch
local -> pipeline -> route.yml)
In your Cribl Stream production CLI run the below command
curl -X POST "http://<leader URL or IP>:9000/api/v1/version/sync" -H "accept: application/json" -H "Authorization: Bearer <bearer token>" -d "ref=<production branch>&deploy=true"
[root@localhost prod]# curl -X POST "http://192.168.56.253:9000/api/v1/version/sync" -H "accept: application/json" -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE2NDYzOTc0MDcsImV4cCI6MTY0NjQwMTAwNywidXNlcm5hbWUiOiJhZG1pbiIsInJvbGVzIjpbImFkbWluIl19.W21hm0cvKFelIgfyYQ4vPYakNKFawh-PaVI8agyhIYI" -d "ref=production&deploy=true"
In the Cribl Stream Production environment, we can see the Route from the Cribl Stream development environment
In this post, we showed how Cribl Stream integration with GitHub can help you move your Routes, Pipelines, and Pack from the development environment to the production environment. For a more in-depth look at GitOps in Cribl, check out the recent Cribl Office Hours below.
The fastest way to get started with Cribl Stream and Cribl Edge is to try the Free Cloud Sandboxes.