Update: With the 4.2 release of Cribl Stream, we now support direct integration with Microsoft Sentinel. You can send events through Cribl Stream Microsoft Sentinel Destinations to the Microsoft Sentinel SIEM in Azure. Visit our Docs to learn how to get started.

—

As version 4.0.4,  we are excited to announce the capability of Cribl’s webhook to write to any destinations and APIs that require OAuth including Microsoft Sentinel. Cribl has long supported OAuth in many destinations through native integrations but with the enhanced Webhook we can now write to any destination that requires OAuth authentication.

A Curious Goat Explores Microsoft Sentinel

Let’s explore how we can use our enhanced Webhook to write to a destination that requires OAuth. Microsoft Sentinel is a cloud-native Security Information and Event Management (SIEM) service offered by Microsoft Azure. It provides intelligent security analytics and threat intelligence across the enterprise, allowing organizations to detect, prevent, and respond to cybersecurity threats.

Organizations often use the Azure Connected Machine Agent, Azure Monitoring Agent (Deprecated), and/or the Microsoft Data connectors to ingest data into Microsoft Sentinel. With the enhanced webhook support, we can use Cribl to post data directly to Microsoft Sentinel!

Table Shopping: Native vs Custom

An important concept within Microsoft Sentinel is the concept of Tables. The first type of table is a custom table. Microsoft Sentinel allows users to create custom tables, which are user-defined data sources that can be ingested into Sentinel for analysis and correlation with other data sources. Custom tables can be written too within Cribl as well but will require customization of content and queries.

Native tables, on the other hand, are predefined data sources that come with Sentinel out of the box. These tables include information about security events, alerts, incidents, and other security-related data that is generated by the built-in connectors and integrations. The AI and machine learning threat and alerting capabilities of Sentinel are based on native tables and they also provide many useful out-of-the-box queries and dashboards designed to work specifically for Native tables enabling users to see rapid value realization.

For the purposes of this guide, we’ll be attempting to write to native tables.

Limitations

Now before we start clicking buttons let’s cover some of the limitations.

• Native Table Limitations – Cribl can write to any custom table but can only write to four native tables which are:
  • Syslog
  • SecurityEvent
  • WindowsEvent
  • CommonSecurityLog
• Data throughput: The maximum size of both a compressed and uncompressed API call is 1 MB and the maximum data per minute of a Data Collection rule is 2 GB.

Sending That Data Goodness to Microsoft Sentinel

Let’s have some fun and write to some native tables within Microsoft Sentinel!

Step 1: Configuring a REST endpoint in Microsoft Sentinel

Before we start, we first need to make some configurations within Azure that allow us to send data to Azure via their API. A rock star goat and Senior Solutions Cribl Engineer named Sidd Shah cracked the code on how to write to Microsoft Sentinel using their REST endpoint and was kind enough to publish some fantastic documentation on how to do this here: When you return, you should have the following information:

• Log Ingestion URL.
• Client/Application ID.
• Client Secret Value.
• DCR (Data Collection Rule) ID.

Step 2: Configuring the Rest API

Now that you’ve got the correct information. Let’s set up a new webhook destination so we can start writing data to Microsoft Sentinel.

We can configure a webhook within both Cribl Stream or Edge by going to: Data -> Destinations -> Webhook -> New Destination.

Under General settings:

1. Output ID: Give it a name! Can be anything in my example, I want to write to the Microsoft Window Event table so we’ll give it a name “microsoft-sentinel-syslog”.
2. URL: Input the URL you created from Step 1. This URL should be in the format:

https:///dataCollectionRules//streams/Custom-?api-version=2021-11-01-preview

In a full example this is what mine looks like this:

https://kevinbrunettesentinel-j0cf.eastus-1.ingest.monitor.azure.com/dataCollectionRules/dcr-d7f6236184d84394a932aab4eb42d986/streams/Custom-Syslog?api-version=2021-11-01-preview

Step 3: Configure the Webhook Oauth Authentication

Now let’s configure the Authentication piece with our information from Step 1 for Microsoft Sentinel. Let’s take a look at a picture of the fields and walk through the important fields.

1. Authentication Type: OAuth
2. Login URL: For Microsoft Sentinel, the URL will be in the following format: https://login.microsoftonline.com/%3Ctenant-id%3E/oauth2/v2.0/token. Make sure to paste in your tenant-id replacing the value obtained in Step 1.
3. OAuth Secret: This is our secret configuration from Step 1 of the guide. Once populated, by default it is obfuscated and can be seen by clicking the “eye” icon on the right.
4. OAuth Secret parameter name: The meat and potatoes of OAuth, this is the secret parameter that is sent in the request.
5. Token attribute name: This is the value of the key token return in the authorization response. In the case of Microsoft Sentinel, it is called: “access_token”.
6. Authorize expression: How the endpoint expects the authorization token.
7. Refresh interval: How long to refresh the token? The default is 3600, but it is a good idea to set it to 3000.
8. OAuth Parameters: The parameters to send. Additional parameters and headers can be sent by clicking “+Add Header” or “+Add Parameter”. Sentinel needs 3 parameters defined below:
   1. scope = Provides the amount of access. In our case, we will use: https://monitor.azure.com//.default
   2. client_id = This is the application/client ID within Azure obtained from Step 1.
   3. grant_type = For Microsoft Sentinel we need “client_crendentials”. OAuth supports a variety of different OAuth types.

Step 3: Commit and Deploy!

Once saved, make sure to hit that “Commit and Deploy” button. Now we’ve got a fully functioning destination that can write to Microsoft Sentinel.

Step 4: Test It

Whew! We’ve clicked a lot of buttons, and gone through a ton of steps and pages but how do we know this thing works? Luckily for us, Cribl lets us test this endpoint before we add it to a pipeline. To do this, click on the Webhook we just configured and deployed called: microsoft-sentinel-native-syslog.

This webhook we configured is designed to specifically write to https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/syslog.

Sample Syslog Payload:

Within our Webhook we can now post this payload into the “Test” input tab, paste in our above payload, and hit “Run Test”.

If all goes well, we should see a “Success” message.

If we log in to Azure and navigate to our Sentinel Logs workspace. We can now use Kql to search for our Syslog payload. We’ll do a search for the host:

Syslog

| where Computer == "CriblStreamWorker"

As you can see, we have successfully confirmed that we can send a Syslog payload to the native tables within Microsoft Sentinel!

Additional Sentinel Data Samples

In the configuration, we wrote to the native Azure Syslog table. What about writing to the other tables? We’ve got you covered. Here are a few other examples.

Microsoft Security Event

Below is a format on how to send a Microsoft security event.

URL

To do this, we’ll need to either create or alter our existing Webhook making sure to alter the table name within the URL. To write to Microsoft Security Event, it will look like this:

https://kevinbrunettesentinel-j0cf.eastus-1.ingest.monitor.azure.com/dataCollectionRules/dcr-d7f6236184d84394a932aab4eb42d986/streams/Custom-SecurityEvent?api-version=2021-11-01-preview

Sample Payload:

Using Cribl pipelines, we’ve transformed a Windows security event into a payload pretty close to what Microsoft Sentinel expects within the Security Event native table.

Verifying Data Ingest:

I’ve got loads of dating coming in from my AMA agent but not to worry! Using Kql I can search for my specific host where my Security event came from.

Cribl Microsoft Windows Events Pack

A beta of the capabilities within Cribl to convert Windows Event data from the Cribl Edge Windows Event forwarder is available here: https://github.com/bronette/cribl-edge-microsoft-sentinel-windows-events. This pack does NOT support the Cribl Stream Windows Event Forwarder.

CommonSecurityLog

Here’s a sample of CEF to CommonSecurityLog.

Configuring the Webhook

We will need to either configure a new webhook or alter an existing webhook making sure to add the “Custom-CommonSecurityLog” within the URL.
https://kevinbrunettesentinel-j0cf.eastus-1.ingest.monitor.azure.com/dataCollectionRules/dcr-d7f6236184d84394a932aab4eb42d986/streams/Custom-CommonSecurityLog?api-version=2021-11-01-preview

Example Payload:

Here’s a sample Palo Alto traffic-based CEF log. Palo Alto Networks has been kind enough to provide a sample on their website here: https://docs.paloaltonetworks.com/cortex/cortex-data-lake/log-forwarding-schema-reference/network-logs/network-traffic-log/network-traffic-cef-fields which we will use in our example.

Using the power of Cribl pipeline we can format the CEF messages into Microsoft’s CommonSecurityLog format which Microsoft has provided the mappings here.

{
	"ProfileToken": "xxxxx",
	"LogSeverity": 3,
	"Computer": "PA-5220-Sensor",
	"DeviceFacility": "user",
	"DeviceVendor": "Palo Alto Networks",
	"DeviceProduct": "LF",
	"DeviceVersion": 2,
	"DeviceTimeZone": "UTC",
	"ReceiptTime": "Mar 16 2023 20:16:21",
	"DeviceExternalID": "xxxxxxxxxxxxx",
	"DestinationNTDomain": "paloaltonetwork",
	"DestinationUserName": "paloaltonetwork\\xxxxx",
	"DestinationUserId": 0,
	"SourceNTDomain": "xxxxx",
	"SourceUserName": "xxxxx\\xxxxx xxxxx",
	"SourceUserID": 0,
	"SourceIP": "xxx.xx.x.xx",
	"DestinationIP": "xxx.xx.x.xx",
	"SourceTranslatedAddress": "xxx.xx.x.xx",
	"DestinationTranslatedAddress": "xxx.xx.x.xx",
	"DeviceCustomString1": "deny-attackers",
	"DeviceCustomString1Label": "Rule",
	"ApplicationProtocol": "fileguri",
	"DeviceCustomString3": "vsys1",
	"DeviceCustomString3Label": "VirtualLocation",
	"DeviceCustomString4": "untrust",
	"DeviceCustomString4Label": "FromZone",
	"DeviceCustomString5": "ethernet4Zone-test1",
	"DeviceCustomString5Label": "ToZone",
	"DeviceInboundInterface": "unknown",
	"DeviceOutboundInterface": "unknown",
	"DeviceCustomString6": "rs-logging",
	"DeviceCustomString6Label": "LogSetting",
	"DeviceCustomNumber1": 25596,
	"DeviceCustomNumber1Label": "SessionID",
	"EventCount": 1,
	"SourcePort": 22871,
	"DestinationPort": 27092,
	"SourceTranslatedPort": 24429,
	"DestinationTranslatedPort": 14744,
	"Protocol": "tcp",
	"DeviceAction": "deny",
	"SentBytes": 400448,
	"ReceivedBytes": 969846,
	"DeviceCustomNumber2": 314,
	"DeviceCustomNumber2Label": "PacketsTotal",
	"DeviceCustomNumber3": 56,
	"DeviceCustomNumber3Label": "SessionDuration",
	"DeviceCustomString2": "custom-category",
	"DeviceCustomString2Label": "URLCategory",
	"ExternalID": "xxxxxxxxxxxxx",
	"Reason": "unknown",
	"DeviceName": "xxxxx",
	"DeviceEventCategory": "unknown",
	"AdditionalEvents": "PanOSApplicationContainer=;PanOSApplicationRisk=5;PanOSApplicationSubcategory=file-sharing;PanOSApplicationTechnology=peer-to-peer;PanOSCaptivePortal=false;PanOSCortexDataLakeTenantID=xxxxxxxxxxxxx;PanOSDestinationDeviceClass=;PanOSDestinationDeviceOS=;PanOSInboundInterfaceDetailsPort=0;PanOSInboundInterfaceDetailsSlot=0;PanOSInboundInterfaceDetailsType=unknown;PanOSInboundInterfaceDetailsUnit=0;PanOSIsClienttoServer=false;PanOSIsContainer=false;PanOSIsDecryptMirror=false;PanOSIsDecrypted=false;PanOSIsDecryptedLog=false;PanOSIsDecryptedPayloadForward=false;PanOSIsDuplicateLog=false;PanOSIsEncrypted=false;PanOSIsIPV6=false;PanOSIsInspectionBeforeSession=true;PanOSIsMptcpOn=false;PanOSIsNonStandardDestinationPort=false;PanOSIsPacketCapture=false;PanOSIsPhishing=false;PanOSIsPrismaNetwork=false;PanOSIsPrismaUsers=false;PanOSIsProxy=false;PanOSIsReconExcluded=false;PanOSIsSaaSApplication=false;PanOSIsServertoClient=false;PanOSIsSourceXForwarded=false;PanOSIsSystemReturn=false;PanOSIsTransaction=false;PanOSIsTunnelInspected=false;PanOSIsURLDenied=false;PanOSLogExported=false;PanOSLogForwarded=true;PanOSLogSource=firewall;PanOSLogSourceTimeZoneOffset=;PanOSNAT=false;PanOSNonStandardDestinationPort=0;PanOSOutboundInterfaceDetailsPort=0;PanOSOutboundInterfaceDetailsSlot=0;PanOSOutboundInterfaceDetailsType=unknown;PanOSOutboundInterfaceDetailsUnit=0;PanOSSDWANFECRatio=0.0;PanOSSanctionedStateOfApp=false;PanOSSessionOwnerMidx=false;PanOSSessionTracker=16;PanOSSourceDeviceClass=;PanOSSourceDeviceOS=;PanOSTunneledApplication=tunneled-app;PanOSUsers=xxxxx\\xxxxx xxxxx;PanOSVirtualSystemID=1;PanOSApplicationCategory=peer2peer;PanOSConfigVersion=10.0;PanOSBytes=1370294;PanOSSessionStartTime=Mar 16 2023 20:15:48;PanOSSourceLocation=east-coast;PanOSDestinationLocation=BR;PanOSPacketsSent=194;PanOSPacketsReceived=120;PanOSDGHierarchyLevel1=11;PanOSDGHierarchyLevel2=0;PanOSDGHierarchyLevel3=0;PanOSDGHierarchyLevel4=0;PanOSVirtualSystemName=;PanOSSourceUUID=;PanOSDestinationUUID=;PanOSIMSI=0;PanOSIMEI=;PanOSParentSessionID=0;PanOSParentStarttime=Mar 16 2023 20:15:40;PanOSTunnel=GRE;PanOSEndpointAssociationID=-3746994889972252628;PanOSChunksTotal=1945;PanOSChunksSent=323;PanOSChunksReceived=1622;PanOSRuleUUID=017e4d76-2003-47f4-8afc-1d35c808c615;PanOSHTTP2Connection=469139;PanOSLinkChangeCount=0;PanOSSDWANPolicyName=;PanOSLinkSwitches=;PanOSSDWANCluster=;PanOSSDWANDeviceType=;PanOSSDWANClusterType=;PanOSSDWANSite=;PanOSDynamicUserGroupName=dynug-4;PanOSX-Forwarded-ForIP=xxx.xx.x.xx;PanOSSourceDeviceCategory=N-Phone;PanOSSourceDeviceProfile=n-profile;PanOSSourceDeviceModel=Nexus;PanOSSourceDeviceVendor=Google;PanOSSourceDeviceOSFamily=LG-H790;PanOSSourceDeviceOSVersion=Android v6;PanOSSourceDeviceHost=pan-301;PanOSSourceDeviceMac=839147449905;PanOSDestinationDeviceCategory=N-Phone;PanOSDestinationDeviceProfile=n-profile;PanOSDestinationDeviceModel=Nexus;PanOSDestinationDeviceVendor=Google;PanOSDestinationDeviceOSFamily=H1511;PanOSDestinationDeviceOSVersion=Android v7;PanOSDestinationDeviceHost=pan-355;PanOSDestinationDeviceMac=530589561221;PanOSContainerID=1873cc5c-0d31;PanOSContainerNameSpace=pns_default;PanOSContainerName=pan-dp-77754f4;PanOSSourceEDL=;PanOSDestinationEDL=;PanOSGPHostID=xxxxxxxxxxxxxx;PanOSEndpointSerialNumber=xxxxxxxxxxxxxx;PanOSSourceDynamicAddressGroup=aqua_dag;PanOSDestinationDynamicAddressGroup=;PanOSHASessionOwner=session_owner-4;PanOSTimeGeneratedHighResolution=Feb 27 2021 20:16:18;PanOSNSSAINetworkSliceType=0;PanOSNSSAINetworkSliceDifferentiator=1bca5;",
	"TimeGenerated": "2023-03-16T20:46:50.000Z",
	"Type": "CommonSecurityLog"
}

Verifying Data Ingest:

To verify data ingest we can do a search for a custom device label to confirm that we are indeed receiving the events from Cribl.

Cribl Microsoft Sentinel CEF to CommonSecurityLog Pack

Special thanks to another fantastic Goat Solutions Engineer Derek Gleich who assisted in helping me write this pack. The Beta pack for CEF to Common Security Log can be found here.

What’s Next?

The goats here are continually working to improve and support more data types within Sentinel. Stay tuned for native integration and more improvements and additions to the Microsoft Sentinel Cribl packs.

The fastest way to get started with Cribl Stream, Edge, and Search is to try the Free Cloud Sandboxes.