Security teams often find it difficult to make the most of their existing security tooling and identify threats targeting their specific organization. Yet, it goes without saying that – when looking to consolidate security investments and reduce operating costs – it is important to leverage Security Information & Event Management (SIEM) technology rather than adding another tool on top that will carry out the same functionality.
This can be a challenge for security teams that want to create rules for automatically identifying Indicators of Compromise (IOCs) specific to their organizations, because threat data is frequently ingested in the wrong way or coming from the wrong sources. For example, threat data going into the SIEM often comes from open sources that do not provide a comprehensive picture of threat actor activity, and this results in high volumes of false positives.
At the same time, security teams that opt to layer a threat intelligence platform on top of the SIEM, need to properly configure both platforms so that they complement each other without creating more noise.
CyberProof leverages customer-specific IOC lists by ingesting them into Microsoft's Sentinel SIEM, augmenting the SIEM's predefined rules and capabilities, and providing our customers with automated and dynamically updated threat detection.
To illustrate how to overcome these issues, we would like to provide you with a real-life example of how CyberProof approaches this challenge. CyberProof leverages customer-specific IOC lists by ingesting them into Microsoft’s Sentinel SIEM, augmenting the SIEM’s predefined rules and capabilities, and providing our customers with automated and dynamically updated threat detection.
Using Analytic Rules to Search for Specific IOCs
Once customer data is in Log Analytics, we apply an Analytic Rule. Analytic Rules are rules that can be created within Azure Sentinel using their native KQL query language to define the logic for automating detection of threats. Within the Analytics Rule Wizard, you can set the parameters, the type of data that forms part of an alert (such as IP addresses or host names, etc.) and schedule queries to run at certain times.
This rule searches for specific IOCs in the customer environment. In a single query, we can search for IOCs in the form of either malicious IP addresses, domains, URLs, or hashes.
In the screenshot below, you can see how CyberProof created an Analytic Rule that provides our team with the capability of searching for IOCs based on CTI research, across customer data – all in real time.
The Analytic Rule searches for a match between the customer data and the dedicated IOCs provided by CyberProof.
The first component of the Analytic Rule is an external lookup table that is dynamically updated as incoming data is ingested. The data, which is ingested from both external open-source and proprietary platforms, is collected by the CyberProof CTI team and our security analysts.
Once an alert is pulled into the CDC, automatic remediation actions take place - based on customized playbooks and with the help of CyberProof's SeeMo AI bot -to further enrich, remediate and respond to the alluded threat.
Each customer has specific IOCs inside the lookup table that include the following elements:
- Indicator – An IP address, domain name/address, URL or unique hash key.
- Campaign – The attack’s campaign name.
- Category – The attack type: spam, phishing, brute-force, PII, etc.
A comparison is run between the data in the external lookup tables and the customer data ingested into Log Analytics. The results are displayed as an alert in Sentinel. The CyberProof Defense Center (CDC) pulls those events, and automatic actions are initiated accordingly. Once an alert is pulled into the CDC, automatic remediation actions take place – based on customized playbooks and with the help of CyberProof’s SeeMo AI bot – to further enrich, remediate and respond to the alluded threat.
Figure 1: CyberProof workflow
Querying Specific IOCs with Microsoft’s KQL Language
In this section, we’ll explain how to query IOCs based on IP addresses using CyberProof’s proprietary IOCs list. The same process can be adjusted and repeated to query network domains, URLs or hash keys.
In general, the analytic rule makes use of the list of IP addresses from the proprietary CyberProof IOC list – let’s call this the “external data” – to search for matches in the customer’s data from their various Log Analytics tables. Later on, we’ll explain how this information is then converted into a user-friendly table with all the relevant IOC information.
First, we use a Let statement to call out the IPs from the external IOC list (sourced in a CSV file), and basically extract the Indicator field (source IP address):
Because we need to convert the table to a list, we need to use the make_list operator.
We use an additional Let statement to create the summary list using the ToScalar operator. It calls the previously defined IP (defined in the first Let statement). This newly created list is compared to the external list of IOCs:
We use the union isfuzzy command to start searching across the different customer data tables and compare them. Using the isfuzzy command prevents the query from failing, even in the case of a failure in a specific section of the query – since it is run on each query separately.
Next, each customer table is compared to the external lookup table. In the following example code line, the Palo Alto table (PaloAlto_CL) is compared to the IP_IOCs lookup table:
The different customer tables are likely to hold different IP field names. We’ll need to use the Extend operator for each log source table, in order to normalize the IP field naming convention for the search run, and to create the columns at run time.
For example, the Palo Alto naming convention for IP addresses is Source_IP_s, while the OfficeActivity naming convention for IP addresses is Client_IP – and so on. All of the different IP fields are inserted into a single field called: normalizedIP.
The same process is followed for the other fields: domains, URLs, hash keys, etc.
Continuing from IP address fields to the network domain fields, we show (in the following example) how the Mimecast mail relay extracts the domain field from the sender field. We then run a comparison between the Mimecast and the CyberProof-supplied Domain IOC:
Now we’ll need to compare the different log tables using a join command. Only the rows that need to be merged are pulled, i.e., those with matches between entities from the customer table and the CyberProof-supplied IOCs:
Finally, we use the summarize operator to aggregate the content of the matching results and print out a customized IOC table that includes the IOC, which is matched to the customer entity, along with Type, Indicator, Campaign and Category fields, and a counter field indicating the number of times the particular row entity was encountered:
In addition to the IP address query we just described here, additional lookups, such as those of the Hash, Domain and URL, are also queried. The Sentinel displays all the query results in a table format with columns including the originating IOC indicator, the log table that lists the malicious indicator, the indicator’s campaign name, and the number of hits counted.
Information gained and arranged in this process significantly enhances CyberProof’s ability to quickly alert customers about ongoing incidents in a relatively short period of time after their initial exposure.
Delegating Access to IOC Data with SAS Tokens
External lookups are based on using Blob Containers, which are publicly available and need to be secured. This is done by generating SAS (Shared Access Signature) tokens. Once Tokens are generated, we need to add these Token to each lookup. The lookups can be wrapped with a function that makes them easier to view. Tokens can be also rotated as part of the CI/CD Process. When generating a SAS token, we need to specify the start and end dates and what the allowed permissions are for the account SAS.
Conclusion
As described in this blog, CyberProof developed an innovative solution that fully integrates with the Azure Security stack.
The CyberProof Log Collection is a SaaS-based service that runs on top of the Azure Kubernetes Service, which enables the solution to be both highly scalable and resilient.
CyberProof pulls security log sources from the various customer data sources – on-premises, Cloud, and SaaS – into a single environment, using CyberProof Log Collection (CLC) technology.
Once we have this information, we continue to parse, normalize, and take the data. The CLC helps us ingest the data into the customer’s Log Analytics; and each customer has its own dedicated environment. The CLC is a SaaS-based service that runs on top of the Azure Kubernetes Service, and leverages cloud resources, which enables the solution to be both highly scalable and resilient.
