Recorded Future for Splunk

Installation and setup

Installing the Recorded Future for SplunkThe app is available at SplunkBase. It can either be installed directly from SplunkBase ordownloaded and installed that way.

WARNING

This app is not compatible with the previous app and add-on from RecordedFuture. The following must be removed from the system before installing theapp:

• Recorded Future app for Splunk Enterprise (TA_recordedfuture-cyber)

• Recorded Future add-on for Splunk ES (TA-recorded_future)

The app is intended to run on Splunk servers with the search head role. It can be installed onSearch Head clusters (see below) and on search heads connected to index clusters.

Once installed the app must be setup.

Installing on a Search Head Cluster1. Download the package into $SPLUNK_HOME/etc/shcluster/apps on the deployer of the Search Head

Cluster.

2. Unpack the package:

Table of ContentsInstallation and setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

Installing the Recorded Future for Splunk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

Installing on a Search Head Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

Initial Setup of the App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  2

Setup Splunk Enterprise Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  8

Technical description of the App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  9

Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  9

Functionality specific to Enterprise Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  11

Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12

Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12

How to Use the Reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12

Raising an issue with Recorded Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  14

1

tar zxvfp recordedfuture_app_for_splunk_XXX.tgz

3. Remove the package file:

rm recorded-future_app_for_splunk_XXX.tgz

4. Push the new app to the Cluster nodes:

splunk apply shcluster-bundle ...

5. Connect to any Search Head Cluster node and ensure that the configuration is set with thecorrect API Credential. The App will ensure that configuration is propagated to all nodes in thecluster.

The Recorded Future App for Splunk will detect when running in a Search Head Cluster to ensurethat only one node, the captain, retrieves the Risk Lists and the alerts.

Initial Setup of the AppWhen the app has been installed on the Splunk server, finalise the initial setup undermenu:Configuration[Configuration].

NOTEThe app is not compatible with the old integrations from Recorded Future(Recorded Future App for Splunk Enterprise and Add-on for Splunk ES). These mustbe removed from the Splunk system.

The Configuration view has three panes:

• Setup

• Risk Lists

• Alerting Rules

Configure the API credential with Connect API access and a working API endpoint under the Setupview.

The API credential must be configured in the Setup pane in order for the app to work. A userneeds the capability of 'list_storage_passwords' to configure API Credential and Proxy settings inthe App.

Setup

2

Figure 1. Setup tab of the Configuration view

API Credential

All the settings except for Risk Lists and Alerting Rules are configured under the Setup tab. Theminimum possible configuration is setting the API Credential here.

NOTE The minimum required configuration is adding the API Credential.

SSL Verification

If you need to disable certificate validation, for example if a networking device in between theSplunk machine and the internet is modifying the SSL certificate, this can be done by uncheckingthe "SSL Verification" checkbox.

Proxy

If the splunk server requires a proxy for Internet access, the "Proxy" checkbox should be checked.This will reveal new fields that need to be filled in. The username and password should only beconfigured if the proxy requires authentication. Proxy host and port are required settings.

API URL

Recorded Future support may in rare circumstances instruct a user to use a different URL to theRecorded Future API, in which case the "Recorded Future API URL" should be modified.

3

Log level

There are five levels of logging: CRITICAL, ERROR, WARNING, INFO and DEBUG.

The recommended log level is INFO. To report an issue with the Recorded Future App, temporarilychange it to DEBUG to collect additional logs that can be used for trouble shooting.

The logs generated by the Recorded Future App are located in the default Splunk log directory$SPLUNK_HOME/var/log/splunk and will be written to the following file:

• ta_recordedfuture_rest.log

The information contained in the log files can be viewed either in the Splunk GUI or as files on theSplunk server.

Example search:

index=_* source="/opt/splunk/var/log/splunk/ta_recordedfuture_rest.log"

More information about troubleshooting is available in <Troubleshooting>

Configure and Manage Risk Lists

Risk Lists can be used to correlate and enrich events. Each element in a Risk List, like an IP numberor Domain, contains has a risk score and the information which contributed to its risk score.

Default Risk Lists

The Recorded Future App for Splunk is shipped with the five Recorded Future Risk Lists:

• IP address

• Domain names

• URLs

• Hashes of files

• Vulnerabilities (mainly CVEs)

With Fusion access, it is possible to setup additional customized Risk Lists.

Add Risk Lists

4

Figure 2. The Risk Lists tab in the Configuration view

Additional Risk Lists can be downloaded by clicking "Add Risk List".

The following fields appear at the top:

Field Significance Comment

Name Risk List name within theSplunk instance.

The lookup file will be named<name>.csv.

Risk List category The type of entity contained inthe Risk List.

IP, Domain, Hash, Vulnerability,or URL.

Fusion file The path to the Fusion Risk List. The path must point to adefined Fusion file stored as anuncompressed CSV file if usedas a lookup.

Update Interval The interval used to check forupdates.

Default is as soon as an updatedversion is available.

When done configuring the new Risk List, click on btn:[Save] to save the new configuration.

Manage Risk Lists

All configured Risk Lists are listed under menu:Configuration[Configuration > Risk Lists].

The list of Risk List inputs is sorted to show any custom Risk Lists at the top and the defaultconfiguration at the bottom. The default Risk List inputs can not be deleted, only disabled.

5

To edit a configured Risk List, just click on btn[Edit] and the fields will unlock. Click btn:[Save]when done editing the settings.

To remove a Risk List, select the corresponding btn:[Delete Risk List] checkbox and click onbtn:[Save].

Figure 3. Risk Lists tab in the Configuration view

Alert Monitoring Setup

There is no default configuration for alert monitoring. Alert monitoring is configured inmenu:Configuration[Configuration > Alerting Rules].

When monitoring alerts, the Recorded Future App will poll the Recorded Future API for alertswhich match the configured criteria.

By default, the alerts are fetched on the fly when needed by a dashboard.

Add Alert Monitoring

To add alert monitoring, click on btn:[Add Alerting Rule] and select the Alerting Rule to fetch alertsfrom. The following fields then appear:

Alert Rules tab in the Configuration view

6

Field Significance Comment

Name Alerting Rule name Name of the Alerting Rule input.

Alert Status Matches any alert status by default The filter can be configured asneeded.

Time Range Filters on the timestamp of the alert. Default is anytime. The notation is thesame as in the Recorded Future portal.

Ex:

• "-2d to now"

• "-2h to -1h"

• "yesterday"

There are a few common choicesavailable as a dropdown.

Limit Amount of alerts to fetch Default is 10. This should be adjusteddepending on the amount of alertsthat trigger for this rule.

Alerting Rule Which alerting rule to fetch This is the rule that you selected whencreating the Alerting Rule input.

7

Manage Alerting Rules

To edit a configured Alerting Rule, just click on btn:[Edit] and the fields will unlock. Click btn:[Save]when done editing the settings.

To remove an Alerting Rule, select the corresponding btn:[Delete Alerting Rule] checkbox and clickon btn:[Save].

Setup Splunk Enterprise SecurityThe app has built-in support for Splunk Enterprise Security. The support is available when the appis installed on a search head together with Splunk ES.

Enable Splunk ES support

In the Recorded Future for Splunk menu, select Configure. Ensure that the switch to enable supportfor Splunk ES is enabled.

Required configuration within Splunk ES

To be able to use the full features of Splunk ES functionality, some configuration has to be done inSplunk Enterprise Security.

• In the Enterprise Security menu bar, click menu:Configure[Incident Management > IncidentReview Settings].

• Click the btn:[Add new entry] in the "Incident Review - Event Attributes" section. Add thefollowing Label and Field Combinations:

Label Field

RF Risk Score rf_a_risk

RF Triggered Rules rf_b_rules

RF Very Malicious Evidence rf_evidence_critical

RF Malicious Evidence rf_evidence_malicious

RF Suspicious Evidence rf_evidence_suspicious

RF Unusual Evidence rf_evidence_unusual

• A restart of the Splunk instance will be required once the installation has completed.

• If you haven’t already done so, enable the Enterprise Security correlation search called "ThreatActivity Detected"

1. In the Enterprise Security menu bar, click menu:Configure[Content Management]

2. In the filter bar, type "Threat Activity Detected"

3. Click btn:[Enable] to enable the correlation search

8

Enrichment of detected events

Splunk ES detects suspicious events using it’s built-in Threat Intelligence framework. RecordedFuture leverages the framework to perform detection of suspicious events.

Once an event has been detected it is however necessary to enrich it to make triage efficient. Thiscan be done in two ways:

1. Using saved searches which adds data from Recorded Future’s Risk Lists to the events.

2. Using the provided Adaptive Response action. This method makes a query to Recorded Future’sAPI to fetch up-to-date information.

See below for instructions on how to activate respective method.

Saved Searches to perform Enrichment

By default the app will enable four saved searches that will perform the enrichment of anycompatible notable events. See below for the steps needed to switch to Adaptive Response basedEnrichment.

Adaptive Response (AR) to perform Enrichment

To activate Adaptive Response (AR) the following steps needs to be performed:

• Turn off the searches that enrich notable events:

1. Go to menu:Configure[Content Management]

2. Disable "RF IP Threatlist Search", "RF Domain Threatlist Search" and "RF Hash ThreatlistSearch" (easier to find if you use the app filter, but not necessary).

• Click on "Threat Activity Detected" to open the settings.

1. Next to "Adaptive Response Action", click btn:[Add New Response Action]

2. Select Recorded Future’s action

3. Leave default "Automatic" selection.

• Click save

Adaptive Response Ad-hoc invocation

Ad-hoc invocations of Adaptive Response can be made - ex from the Incident Review dashboard.The user invoking the Adaptive Response in this way must have the list_storage_passwordscapability.

Technical description of the App

FunctionalityThe App provides three major functions:

9

1. Threat detection support

2. Alert triage support

3. Makes Alerts from Recorded Future available on the Splunk system

All functionality is implemented with saved searches which in some cases call the App’s customREST handler.

Threat Detection (Correlation)

Threat Detection is implemented by correlating selected log sources with lookup files available tothe Splunk system. The App makes a number of lookup files available, their content corresponds toRecorded Future Risk Lists.

Recorded Future Risk Lists are CSV files (columns are Name, Risk, RiskString and EvidenceString).For each configured Risk List the App will monitor the API for updates and whenever an update isavailable this will be downloaded and stored in the lookup folder within the app.

The saved search "Recorded Future - Download Risk Lists" is scheduled to run every 5 minutes.During each run the following steps are performed:

1. Configuration for the Risk Lists is checked.

2. For each configured Risk List the "Update Interval" is checked to determine if the Risk List canbe updated. The default is to always check.

3. If the Risk List is to be checked a call to the Recorded Future API is done using the HTTP HEADmethod. The response contains a checksum for Risk List available at the API. This checksum iscompared with the checksum of the Risk List stored locally.

4. If the checksum differ the Risk List is updated. The custom REST handler creates and executes asearch on the Splunk system:

a. The Risk List is downloaded from Recorded Future’s API using a dedicated endpoint withinthe handler.

b. The content is fed to an outputlookup command which stores the data as a lookup file.

c. The new checksum for the local copy of the Risk List is calculated and stored as acheckpoint.

Alert Triage (Enrichment)

The Alert Triage is implemented via a number of Enrichment dashboards:

• IP Enrichment

• Domain Enrichment

• Hash Enrichment

• Vulnerability Enrichment

• Malware Enrichment

• URL Enrichment

10

Each dashboard takes an entity and fetches information about it by making a call the App’s customREST handler. The handler makes a REST call to Recorded Future’s API and adapts the response tomake it easier to render. The results are rendered in the dashboard.

Alerts from Recorded Future

Alerts from Recorded Future are shown using a dashboard. From a drop-down menu any of theconfigured Alerts configurations can be selected.

1. When a configuration has been selected the dashboard makes a REST call to the App’s customREST handler requesting alerts matching the configuration.

2. The REST handler makes a REST call to Recorded Future’s API and fetches all alerts matchingthe search criteria. Some adaptations are made to the returned data before it is returned.

3. The dashboard renders the alerts.

If more details is required there is a separate endpoint withing the handler that can fetch allavailable info for a given alert.

Functionality specific to Enterprise SecurityThe App will detect if it is running on a search head with Splunk Enterprise Security (ES) installed.If this is the case an additional configuration setting is activated which enables or disables the ESspecific functions. The rest of this section describes what those are.

Threat Intelligence Framework integration

Splunk ES includes the Threat Intelligence framework (TI) which is a very efficient method to docorrelation between events and threat information. TI support detection on IPs, domains, URLs andhashes.

If the ES support is activated an additional step is done whenever a Risk List is updated. This stepcalls an endpoint within the custom REST handler which convert the list into a format optimizedfor TI. The list in TI is then updated with the contents of the new Risk List (old data is alsoremoved). Only entries added by the App is touched.

Enrichment of Notable Events

If the Splunk ES installation has been configured to promote events detected by the ThreatIntelligence framework (TI) to Notable Events these can be viewd in the "Incident review"dashboard.

TI does not support additional information about why an entity is dangerous. The app provides twomethods to add this information to relevant Notable Events.

Saved search to enrich Notable Events

The App offers four saved searches (one per entity type supported by TI) that can enrich Notableevents.

11

When active the searches will look for any Notable Event created using information from RecordedFuture. When one is found a new Notable Event is created which contains the Risk Score, thenumber of triggered Risk Rules and a list of which Risk Rules that have triggered for the entity.

By updating the displayed fields in the Incident Review configuration panel (which is a setup stepduring installation of the app on an ES system) these new fields a displayed whenever a Notableevent is reviewed.

Adaptive Response to enrich Notable Events

The App offers an Adaptive Response action which can add the same information as the savedsearches do above. The main difference is that this method makes a call via an endpoint in thecustom REST handler to Recorded Future’s API to fetch up-to-date information (the saved searchuses information from the stored Risk List).

Troubleshooting

TroubleshootingThe types of issues involving the Recorded Future App for Splunk can be divided into threecategories. The Recorded Future App contains three reports, one for each category, to assisttroubleshooting:

Category Report name Purpose

Credential/Network Validate app deployment This report displays the resultof a number of tests andlookups that is performed whenthe report is run.

Risk List Download/Frequency Latest updates of all Risk Lists This report show the last 5 RiskList updates.

Other All logs from the App This report displays all the logsproduced by the app in oneview.

How to Use the Reports

Check configuration/network connectivity

Run the report "Validate App Deployment" when the Recorded Future App for Splunk has beendeployed and configured or as an initial step during troubleshooting. The built-in validatorperforms a number of tests and collects useful troubleshooting information. "Ok" and "NA"indicates that the App’s connectivity/setup is working so anything else, ie "Warning" or "Error",should be investigated.

12

Figure 4. Validation Report

Verify that Risk Lists are downloaded correctly

The Recorded Future Risk Lists are available from the Recorded Future API. The report "LatestUpdate of all Risk Lists" shows all Risk Lists that have been downloaded successfully. We save thetimestamps from the last 5 successful downloads. Any Risk Lists not shown in the report have neverbeen downloaded successfully.

The recommended update frequency of the Recorded Future Risk Lists depends on how often theyare updated. The current schedule can be found on the Recorded Future Support site.

There are several issues that can impact the download of a Risk List. Follow the following guide totroubleshoot Risk Lists which that are not updated as expected:

1. If all Risk Lists fail to be updated, it is likely that there is an issue with network connectivity orthe API Credential used. Run the report "Validate app deployment" described above.

2. Check that the configuration specifies the correct interval for updates on the configurationpage.

3. The Fusion path may not exist or it was spelled wrong. This can be verified by performing thefollowing search:

index=_* sourcetype="tarecordedfuture:log" ERROR 404 "File or directory" path=*

4. Check that the path field corresponds to a Fusion file. Note that it is URL encoded which meansthat the Fusion file path /home/custom.csv will read %2Fhome%2Fcustom.csv.

5. Ensure that the Recorded Future API Credential used by the app belongs to the correctenterprise in Recorded Future’s system. With the exception of public Fusion files (paths startingwith /public/), no Fusion files are available outside of the Enterprise that owns them.

6. Ensure that the Fusion Flow responsible for generating the Fusion file was successfullyexecuted.

Other issues

The report "All logs from the App" lists all the events created by the app. The log level can beadjusted in the "Configuration" page. Default is INFO but when troubleshooting it may be

13

appropriate to increase the level to DEBUG.

A good starting place is to look for errors (loglevel ERROR). The report can be opened in the searchview: select menu:Open in Search[] via the btn:[Edit] button.

Raising an issue with Recorded FutureWhen reporting an issue to Recorded Future, the following procedure will generate a good set ofinformation for further analysis:

1. Short summary of the issue: what is or is not happening? Is it happening all the time or is itintermittent or limited to a subset of entities?

2. Take screenshots showing the results of reports "Validate App Deployment" and "Latest Updateof all Risk Lists".

3. Increase the log level to DEBUG

4. Trigger the issue.

5. Note the date and time the issue was triggered. Make sure to include this into the report toRecorded Future.

6. Run the report "All logs from the App" and export the results as a CSV file.

7. Reset the log level.

14