Embed Size (px)
Citation preview
© All rights reserved.
Contents Introduction....................................................................................................................................................................... 3
Creating User Account .................................................................................................................................................. 4
Windows ............................................................................................................................................................................. 5
Install Comae Toolkit................................................................................................................................................. 5
Create and Upload Dump Files ............................................................................................................................. 7
Create Dump Files using DumpIt ......................................................................................................................... 7
Upload the Dump Files to Stardust ..................................................................................................................... 9
Create and upload Snapshot Files .................................................................................................................... 10
Create the Snapshot ............................................................................................................................................... 10
Upload the Snapshot Files ................................................................................................................................... 12
Convert a Dump File to a Snapshot.................................................................................................................. 13
Combining Conversion and Upload ................................................................................................................. 13
Managing Machines and Snapshots ................................................................................................................ 14
Machine Memory’s State Acquisition .............................................................................................................. 14
Comae Snapshot Pre-Processing ...................................................................................................................... 15
Linux .................................................................................................................................................................................. 16
Send Memory Snapshot to Comae Stardust ................................................................................................. 16
Send Full Memory Image to Comae Stardust............................................................................................... 17
Send Memory Dump to Google Cloud Platform ......................................................................................... 18
Send Memory Dump to Azure Storage ........................................................................................................... 21
Send Memory Dump to Amazon Web Services S3 .................................................................................... 22
Comae Stardust ............................................................................................................................................................ 24
Drag and Drop Files on Stardust Dashboard ................................................................................................ 24
Managing Uploads Using Tasks ......................................................................................................................... 25
Aggregating Machine Data ................................................................................................................................. 26
Machine Classification ........................................................................................................................................... 27
Support ............................................................................................................................................................................ 28
About Comae Technologies ..................................................................................................................................... 29
© All rights reserved.
Introduction
This User Guide provides direction on the use of the Comae Stardust system. Stardust allows users
and system administrators to further protect their systems and networks through memory-based
analytics. This guide shows how to capture on-demand and scheduled snapshots of data and
identify threats.
Using this guide, users will learn how to look for possible threats, thus providing each organization
with important intelligence on how to protect resources.
Proactively analyzing memory dumps saves organizations from preventable downtime, wasted
administrative time, and unnecessary expense.
© All rights reserved.
Creating User Account To analyze dumps, an account must first be registered on the Stardust system. Each user must be
registered under their respective organization and be verified using a valid email address.
As shown in Screenshot 2-1, go to https://my.comae.com and register an account using a valid
work email address.
Screenshot 2-1 Comae Stardust Registration
Comae Support sends an email requesting confirmation of the registered email address. If an
email is not received in a reasonable amount of time, check the spam folder or simply add
[email protected] to your contacts or address book. Direct questions to [email protected] and
receive a prompt response.
© All rights reserved.
Windows Acquire and send Windows Machine’s full memory image or snapshot to Comae Stardust.
Install Comae Toolkit
Once the registered email is successfully confirmed, use it to log into Stardust and display the
Stardust Dashboard, as depicted in Screenshot 3-1 below.
Screenshot 3-1 Stardust Dashboard
Each user must download and install the Comae-Toolkit.
• Click on Download Comae Toolkit to navigate to Utilities screen.
© All rights reserved.
• From the Utilities page, click on Download Comae-Toolkit-<version>.zip as depicted in
Screenshot 3-2.
Screenshot 3-2 Comae-Toolkit Download
Once clicked, a compressed (zip) file downloads to your machine.
• After the download is complete, navigate to the folder where it resides (typically the
Downloads folder) and extract the contents of the compressed file to the default location
or one of your choosing.
A folder named Comae-Toolkit-<version> is created with a license file, readme file and
two folders containing executables for both x86- and x64-based operating systems.
Note: System Type information is located on Windows machines in Control Panel/System. Most
modern Windows machines can run either type, however.
If the Windows requires the Comae.ps1 file to be unblocked, do so by right-clicking the file,
selecting Properties, and click Unblock button.
Important Callout: The following instructions require a basic understanding on the use of the
PowerShell utility in order to use the Comae Stardust tool and related commands.
Open a Windows PowerShell session as an administrator to begin the process.
• Type PowerShell in the Start | Search Programs and Files command box and, from the
programs list, right-click PowerShell and select Run as Administrator.
• From the PowerShell session, navigate to the folder that contains the Comae.ps1 file.
• To access different Stardust commands from within the PowerShell, run the Import-Module
cmdlet to import the Comae module:
Import-Module .\Comae.ps1
© All rights reserved.
If successful no errors are displayed and an empty command prompt is shown ready to accept
more commands. Success can also be verified by running the following command to see the
signature of the New-ComaeDumpFile cmdlet:
New-ComaeDumpFile -?
The following displays if the Comae.ps1 installed successfully:
New-ComaeDumpFile [-Directory] <string> [-IsCompress]
Important Callout: The Comae.ps1 module must be installed each time a new PowerShell session
is initiated: it does not remain installed once the PowerShell console is closed.
Create and Upload Dump Files
Dump files are created from the user’s system and sent to Stardust for examination for
unauthorized use and/or illegal activity. There are several different methods to create these files:
1) Executing Dumpit command created by Comae Stardust.
2) Taking a snapshot of memory metadata.
3) Designating a startup date/time within a Windows task startup.
If not already open, open Windows PowerShell session to begin the process for creating a dump.
• Type PowerShell in the Start | Search Programs and Files command box and, from the
programs list, right-click PowerShell and select Run as Administrator.
Create Dump Files using DumpIt
Dump files are the exact copy of the entire memory state of a machine as a Microsoft Crash Dump.
They are generated on the fly by the Comae DumpIt utility.
The full signature of the DumpIt command was previously provided. This section focuses on the
Directory parameter and related value and the IsCompress param.
The Directory parameter tells the cmdlet what directory to deposit the 2 files that are generated
as part of its output. The directory is created by the command if it doesn’t already exist.
IsCompress compresses the output crash dump in an internal format created specifically by Comae
Stardust to support large files e.g. 100Gb. The file extension is zdmp instead of dmp.
Execute the New-ComaeDumpFile command from the Powershell session:
New-ComaeDumpFile -Directory “C:\Comae-CrashDumps” -IsCompress
© All rights reserved.
The cmdlet takes a few minutes to complete its analysis and create a dmp and json files (See
screenshot 4-1)
Screenshot 4-1
© All rights reserved.
Upload the Dump Files to Stardust
Once the dump files are created, they need to be uploaded to the remote Stardust system for
pattern analysis. The dump file can be somewhat large and is compressed as a part of the Send
command. The full signature of the Send-ComaeDumpFile command is as follows:
Send-ComaeDumpFile [-Key] <string> [-Path] <string> [-ItemType]
<string> [-IsCompress]
The following cmdlet parameters are in scope to send one or both files to Stardust:
- Key parameter us the user access token generated through the platform to enable the
use of the API.
- Path parameter is the input file or directory as indicated by the ItemType parameter.
- ItemType parameter can be either File or Directory.
Note: To retrieve the Key value, run the Get-ComaeAPIKey command with the -ClientId and -
ClientSecret params with the respective values that can be found in your Stardust account in
Settings page > Integrations tab.
$APIKey = Get-ComaeAPIKey [-ClientId] <string> [-ClientSecret] <string>
Note: The IsCompress parameter is also available for use in the Send-ComaeDumpFile if not
previously used when executing the New-ComaeDumpFile command.
• From the PowerShell session, execute the Send-ComaeDumpFile cmdlet with the
following parameters, based on preference.
Send only the compressed dump file:
Send-ComaeDumpFile -Key $APIKey -Path “C:\ComaeCrashDumps\FileName.zdmp” -ItemType “File”
Create the crash dump into the provided directory before sending it to the server:
Send-ComaeDumpFile -Key $APIKey -Path “C:\Comae-CrashDumps” -ItemType “Directory”
© All rights reserved.
Screenshot 4-2 shows the stdout from running the Send-ComaeDumpFile command.
Screenshot 4-2
Note: For added privacy, instead of sending full memory dumps to Stardust, the metadata archive
(compressed .json files) cam be sent. Typically used for hybrid-cloud models, the memory dump
is pre-processed locally instead of relying completely on the Stardust platform for analysis.
Create and upload Snapshot Files
Snapshots are the extracted metadata from dump files. They are referred to as Comae snapshot
archives.
Create a snapshot from a running machine using the live parameter of Dmp2Json program and
/L option of Comae DupmIt.
• If not already open, open a Windows PowerShell session to begin the process for creating
a dump.
o Type PowerShell in the Start | Search Programs and Files command box and, from
the programs list, right-click PowerShell and select Run as Administrator.
Create the Snapshot
New-ComaeSnapshot simulates a live mode and generates the metadata directly. Using this
command prevents the need to re-run analysis in the future as it doesn’t archive a copy of the
physical memory. The full signature of the New-ComaeSnapshot command is as follows:
New-ComaeSnapshot [-Directory] <string>
The following cmdlet parameters are in scope to create the Snapshot:
- Directory parameter is the output directory.
From the PowerShell session execute the New-ComaeSnapshot cmdlet with the Directory
parameter:
New-ComaeSnapshot -Directory “C:\Comae-Snapshots”
© All rights reserved.
Screenshots 5-1 and 5-2 show the stdout from the New-ComaeSnapshot command. The
command takes a while to complete.
Screenshot 5-1
Screenshot 5-2
© All rights reserved.
As seen in Screenshot 5-3, several directories and files are created by running the New-
ComaeSnapshot cmdlet.
Screenshot 5-3
Upload the Snapshot Files
Once the Snapshot completes the output needs to be uploaded to the remote Stardust system
for analysis. The files can be somewhat large and is compressed as part of the Send command.
The full signature of the Send-ComaeSnapshot command is as follows:
Send-ComaeSnapshot [-Key] <string> [-Path] <string> [-ItemType]
<string>
Parameters:
- Key parameter is the user access token generated through the platform to able the use
of the API
- Path is the input file or directory given the ItemType value.
- ItemType can either be File or Directory
From the PowerShell session, execute the Send-ComaeSnapshot cmdlet with the following
parameters, based on preference.
Send only the json file:
Send-ComaeSnapshot -Key $APIKey -Path “C:\Comae-Snapshots\FileNmae.json.zip” -ItemType “File”
Create the snapshot in the provided directory before sending it to the server.
Send-ComaeSnapshot -Key $APIKey -Path “C:\Comae-Snapshots” -ItemType “Directory”
© All rights reserved.
Note: For added privacy, instead of sending full memory of sending full memory snapshot to
Stardust, the metadata archive (compressed .json files) can be sent. Typically used for hybrid-
cloud models, the snapshot is pre-processed locally instead of relying completely on the Stardust
platform for analysis.
Convert a Dump File to a Snapshot
The Convert-DumpFileToSnapshot cmdlet converts a Microsoft crash dump file into a Comae
Snapshot using the Dmp2Json program. The full signature of the Convert-DumpFileToSnapshot
command is as follows:
Convert-DumpFileToSnapshot [-FilePath] <string> [-Directory]
<string> [[-SymbolPath] <string>] [[-SymbolServer] <string>]
In scope parameters:
- FilePath parameters is the input Microsoft crash dump file.
- Directory is the output directory where the Comae snapshot archive will be located.
- SymbolPath (optional) is the input directory for pre-downloaded Microsoft PDB symbols.
- SymbolServer (optional) is the input server address for scenarios with custom symbol
servers.
Convert-DumpFileToSnapshot -FilePath “TEST-MEMORY.dmp” -
Directory “C:\Comae-Snapshots”
In certain cases, user may want to provide a custom symbol directory path, or a custom symbol
server path.
Convert-DumpFileToSnapshot -FilePath “TEST-MEMORY.dmp” -
Directory “C:\Comae-Snapshots” -SymbolPath “C:\Symbols” -
SymbolServer https://msdl.microsoft.com/downloads/symbols
Combining Conversion and Upload
Using the above commands, user can combine the conversion & upload procedures for multiple
files within a given folder.
Get-ChildItem -Path “C:\DumpFiles” -File | ForEach-Object
{ Convert-DumpFileToSnapshot -FilePath $_.FullName -Directory
“C:\Snaphots” }
Get-ChildItem -Path “C:\Snapshots” -File | ForEach-Object
{ Send-ComaeSnapshot -Key $APIKey -Path $_.FullName -ItemType “File” }
© All rights reserved.
Managing Machines and Snapshots
The Stardust platform manages the uploaded snapshots and the information contained within the
files generated by the PowerShell commands.
User can upload the following the Stardust platform:
- Microsoft crash dump files uncompressed or compressed (Only Zip archives are supported)
- Comae snapshot archives (smaller, pre-processed by Mem2Json)
o This often used in hybrid cloud scenarios where the user prefers to keep a copy of crash
dump files in local storage rather than in the cloud.
Machine Memory’s State Acquisition
Running the Comae DumpIt utility with the /Q (for quiet) option is used to automatically answer
confirmation prompts, such as Proceed with the acquisition? [y/n], when running memory
acquisition in a script (see Screenshot 6-1).
Screenshot 6-1
Windows Scheduled Tasks can be setup to run the DumpIt program as a time-based utility and
generate a historical record of machine activity. Doing so enables retro-hunting investigations.
© All rights reserved.
Comae Snapshot Pre-Processing
The output directory can be manually specified to either a local folder or a remote file share if
user preference id to manage their own local memory copy.
If pre-processing crash dump files locally to generate a Comae snapshot archive instead of
sending an entire memory copy to the Stardust platform is preferred, the Dmp2Json program is
used to perform the pre-processing. The command is run outside the Powershell using DOS
command session.
Dmp2Json.exe /Y
srv*C:Symbols*http://msdl.microsoft/download/symbols /Z
C:FileName.dmp /c “/all /datetime /archive /snapshot
C:\Snapshots\Snapshot”
© All rights reserved.
Linux DumpItForLinux can be used conveniently in Linux machines using its containerized version. You
just need to install Docker in your machine.
Send Memory Snapshot to Comae Stardust
Update the apt repository and install the latest version of Docker to be able to run the free
containerized version of DumpItForLinux.
sudo apt-get update
sudo apt install docker.io
Run the DumpItForLinux command using docker with “--snap-it” and “--action upload-comae”
flags with your Comae Stardust credentials.
sudo docker run --privileged comaeio/dumpit-linux --snap-it --comae-client-id
<Client ID> --comae-client-secret <Secret ID> --action upload-comae
DumpItForLinux will send the pre-processed data to Comae Stardust.
© All rights reserved.
Important Callout: Client ID and Secret ID can be found when you log in into your Stardust
account under Settings > Integrations.
Send Full Memory Image to Comae Stardust
Update the apt repository and install the latest version of Docker.
sudo apt-get update
sudo apt install docker.io
Run the DumpItForLinux command using docker with “--dump-it” and “--action upload-comae”
flags.
sudo docker run --privileged comaeio/dumpit-linux --dump-it --comae-client-id
<Client ID> --comae-client-secret <Secret ID> --action upload-comae
DumpItForLinux will send a full memory image to Comae Stardust.
© All rights reserved.
Important Callout: Client ID and Secret ID can be found when you log in to your Stardust account
under Settings > Integrations.
Send Memory Dump to Google Cloud Platform
To be able to interact with the Google Cloud Platform through DumpItForLinux, you will need a
service account and a credential file in JSON format.
Please check the official documentation for service accounts and credential files in this link:
https://cloud.google.com/iam/docs/creating-managing-service-account-keys
You can optionally generate and download the credential file using gcloud CLI commands.
Inside the CLI, log in to your GCP account.
gcloud auth login
You will be prompted with a link to authenticate you as a GCP user. Open that link, login
with your GCP account and copy the code provided. Paste it in the console to finish the
authentication process.
Set the GCP project you are working on by using the following command.
gcloud config set project [PROJECT_ID]
Create a service account.
gcloud iam service-accounts create [YOUR_SERVICE_ACCOUNT_NAME]
gcloud projects add-iam-policy-binding [PROJECT_ID] --member “serviceAccount:
[YOUR_SERVICE_ACCOUNT_NAME]@[PROJECT_ID].iam.gserviceaccount.com” --role
“roles/owner”
© All rights reserved.
Create a service account key.
gcloud iam service-accounts keys create /tmp/[FILE_NAME].json --iam-account
[YOUR_SERVICEACCOUNT_NAME]@[PROJECT_ID].iam.gserviceaccount.com
Important Callout: A bucket should be created in your GCP Storage before running Docker
command for DumpItForLinux.
Install the latest version of Docker to be able to run the free containerized version of
DumpItForLinux.
sudo apt install docker.io
Run the DumpItForLinux commands using docker with “--snap-it” and “--action upload-gcp” flag.
You need to provide the path to the json file that contains your service account key and the bucket
name.
sudo docker run -v /tmp/[FILE_NAME].json:/tmp/[FILE_NAME].json --privileged
comaeio/dumpit-linux --snap-it --action upload-gcp --gcp-creds-file
/tmp/[FILE_NAME].json --bucket [BUCKET_NAME]
© All rights reserved.
DumpItForLinux will upload the preprocessed data to your specified GCP Storage bucket.
To upload a full memory image to GCP Storage, replace the “--snap-it” flag with “--dump-it” using
the same docker command.
© All rights reserved.
Send Memory Dump to Azure Storage
You will need your Storage account’s Storage Account Name and Storage Account Key. Both can
be found when you log in to your Azure account in Storage accounts > [Your-Storage-Account] >
Access Keys.
Inside your Ubuntu instance, update the apt repository and install the latest version of Docker.
sudo apt-get update
sudo apt install docker.io
Run the DumpItForLinux commands using docker with “--dump-it” and “--action upload-az” with
your Azure Storage credentials and bucket name.
sudo docker run --privileged comaeio/dumpit-linux --dump-it --actiion upload-
az --bucket [BUCKET_NAME] --az-account-name [STORAGE_ACCOUNT_NAME] --az-
account-key [STORAGE-ACCOUNT_KEY]
DumpItForLinux will upload the full memory image data to your Azure Storage bucket.
© All rights reserved.
To upload the snapshot of the memory to Azure Storage, replace the “--dump-it” flag with “--
snap-it” using the same docker command.
Send Memory Dump to Amazon Web Services S3
Log in to your AWS account and in IAM > Users page, add AmazonS3FullAccess policy in the
Permissions tab. You also need the user’s Access Key Id and Access Key Secret. You can create these
credentials in the Security credentials tab if you haven’t done yet. A bucket is also required, you
can use your existing bucket or create a new one in your S3. Just make sure the bucket exists
before running the DumpItForLinux command.
Inside your Ubuntu instance, update the apt repository and install the latest version of Docker.
sudo apt-get update
sudo apt install docker.io
Run the DumpItForLinux commands using docker with “--dump-it” and “--action upload-s3” with
your AWS User credentials and bucket name.
sudo docker run --privileged comaeio/dumpit-linux --dump-it --action upload-
s3 --bucket [BUCKET_NAME] --aws-access-id [ACCESS_KEY_ID] --aws-access-secret
[ACCESS_KEY_SECRET]
© All rights reserved.
DumpItForLinux will upload the full memory image data to your AWS S3 bucket.
To upload the snapshot of the memory to AWS S3, replace the “--dump-it” flag with “--snap-it”
using the same docker command.
© All rights reserved.
Comae Stardust
Drag and Drop Files on Stardust Dashboard
As an alternative to using Send commands in PowerShell, crash dumps and/or snapshot files can
be copied to the platform interface by simply dragging the files from the local machine to the
upload area of the Dashboard page on the website (see Screenshot 6-2).
Screenshot 6-2
© All rights reserved.
Managing Uploads Using Tasks
The progress of the latest operations of uploaded files are tracked within the tasks tab. (see
Screenshot 6-3 and 6-4)
Screenshot 6-3
Screenshot 6-4
© All rights reserved.
Aggregating Machine Data
Crash dump files or snapshots are automatically aggregated within the same machine. (see
Screenshot 6-5)
Screenshot 6-5 Machine Data Aggregation
Each machine can then be selected to see Snapshot details at different points in time. (see
Screenshot 6-6)
Screenshot 6-6 Machine Snapshot Detail
© All rights reserved.
Machine Classification
Browsing a machine content is now possible for each archived machine on the Stardust platform.
The platform automatically analyzes and classifies the snapshot as Clean, Suspicious, or Malicious.
In case of suspicious snapshots, the user can verify results and explore data directly. (see
Screenshot 6-7)
Screenshot 6-7 Machine Classification
© All rights reserved.
Support [email protected] for questions or inquiries.
Platform address https://my.comae.com
Github (PowerShell interface): https://github.com/comaeio/comae-cli
© All rights reserved.
About Comae Technologies Comae Technologies is a privately-owned global cybersecurity firm that provides organizations
with high-end security research services and in-memory threat-hunting capabilities through its
product Comae Stardust. Founded in 2016 by cybersecurity researcher Matt Suiche, Comae is also
recognized by the financial industry as of the pioneer companies in the field of BlockChain Security
and globally for its pro-active response to WannaCry & NotPetya during 2017 ransomware
epidemics.
