Click here to load reader

AQL Flow and Event Query CLI Guide - Juniper Networks Networks, Inc. 1194 North Mathilda Avenue Sunnyvale, CA 94089 USA 408-745-2000 Published: 2010-11-16 Security Threat Response

  • View
    222

  • Download
    5

Embed Size (px)

Text of AQL Flow and Event Query CLI Guide - Juniper Networks Networks, Inc. 1194 North Mathilda Avenue...

  • Security Threat Response Manager

    AQL Flow and Event Query CLI Guide

    Release 2010.0

    Juniper Networks, Inc.1194 North Mathilda AvenueSunnyvale, CA 94089USA408-745-2000

    www.juniper.net

    Published: 2010-11-16

  • 2

    Copyright NoticeCopyright 2010 Juniper Networks, Inc. All rights reserved. Juniper Networks and the Juniper Networks logo are registered trademarks of Juniper Networks Inc. in the United States and other countries. All other trademarks, service marks, registered trademarks, or registered service marks in this document are the property of Juniper Networks or their respective owners. All specifications are subject to change without notice. Juniper Networks assumes no responsibility for any inaccuracies in this document or for any obligation to update information in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.

    FCC StatementThe following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. The equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense. The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with NetScreens installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause harmful interference to radio or television reception, which can be determined by turning the equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures: Reorient or relocate the receiving antenna. Increase the separation between the equipment and receiver. Consult the dealer or an experienced radio/TV technician for help. Connect the equipment to an outlet on a circuit different from that to which the receiver is connected.

    Caution: Changes or modifications to this product could void the user's warranty and authority to operate this device.

    DisclaimerTHE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR JUNIPER NETWORKS REPRESENTATIVE FOR A COPY.

    AQL Flow and Event Query CLI GuideRelease 2010.0

    Copyright 2010, Juniper Networks, Inc.

    All rights reserved. Printed in USA.

    Revision History

    November 2010R2 AQL Flow and Event Query CLI Guide

    The information in this document is current as of the date listed in the revision history.

  • CONTENTS

    ABOUT THIS GUIDEConventions 1Technical Documentation 1Contacting Customer Support 2

    1 USING THE AQL QUERY CLIAbout the AQL Query CLI 3Accessing the AQL Query CLI 4Using a Select Statement 5Using Where Clauses 10Using the Group By Clause 11Using the Order By Clause 12Using the Count(*) Clause 12Using the Distinct Clause 12Using the Count (Distinct ...) Clause 13Using the Materialize View Clause 13Using the Like Clause 14Using the Boolean Clause 14Identifying AQL Fields 15Listing AQL Fields 16

  • ABOUT THIS GUIDE

    The AQL Event and Flow Query CLI Guide provides you with information for using the AQL CLI. This guide assumes you have advanced knowledge of Linux command line functionality.

    Conventions Table 1 lists conventions that are used throughout this guide.

    Technical Documentation

    You can access technical documentation, technical notes, and release notes directly from the Juniper customer support web site at www.juniper.net/support/. Once you access the Juniper customer support web site, locate the product and software release for which you require documentation.

    Your comments are important to us. Please send your e-mail comments about this guide or any of the Juniper Networks documentation to:[email protected]

    Include the following information with your comments:

    Document title Page number

    Table 1 Icons

    Icon Type DescriptionInformation note Information that describes important features or

    instructions.

    Caution Information that alerts you to potential loss of data or potential damage to an application, system, device, or network.

    Warning Information that alerts you to potential personal injury.

    AQL Event and Flow Query CLI Guide

  • 2 ABOUT THIS GUIDE

    Contacting Customer Support

    To help resolve any issues that you may encounter when installing or maintaining STRM, you can contact Customer Support as follows:

    Open a support case using the Case Management link at www.juniper.net/support/.

    Call 1-888-314-JTAC (from the United States, Canada, or Mexico) or1-408-745-9500 (from elsewhere).

    AQL Event and Flow Query CLI Guide

  • 1

    USING THE AQL QUERY CLI

    You can use the AQL Event and Flow Query Command Line Interface (CLI) to access flows and events stored in the Ariel database. This document provides information on accessing and using the AQL query CLI including:

    About the AQL Query CLI Accessing the AQL Query CLI Using a Select Statement Using Where Clauses Using the Group By Clause Using the Order By Clause Using the Count(*) Clause Using the Distinct Clause Using the Count (Distinct ...) Clause Using the Materialize View Clause Using the Like Clause Using the Boolean Clause Identifying AQL Fields Listing AQL Fields

    About the AQL Query CLI

    The AQL Event and Flow QueryCLI allows you to access raw flows and events stored in the Ariel database. The AQL query CLI includes syntax that is a subset of the SQL92 standard and provides support for two tables: events and flows.

    Note: The AQL CLI does not provide support for joining tables.

    The AQL Event and Flow Query CLI functions in the following modes: Interactive mode - Using a simple shell, you can enter queries interactively

    and view the results in a standard output. At the query prompt, any valid AQL statement is accepted. If time is not specified (using -start and -end options), the last minute is assumed as the time range. You can also access previous commands by using your up arrow. This is the default mode.

    AQL Event and Flow Query CLI Guide

  • 4 USING THE AQL QUERY CLI

    Non-interactive mode - You can enter the non-interactive mode by adding the -execute parameter to the command. The -execute command must be followed by a valid AQL query surrounded by double quotes. The non-interactive mode does not include a prompt allowing you to redirect the output to a file with a regular UNIX pipe syntax. By default, the results are sent to a standard output.

    Accessing the AQL Query CLI

    To access the AQL query CLI:

    Step 1 Log in to STRM, as root.

    Step 2 Enter the following command:/opt/qradar/bin/arielClient

    The Query prompt appears.

    CLI Options Table 1-1 lists the supported CLI options:

    Table 1-1 AQL CLI Options

    Option Description-range

    Limits the number of records sent to the output within the specified range. This is useful for viewing a selection of records generated by an ordered query. For example, if you want to view the first ten records, you must specify -range 1 10.

    -debug Generates debugging output during execution. -start , -end

    Specifies the start and end time of the query. Where specifies the time. You must specify the time as either a UNIX timestamp or a date using the following format: yyyy/mm/dd-hh:mm:ss.For example:/opt/qradar/bin/arielClient - start 2007/08/11-01:15:00 -end 2007/08/11-01:17:00

    -exectime

    Specifies the maximum period of time, in seconds, a single query may continue processing.

    -execute

    Allows you to enter non-interactive mode that enables you to process a query that is sent to standard output. If you do not include this option, the command is entered in interactive mode. You must include your query in double quotes.

    -f

    Allows you to specify the output format for the query results. The table format is an ASCII drawing of a multi-column table while the CSV format provides a comma separated list. Where indicates the output format. The options are table or csv.

    AQL Event and Flow Query CLI Guide

  • Using a Select Statement 5

    For example:

    To enter a command in interactive mode:/opt/qradar/bin/arielClient -start 2007/08/11-01:15:00 -end 2007/08/11-01:17:00 -exectime 60

    /opt/qradar/bin/arielClient

    /opt/qradar/bin/arielClient -start 2007/08/11-01:15:00 -end 2007/08/11-01:17:00 -f csv

    To enter a command in non-interactive mode:/opt/qradar/bin/arielClient -start 2007/08/11-01:15:00 -end 2007/08/11-01:17:00 -exectime 60 -execute "select * from flows where sourceIP = '231.12.37.17' and protocol != 'TCP.tcp_ip'"

    Using a Select Statement

    You can use a select statement that includes one or more fields of a flow or event table. You can also use an asterisk (*) to denote all columns. All field names are case sensitive, however, the terms select and from are not case sensitive.

    For example:select sourceIP, destinationIP, application from flows where protocol = TCP.tcp_ip

    select category, credibility from events where severity > 8

    select * from events where credibility >=9

    The supported flow and event fields include:

    -remote

    Specifies that you want to connect to a specific Ariel query host and port.

    Table 1-1 AQL CLI Options (continued)

    Option Description

    Table 1-2 Supported Flow and Event Fields

    Table Field Name Description Field TypeIdentified in idlist.sh?

    Flows anyDestinationFlag Destination Flags Numeric NoanySourceFlag Source Flags Numeric Noapplication Application String YesapplicationId Application Numeric YesbytesIn Bytes In Numeric NobytesOut Bytes Out Numeric NodestinationAssetName Destination Asset Name String NodestinationASN Destination ASN Numeric No

    AQL Event and Flow Query CLI Guide

  • 6 USING THE AQL QUERY CLI

    destinationBytes Destination Bytes Numeric NodestinationByteRatio Destination Byte Ratio Numeric NodestinationDSCP Destination DSCP Numeric NodestinationDscpOnly Destination DSCP Numeric NodestinationFlags Destination Flags Numeric NodestinationIP Destination IP String NodestinationIPSearch Destination IP Search String NodestinationIfIndex Destination If Index Numeric NodestinationNetwork Destination Network String NodestinationPackets Destination Packets Numeric NodestinationPacketRatio Destination Packet Ratio Numeric NodestinationPayload Destination Payload String NodestinationPayloadHex Destination Payload As Hex Hexadecimal NodestinationPort Destination Port Numeric NodestinationPrecedence Destination Precedence Numeric NodestinationPrecedanceOnly Destination Precedence Numeric Nodestinationv6 IPv6 Destination Hexadecimal NofirstPacketTime First Packet Time Numeric NoflowBias Flow Bias String NoflowDirection Flow Direction:

    Local-to-Local (L2L) Local-to-Remote (L2R) Remote-to-Local (R2L) Remote-to-Remote (R2R)

    String No

    flowSource Flow Source Numeric NoflowType Flow Type Numeric Yesgeographic Matches Geographic Location String NohasDestinationPayload Has Destination Payload Boolean NohasSourcePayload Has Source Payload Boolean NoicmpType ICMP Type/Code Numeric Nointerface Flow Interface String NolastPacketTime Last Packet Time Numeric NopacketsIn Packets In Numeric NopacketsOut Packets Out Numeric Noprotocol Protocol String Yes

    Table 1-2 Supported Flow and Event Fields (continued)

    Table Field Name Description Field TypeIdentified in idlist.sh?

    AQL Event and Flow Query CLI Guide

  • Using a Select Statement 7

    remoteHost Remote Host String NoremoteNet Matches Remote Network String NoremoteServices Matches Remote Service String NosourceASN Source ASN Numeric NosourceAssetName Source Asset Name String NosourceByteRatio Source Byte Ratio Numeric NosourceBytes Source Bytes Numeric NosourceDSCP Source DSCP Numeric NosourceDscpOnly Source DSCP Numeric NosourceFlags Source Flags Numeric NosourceIP Source IP String NosourceIPSearch Source IP Search String NosourceIfIndex Source If Index Numeric NosourceNetwork Source Network String NosourceOrDestinationIP Source or Destination IP String NosourcePackets Source Packets Numeric NosourcePacketRatio Source Packet Ratio Numeric NosourcePayload Source Payload String NosourcePayloadHex Source Payload As Hex Hexadecimal NosourcePort Source Port Numeric YessourcePrecedence Source Precedence Numeric NosourcePrecedanceOnly Source Precedence Numeric Nosourcev6 IPv6 Source Hexadecimal NototalBytes Total Bytes Numeric NototalPackets Total Packets Numeric NoviewObjectPair View/Object String No

    Events category Low Level Category Numeric YescreEventList Matched Custom Rule String Nocredibility Credibility Numeric NodestinationAssetName Destination Asset Name String NodestinationIP Destination IP String NodestinationMAC Destination MAC Hexadecimal NodestinationNetwork Destination Network String NodestinationPort Destination Port Numeric Nodestinationv6 IPv6 Destination Hexadecimal No

    Table 1-2 Supported Flow and Event Fields (continued)

    Table Field Name Description Field TypeIdentified in idlist.sh?

    AQL Event and Flow Query CLI Guide

  • 8 USING THE AQL QUERY CLI

    device Log Source Numeric YesdeviceGroup Log Source Group Numeric NodeviceTime Log Source Time Numeric NodeviceType Log Source Type Numeric Yesduration Duration Numeric NoendTime End Time Numeric NoeventCount Event Count Numeric NoeventDirection Event Direction:

    Local-to-Local Local-to-Remote Remote-to-Local Remote-to-Remote

    String No

    eventProcessor Event Processor Numeric YeshasOffense Associated With Offense Boolean NohighLevelCategory High Level Category Numeric YesisCREEvent Is CRE Event Boolean NoidentityExtendedField Identity Extended Field String NoidentityGroupName Identity Group Name String NoidentityHostName Identity Host Name String NoidentityIP Identity IP String NoidentityMAC Identity MAC Hexadecimal NoidentityNetBiosName Identity NetBIOS Name String NoidentityUserName Identity User Name String Nomagnitude Magnitude Numeric Nopayload Payload String NopayloadHex Payload As Hex Hexadecimal NopostNatDestinationIP Post NAT Destination IP String NopostNatDestinationPort Post NAT Destination Port Numeric NopostNatSourceIP Post NAT Source IP String NopostNatSourcePort Post NAT Source Port Numeric NopreNatDestinationIP Pre NAT Destination IP String NopreNatDestinationPort Pre NAT Destination Port Numeric NopreNatSourceIP Pre NAT Source IP String NopreNatSourcePort Pre NAT Source Port Numeric Noprotocol Protocol String No

    Table 1-2 Supported Flow and Event Fields (continued)

    Table Field Name Description Field TypeIdentified in idlist.sh?

    AQL Event and Flow Query CLI Guide

  • Using a Select Statement 9

    You can also use CIDR-based queries using the select statement. To query by source IP address (sourceIP) or by destination IP address (destinationIP) using a CIDR, use the following format:

    select from where =

    For example:

    select * from flows where sourceCIDR = '10.100.100/24'

    This command returns all flows coming from the 10.100.100 subnet. To capture flows coming from and into the subnet, use the regular OR expression as follows:select * from flows where sourceCIDR = '10.100.100/24' OR destinationCIDR = '10.100.100/24'

    For more information on flow or event fields described using idlist.sh, see Identifying AQL Fields.

    You can use the following statements with any statement to assist you to filter AQL queries.

    qid Event Name ID Numeric Yesrelevance Relevance Numeric Noseverity Severity Numeric NosourceAssetName Source Asset Name String NosourceIP Source IP String NosourceMAC Source MAC Hexadecimal NosourceNetwork Source Network String NosourcePort Source Port Numeric Nosourcev6 IPv6 Source Hexadecimal NostartTime Start Time Numeric Notoken Token Associated With Offense Numeric Nounparsed Event Is Unparsed Boolean NouserName Username String No

    Table 1-2 Supported Flow and Event Fields (continued)

    Table Field Name Description Field TypeIdentified in idlist.sh?

    Table 1-3 Fields Used with Any Statement Type

    Table Field Name Description Field TypeFlow anyASN Source or Destination ASN Numeric

    anyHost Source or Destination IP String

    AQL Event and Flow Query CLI Guide

  • 10 USING THE AQL QUERY CLI

    Using Where Clauses

    You can restrict your AQL queries using where clauses. The supported logical operators in the clause include and, OR, and parentheses. AQL queries also support relational operators include, =, , >=, 9 and category = 5013

    select sourceIP, category, credibility from events where (severity > 9 and category = 5013) or (severity < 5 and credibility > 8)

    The where clause also supports the arieltime variable, which overrides the time settings passed to the AQL CLI. The arieltime variable must be used with the between keyword to specify the start and end time bounds of the query. All time constraints must be entered as either UNIX timestamps or formatted date/time strings.

    You can only use the arieltime variable once in a single query. Therefore, you can only query a continuous span of time in a single AQL command.

    The logical operator for the arieltime variable and the remainder of the where clause should be the and operator. We recommend that you use the arieltime variable as the last constraint of the query and the and operator between the arieltime variable and the rest of the where clause.

    anyNetwork Source or Destination Network

    String

    destinationTOS Destination QoS CompositeicmpType ICMP Type/Code NumericsourceOrDestinationIP Source or Destination IP StringsourceTOS Source QoS Composite

    Events anyIP Any IP StringanyMac Source or Destination MAC

    AddressHex

    anyPort Any Port NumericsourceOrDestinationIP Source or Destination IP StringsourceOrDestinationNetwork Source or Destination

    NetworkString

    sourceOrDestinationPort Source or Destination Port Numeric

    Table 1-3 Fields Used with Any Statement Type (continued)

    Table Field Name Description Field Type

    AQL Event and Flow Query CLI Guide

  • Using the Group By Clause 11

    Using the Group By Clause

    You can use the group by clause to aggregate your data. Typically, data aggregation is combined with arithmetic functions on remaining columns to provide meaningful results of the aggregation. For example, to enter a query to investigate the IP addresses that sent more than 1 million bytes within all flows in a specific time frame, you must enter:

    select sourceIP, SUM(sourceBytes) from flows where sourceBytes > 1000000 group by sourceIP

    The output resembles:-----------------------------------| sourceIP | SUM_sourceBytes |-----------------------------------| 64.124.201.151 | 4282590.0 || 10.105.2.10 | 4902509.0 || 10.103.70.243 | 2802715.0 || 10.103.77.143 | 3313370.0 || 10.105.32.29 | 2467183.0 || 10.105.96.148 | 8325356.0 || 10.103.73.206 | 1629768.0 |-----------------------------------

    However, if you compare this information to a non-aggregate query, the output displays all the IP addresses that are unique:

    select sourceIP, sourceBytes from flows where sourceBytes > 1000000

    ------------------------------| sourceIP | sourceBytes |------------------------------| 64.124.201.151 | 1448629 || 10.105.2.10 | 2412426 || 10.103.70.243 | 1793095 || 10.103.77.143 | 1449148 || 10.105.32.29 | 1097523 || 10.105.96.148 | 4096834 || 64.124.201.151 | 2833961 || 10.105.2.10 | 2490083 || 10.103.73.206 | 1629768 || 10.103.70.243 | 1009620 || 10.105.32.29 | 1369660 || 10.103.77.143 | 1864222 || 10.105.96.148 | 4228522 |------------------------------

    In addition to the sum operator, the min, max, avgnonzero and avg arithmetic aggregation functions are also supported.

    AQL Event and Flow Query CLI Guide

  • 12 USING THE AQL QUERY CLI

    For example:

    To view the maximum number of events.

    select max(eventCount) from events

    To see the number of average events from a source IP.

    select avg(eventCount) from events group by sourceIP

    Using the Order By Clause

    You can add a single order by clause to the end of your AQL CLI query. Only one field can be used in the order by clause. Also, sorting can be switched between ascending or descending by appending the asc or desc keyword to the order by clause, respectively. By default, the query returns results in descending order.

    For example:select sourceBytes, sourceIP from flows where sourceBytes > 1000000 order by sourceBytes

    To display results in ascending order:select sourceBytes, sourceIP from flows where sourceBytes > 1000000 order by sourceBytes asc

    Combing the group by and the order by clauses in a single query is useful for creating data, such as, TopN lists to determine the most abnormal events or the most bandwidth intensive IP addresses. For example, the following query displays the top traffic intensive IP address in a descending order:

    select sourceIP, sum(sourceBytes) from flows group by sourceIP order by sum(sourceBytes) desc

    Using the Count(*) Clause

    You can use the count(*) clause to count the number of records matching your query. For example, if you want to count all events with credibility equal to or greater than 9:

    select count(*) from events where credibility >= 9

    Using the Distinct Clause

    You can use the distinct clause to select unique rows based on a column or a group of columns. This clause is similar to the group by clause, however, the distinct clause ensure ANSI SQL compatibility. For example:

    select distinct sourceIP, sourcePort from flows where sourceBytes > 1000000

    AQL Event and Flow Query CLI Guide

  • Using the Count (Distinct ...) Clause 13

    Using the Count (Distinct ...) Clause

    You can use the standard SQL Count(Distinct ...) clause to obtain unique counts. Using the AQL CLI, you can only use one field. For example, if you want to view all the IP addresses that are connected to a specific IP address over time:select count(distinct sourceIP) from flows where destinationIP = '192.168.61.71'

    Or, if you want to view the number of unique source IP addresses communicating with a particular destination IP address:select destinationIP, count(distinct sourceIP) from flows group by destinationIP

    Using UniqueCount also returns unique counts for items in the table:

    select destinationIP, UniqueCount(sourceIP) from flows

    Note: Using this clause may require additional system resources. Therefore, depending on the query, the amount of time to return results may vary.

    Using the Materialize View Clause

    The materialize view clause allows you to produce query results as a virtual table and run subsequent queries against the view. You can also specify the period of time that the materialized view is accessible.

    The syntax for the materialized view includes:

    materialize view NameOfView as select

    Where:

    specifies the time you want the materialized view to be accessible.

    specifies a valid select statement.

    For example, if you want to create a materialized view containing flows with more than 1,000,000 source bytes, enter the following:

    materialize view LargeSourceBytesFlows as select * from flows where sourceBytes >1000000

    To select from this view, enter the select statement as you would a valid table:

    select * from LargeSourceBytesFlows

    You can also use an aggregation statement on a materialized view:

    select sourceIP, sum(sourceBytes) from LargeSourceBytesFlows group by sourceIP

    Note: You cannot create a materialized view statement based on a previously created materialized view.

    AQL Event and Flow Query CLI Guide

  • 14 USING THE AQL QUERY CLI

    If you want to create a materialized view to select from a record set with ambiguous column names, you can define aliases for all computed columns. For example:

    materialize view MyView as select sourceIP, sum(sourceBytes) as srcBytesSum from flows group by sourceIP

    Then you can refer to the alias in a subsequent query against MyView:

    select * from MyView orderBy srcBytesSum

    Using the Like Clause

    You can search text fields using the standard like clause. You can also use the two wild card options supported by the AQL Query CLI: % and _. The percentage (%) wild card option matches zero or more characters while the _ wild card option only matches one character.

    For example:

    To match names such as Joe, Joanne, Joseph, or any other name beginning with Jo, enter the following clause:select * from events where userName like jo%

    If you want to match names beginning with Jo that are three characters long, such as, Joe or Jon, enter the following clause:select * from events where userName like jo_

    You can enter the wild card option at any point in the command. For example:select * from flows where sourcePayload like %xyz

    select * from events where payload like %xyz%

    select * from events where payload like _yz

    Using the Boolean Clause

    Boolean clauses allow you to restrict your AQL queries to return data matching values you specify using true or false with relational operators. AQL queries support the following relational operators, = and !=.

    For example:To sort events that are unparsed, enter the following clause:select * from events where payload = false

    If you want to sort flows to find a specific source IP address that has an offense, enter the following clause:select * from events where sourceIP = '231.12.37.17' and hasOffense = true

    If you want a list of source IP addresses and protocols that have an offense you can use BooleanTrueCount, and enter the following clause:

    AQL Event and Flow Query CLI Guide

  • Identifying AQL Fields 15

    select sourceIP, protocol, BooleanTrueCount(hasOffense) from events group by sourceIP

    ---------------------------------| sourceIP | protocol |---------------------------------| 64.124.201.151 | TCP.tcp.ip || 10.105.2.10 | UDP.udp.ip || 10.103.70.243 | UDP.udp.ip || 10.103.77.143 | UDP.udp.ip || 10.105.32.29 | TCP.tcp.ip || 10.105.96.148 | TCP.tcp.ip || 64.124.201.151 | TCP.tcp.ip || 10.105.2.10 | ICMP.icmp.ip |---------------------------------

    Identifying AQL Fields

    AQL queries to event or flow fields may return numeric codes making query responses or searches difficult. The shell script idlist.sh provides additional information from numeric AQL fields in the events and flows table.

    To access the shell script idlist.sh:Step 1 Log in to STRM, as root.

    Step 2 Enter the following command:/opt/qradar/bin/idlist.sh

    Step 3 Enter the appropriate parameters:

    Note: You can search the idlist.sh output using standard operators such as fowardslash. Search terms are case sensitive in the idlist.sh output.

    Table 1-4 AQL Field Parameters

    Parameters Description-f Specify that the script is to identify a field from the flows table.

    For example:/opt/qradar/bin/idlist.sh -f application

    -e Specify that the script is to identify a field from the events table. For example:/opt/qradar/bin/idlist.sh -e category

    Specify the field name of the event or flow you want to identify. See Table 1-2 for a complete list of fields that can be identified using idlist.sh.

    AQL Event and Flow Query CLI Guide

  • 16 USING THE AQL QUERY CLI

    Listing AQL Fields AQL supports a large number of field names for the events and flows table. To view a list of the most commonly used field names for a select statement, you can use the describe command.

    To access the AQL describe command:

    Step 1 Log in to STRM, as root.

    Step 2 Enter the following command:/opt/qradar/bin/arielClient

    The Query prompt appears. Step 3 Enter the following command to view a list of fields from the flows table:

    describe flows

    The following flows are listed.

    Table 1-5 Describe Flows

    Table Field Name Description Field TypeFlow application Application String

    destinationASN Destination ASN NumericdestinationBytes Destination Bytes NumericdestinationDSCP Destination DSCP NumericdestinationFlags Destination Flags NumericdestinationIP Destination IP StringdestinationIfIndex Destination If Index NumericdestinationNetwork Destination Network StringdestinationPackets Destination Packets NumericdestinationPayload Destination Payload StringdestinationPort Destination Port NumericdestinationPrecedence Destination Precedence NumericdestinationTOS Destination QOS CompositefirstPacketTime First Packet Time NumericflowBias Flow Bias StringflowDirection Flow Direction:

    Local-to-Local (L2L) Local-to-Remote (L2R) Remote-to-Local (R2L) Remote-to-Remote (R2R)

    String

    flowSource Flow Source NumericflowType Flow Type Numeric

    AQL Event and Flow Query CLI Guide

  • Listing AQL Fields 17

    Step 4 Enter the following command to view a list of fields from the events table:describe events

    The following events are listed.

    geographic Matches Geographic Location

    String

    icmpType ICMP Type/Code Numericinterface Flow Interface StringintervalId Interval ID NumericlastPacketTime Last Packet Time NumericprotocolId Protocol ID NumericsourceASN Source ASN NumericsourceBytes Source Bytes NumericsourceDSCP Source DSCP NumericsourceFlags Source Flags NumericsourceIP Source IP StringsourceIfIndex Source If Index NumericsourceNetwork Source Network StringsourcePackets Source Packets NumericsourcePayload Source Payload StringsourcePort Source Port NumericsourcePrecedence Source Precedence NumericsourceTOS Source QOS Compositetoken Associated With Offense Numeric

    Table 1-5 Describe Flows (continued)

    Table Field Name Description Field Type

    Table 1-6 Describe Events

    Table Field Name Description Field TypeEvent category Low Level Category Numeric

    credibility Credibility NumericdestinationIP Destination IP StringdestinationMAC Destination MAC HexadecimaldestinationNetwork Destination Network StringdestinationPort Destination Port Numericdevice Log Source NumericdeviceGroup Log Source Group NumericdeviceType Log Source Type Numericduration Duration Numeric

    AQL Event and Flow Query CLI Guide

  • 18 USING THE AQL QUERY CLI

    endTime End Time NumericeventCount Event Count NumericeventDirection Event Direction:

    Local-to-Local Local-to-Remote Remote-to-Local Remote-to-Remote

    String

    hasIdentity Has Identity BooleanhasOffense Associated With Offense BooleanhighLevelCategory High Level Category NumericidentityExtendedField Identity Extended Field StringidentityGroupName Identity Group Name StringidentityHostName Identity Host Name StringidentityIP Identity IP StringidentityMAC Identity MAC HexadecimalidentityNetBiosName Identity NetBIOS Name StringidentityUserName Identity User Name Stringmagnitude Magnitude Numericpayload Payload StringpostNatDestinationIP Post NAT Destination IP StringpostNatDestinationPort Post NAT Destination Port NumericpostNatSourceIP Post NAT Source IP StringpostNatSourcePort Post NAT Source Port NumericpreNatDestinationIP Pre NAT Destination IP StringpreNatDestinationPort Pre NAT Destination Port NumericpreNatSourceIP Pre NAT Source IP StringpreNatSourcePort Pre NAT Source Port Numericprotocol Protocol Stringqid Event Name ID Numericrelevance Relevance Numericseverity Severity NumericsourceIP Source IP StringsourceMAC Source MAC HexadecimalsourceNetwork Source Network StringsourcePort Source Port NumericstartTime Start Time Numeric

    Table 1-6 Describe Events (continued)

    Table Field Name Description Field Type

    AQL Event and Flow Query CLI Guide

  • Listing AQL Fields 19

    token Token Associated With Offense

    Numeric

    unparsed Event Is Unparsed BooleanuserName Username String

    Table 1-6 Describe Events (continued)

    Table Field Name Description Field Type

    AQL Event and Flow Query CLI Guide

  • About This GuideConventionsTechnical DocumentationContacting Customer Support

    Using the AQL Query CLIAbout the AQL Query CLIAccessing the AQL Query CLICLI Options

    Using a Select StatementUsing Where ClausesUsing the Group By ClauseUsing the Order By ClauseUsing the Count(*) ClauseUsing the Distinct ClauseUsing the Count (Distinct ...) ClauseUsing the Materialize View ClauseUsing the Like ClauseUsing the Boolean ClauseIdentifying AQL FieldsListing AQL Fields