Usage¶
usage: aristotle.py [-h] -r RULES [-f METADATA_FILTER]
[--summary [DISPLAY_MAX]] [-o OUTFILE]
[-s [STATS [STATS ...]]] [-i] [-c] [-n] [-e] [-t] [-g]
[-m] [-p PFMOD_FILE] [-q] [-d]
Filter Suricata and Snort rulesets based on metadata keyword values.
optional arguments:
-h, --help show this help message and exit
-r RULES, --rules RULES, --ruleset RULES
path to a rules file, a directory containing '.rules'
file(s), or string containing the ruleset
-f METADATA_FILTER, --filter METADATA_FILTER
Boolean filter string or path to a file containing it
--summary [DISPLAY_MAX]
output a summary of the filtered ruleset to stdout,
limited to DISPLAY_MAX number of lines (or 16 if no
value given); if the option to output to a file is
set, the full, filtered ruleset will still be written.
-o OUTFILE, --output OUTFILE
output file to write filtered ruleset to
-s [STATS [STATS ...]], --stats [STATS [STATS ...]]
display ruleset statistics about specified key(s). If
no key(s) supplied, then summary statistics for all
keys will be displayed.
-i, --enable-all-rules, --enable-all, --include-disabled
enable all valid rules, including those
disabled/commented out in the given rules file(s),
when applying the filter
-c, --output-disabled-rules
include disabled rules in the output as commented out
lines.
-n, --normalize, --better, --iso8601
try to convert date and cve related metadata values to
conform to the BETTER schema for filtering and
statistics. Dates are normalized to the format YYYY-
MM-DD and CVEs to YYYY-<num>. Also, 'sid' is removed
from the metadata.
-e, --enhance enhance metadata by adding additional key-value pairs
based on the rules.
-t, --ignore-classtype, --ignore-classtype-keyword
don't incorporate the 'classtype' keyword and value
from the rule into the metadata structure for
filtering and reporting.
-g, --ignore-filename
don't incorporate the 'filename' keyword (filename of
the rules file) into the metadata structure for
filtering and reporting.
-m, --modify-metadata
modify the rule metadata keyword value on output to
contain the internally tracked and normalized metadata
data.
-p PFMOD_FILE, --pfmod PFMOD_FILE, --pfmod-file PFMOD_FILE
YAML file of directives to apply actions on post-
filtered rules based on filter strings.
-q, --quiet, --suppress_warnings
quiet; suppress warning logging
-d, --debug turn on debug logging
A filter string defines the desired outcome based on Boolean logic, and uses
the metadata key-value pairs as values in a (concrete) Boolean algebra.
The key-value pair specifications must be surrounded by double quotes.
Example:
python3 aristotle/aristotle.py -r examples/example.rules --summary -n
-f '(("priority high" AND "malware <ALL>") AND "created_at >= 2018-01-01")
AND NOT ("protocols smtp" OR "protocols pop" OR "protocols imap") OR "sid 80181444"'
Example Files¶
The examples
directory has .filter
files that show examples of Boolean
filter strings.
Also in the examples
directory is an example.rules
file that has a dummy
Suricata ruleset that implements the BETTER Schema.
While the example
ruleset is syntactically correct, it is not a real ruleset
intended to be used by a Suricata sensor.
It is provided to assist in demonstrating the functionality of
Aristotle and to provide examples of rules with metadata
keywords that
conform to the BETTER Schema.
Example Usage¶
Note
aristotle.py
in the root of the repository is a symlink
to aristotle/aristotle.py
. If the environment in use does
not recognize symlinks, adjust the paths accordingly.
Show high level statistics on all the keys in the example.rules
file:
python aristotle.py -r examples/example.rules -s
Show statistics on the protocols
key in the example.rules
file:
python aristotle.py -r examples/example.rules -s protocols
Apply the Boolean filter defined in the example1.filter
file against the
rules in the example.rules
file and output summary results to stdout:
python aristotle.py -r examples/example.rules -f examples/example1.filter --summary
Apply the Boolean filter defined in the example1.filter
file against the
rules in the example.rules
file and output the results to the file newrules.rules
:
python aristotle.py -r examples/example.rules -f examples/example1.filter -o newrules.rules
Apply the Boolean filter defined specified on the command line against the
rules in the example.rules
file and output the results to the file newrules.rules
:
python aristotle.py -r examples/example.rules -f '"malware <ALL>" AND ("attack_target http-server" or "attack_target tls-server")' -o newrules.rules
Consume the rules defined in the examples/example.rules
, Normalize the metadata,
apply the Boolean filter defined in the example1.filter
file against the
rules in the example.rules
file, and output the results – with updated metadata – to
the file newrules.rules
:
python aristotle.py -r examples/example.rules -f examples/example1.filter -o newrules.rules --normalize --modify-metadata
Important
Because Aristotle requires key-value pairs (values) in the filter string to be enclosed in double quotes, a filter string specified on the command line must be enclosed in single quotes.
Statistics¶
The statistics command line option allows a user to to easily see what metadata key-value pairs the ruleset contains to assist in building a filter string.
Note
If a filter string is provided, or other options to manipulate the metadata (e.g. Normalize, Modify Metadata) are set, along with a request for statistics, then the filter and/or other manipulations are performed first, and the statistics outputted apply to the filtered/modified ruleset.
If no key names are passed, summary info on all present keys is displayed:
$ python aristotle.py -r examples/example.rules -s
Aristotle
Ruleset Metadata Tool
All Rules: Total: 6799; Enabled: 4977; Disabled: 1822
attack_target (Total: 6028; Enabled: 4554; Disabled: 1474)
malware (Total: 3467; Enabled: 3330; Disabled: 137)
cve (Total: 1570; Enabled: 887; Disabled: 683)
hostile (Total: 5962; Enabled: 4403; Disabled: 1559)
created_at (Total: 6799; Enabled: 4977; Disabled: 1822)
capec_id (Total: 2669; Enabled: 1191; Disabled: 1478)
updated_at (Total: 6799; Enabled: 4977; Disabled: 1822)
cwe_id (Total: 5199; Enabled: 4332; Disabled: 867)
priority (Total: 6799; Enabled: 4977; Disabled: 1822)
cvss_v3_base (Total: 271; Enabled: 259; Disabled: 12)
infected (Total: 2679; Enabled: 2520; Disabled: 159)
sid (Total: 6799; Enabled: 4977; Disabled: 1822)
cvss_v2_base (Total: 1130; Enabled: 829; Disabled: 301)
rule_source (Total: 6799; Enabled: 4977; Disabled: 1822)
cvss_v3_temporal (Total: 271; Enabled: 259; Disabled: 12)
filename (Total: 6799; Enabled: 4977; Disabled: 1822)
cvss_v2_temporal (Total: 1130; Enabled: 829; Disabled: 301)
protocols (Total: 6799; Enabled: 4977; Disabled: 1822)
If one or more key names are passed, summary info is displayed for those keys:
$ python aristotle.py -r examples/example.rules -s malware protocols
Aristotle
Ruleset Metadata Tool
All Rules: Total: 6799; Enabled: 4977; Disabled: 1822
malware (Total: 3467; Enabled: 3330; Disabled: 137)
download-attempt (Total: 178; Enabled: 171; Disabled: 7)
malware (Total: 135; Enabled: 117; Disabled: 18)
post-infection (Total: 2647; Enabled: 2589; Disabled: 58)
pre-infection (Total: 507; Enabled: 453; Disabled: 54)
protocols (Total: 6799; Enabled: 4977; Disabled: 1822)
smtp (Total: 143; Enabled: 82; Disabled: 61)
pop (Total: 64; Enabled: 45; Disabled: 19)
rpc (Total: 16; Enabled: 4; Disabled: 12)
dnp3 (Total: 5; Enabled: 0; Disabled: 5)
vnc (Total: 1; Enabled: 0; Disabled: 1)
ftp (Total: 130; Enabled: 65; Disabled: 65)
sip (Total: 5; Enabled: 3; Disabled: 2)
iccp (Total: 4; Enabled: 0; Disabled: 4)
dns (Total: 20; Enabled: 6; Disabled: 14)
ldap (Total: 1; Enabled: 1; Disabled: 0)
irc (Total: 21; Enabled: 19; Disabled: 2)
nntp (Total: 4; Enabled: 0; Disabled: 4)
smb (Total: 60; Enabled: 42; Disabled: 18)
http (Total: 5447; Enabled: 4199; Disabled: 1248)
telnet (Total: 9; Enabled: 3; Disabled: 6)
dcerpc (Total: 1; Enabled: 1; Disabled: 0)
tcp (Total: 6788; Enabled: 4976; Disabled: 1812)
imap (Total: 55; Enabled: 25; Disabled: 30)
tls (Total: 145; Enabled: 128; Disabled: 17)
modbus (Total: 7; Enabled: 0; Disabled: 7)
tftp (Total: 1; Enabled: 0; Disabled: 1)
ssh (Total: 9; Enabled: 4; Disabled: 5)
Classtype¶
Suricata and Snort support the classtype
keyword and many rulesets choose to utilize this rule keyword
instead of putting the classtype
key-value pair into the metadata. Therefore, by default, Aristotle
will take the classtype
value from the rule keyword and add a classtype
metadata key and value into the
(internal data structures representing the) metadata so that it can be used for filtering and statistics generation.
If multiple classtype
keywords are used in a rule, only the first one seen (from left-to-right) will be
incorporated. The classtype
keyword can be used in a rule and defined in the metadata without issue; only
the unique values will be considered.
This default behavior can be changed with a command line switch or in
the Ruleset class constructor.
Filename¶
If a rule was loaded from a file, Aristotle will add the filename
key-value pair so that it can be
used for filtering and statistics generation. The value will be the filename the rule was read from.
This default behavior can be changed with a command line switch or in
the Ruleset class constructor.
Disabled Rules¶
Internally, an originally_disabled
key and boolean value (true
or false
, case insensitive) is added to
the metadata of each rule. If a rule from a rule file is a valid rule but commented out, the originally_disabled
value will be true
, otherwise it will be false
. The originally_disabled
metadata can be used for filtering,
including Post Filter Modification. For more details see the Disabled Rules doc.
Normalize¶
The normalize command line option (also supported in the Ruleset class constructor will do the following to the internal data structure used to store metadata and filter against:
cve
key value(s) normalized toYYYY-<num>
. If multiple CVEs are represented in the value and strung together with a_
(e.g.cve_2021_27561_cve_2021_27562
[sic]) then all identified CVEs will be normalized and included.- Values from non-BETTER schema keys
mitre_technique_id
andmitre_tactic_id
will be put into the standards compliantmitre_attack
key.- date key values – determined by any key names that end with
_at
or-at
– will be attempted to be normalized to ISO 8601 formatYYYY-MM-DD
. A failure to parse or normalize the value will result in a warning message and the value being unchanged.
If the Modify Metadata
option is also set then the normalized values, rather than the originals, will be
included in the output, and the sid
key will be removed from the metadata.
Enhance¶
The enhance command line option (also supported in the Ruleset class constructor will analyze the rule(s) and attempt to update the metadata on each. Specifically, each rule will be examined ontologically, and new metadata keys and values will be identified, deduced, and induced. These new metadata entries will be added to the rule (or merged if the key already exists). Such metadata keys include:
flow
key with values normalized to beto_server
orto_client
.protocols
key and applicable values, per the BETTER Schema.cve
key and applicable values, per the BETTER Schema. The value(s) are based on data extracted from the raw rule, e.g.msg
field,reference
keyword, etc.mitre_attack
key and applicable values, per the BETTER Schema. The value(s) are based on data extracted from the rule’sreference
keyword.hostile
key and applicable values (dest_ip
orsrc_ip
), per the BETTER Schema. The values are the inverse of values taken from thetarget
keyword.classtype
* key and applicable values, per the BETTER Schema. See the Classtype section.filename
* key and applicable values, per the BETTER Schema. The value will be the filename the rule came from, if the rule was loaded from a file. See the Filename section.originally_disabled
key and boolean value gets added on each rule internally, and can be used for filtering, but only is included in the output if “enhance” is enabled (along with Modify Metadata). See also the Disabled Rules section.detection_direction
key (see below).
* Key added by default unless explicitly disabled.
Detection Direction¶
The detection_direction
metadata key attempts to induce and normalize the directionality of traffic the rule
detects on. To do this, the source and destination (IP/IPVAR) sections of the rule are reduced down to “$HOME_NET”,
“$EXTERNAL_NET”, “any”, or “UNDETERMINED” and used to set the detection_direction
value as follows:
detection_direction value | reduced condition |
---|---|
inbound | $EXTERNAL_NET -> $HOME_NET |
inbound-notexclusive | any -> $HOME_NET |
outbound | $HOME_NET -> $EXTERNAL_NET or
$EXTERNAL_NET -> $EXTERNAL_NET |
outbound-notexclusive | $HOME_NET -> any |
internal | $HOME_NET -> $HOME_NET |
any | any -> any |
both | direction in rule is <> |
unknown | UNDETERMINED |
Modify Metadata¶
Note
No metadata is altered on output unless the Modify Metadata option is set!
The command line and the Ruleset class constructor offer
the option to update the metadata keyword value on output. If this option is not set,
Aristotle does not modify rules, it just enable or disables them based on the given
filter. However, if the modify metadata option is set, then the value of the metadata
keyword will be replaced with a string sourced from the internal data structure that
Aristotle uses to track, parse, and filter metadata. Practically, the metadata will
be updated accordingly:
sid
key and value added to metadata (unless the Normalize option is set).classtype
key and value added to metadata, if theclasstype
keyword is present in the rule and the option to ignore classtype is not set.filename
key and value added to metadata, if the rule(s) came from a file and the option to ignore filename is not set.- if the Normalize option is set, any changes done by that will be included.
- if the Enhance options is set, any changes done by that will be included.
Additionally, the order of the key-value pairs in the metadata will be sorted by key and then value.
Important
To enable efficient boolean logic application and because metadata is considered by Aristotle to be case insensitive per the BETTER Schema, metadata key-value pairs are represented internally as lowercase. Therefore, if Modify Metadata is enabled, the outputted metadata key-value pairs will be all lowercase.