v5.5.6

Generate MITRE Attack Output layer.
Fix a bug intrdocued in tyro v0.0.9.23 where if an extremely large number of files (greater than 530 or so) are passed to mode:selected --mode.files ..., the command line parser crashes.

What's Changed

• Bump MITRE ATT&CK version in output layer by @ljstella in #417
• Update pyproject.toml by @pyth0n1c in #419

Full Changelog: v5.5.5...v5.5.6

v5.5.5

Added some "allowed macros" to validation because they exist in Enterprise Security.

What's Changed

• Adding new macro & lookup exceptions by @ljstella in #415

Full Changelog: v5.5.4...v5.5.5

v5.5.4

contentctl report has been updated to output MITRE Attack Navigator in the 5.1.0 format.

What's Changed

• TR-3506 MITRE MAP Update by @josehelps in #413

Full Changelog: v5.5.3...v5.5.4

v5.5.3

What's Changed

• Catch duplicate analytic stories and other mapped content by @pyth0n1c in #410

Full Changelog: v5.5.2...v5.5.3

v5.5.2

This just bumps the names of the objects generated in dist/api to end in _v2. This is because the detection schema changed slightly, so we want to differentiate them from the old objects.

What's Changed

• Bump api to v2 by @pyth0n1c in #408

Full Changelog: v5.5.1...v5.5.2

v5.5.1

Minor typing changes to a field from str to int

What's Changed

• Risk from str to int by @pyth0n1c in #406

Full Changelog: v5.5.0...v5.5.1

v5.5.0

Most notably, this PR adds support for a cached version of the attack_data repo that is usable during validate and test operations.
This offers a number of distinct advantages in terms of runtime performance and up-front error checking:

• If attack_data links were incorrect (for example, they link to http/s files that do not exist), then testing could fail at test runtime rather than validation time.
• Downloading files sometimes fails due to rate limiting from GitHub
• Attack Data files may be very large (hundreds of MB) and the https://media.githubusercontent endpoint does not compress these files (as it assumes they are media and, thus, already highly compressed)
• The same attack data file is often downloaded multiple times if it is used by multiple detections

A cache of the https://github.com/splunk/attack_data repo is now hosted and available, compressed with zstd at https://attack-range-attack-data.s3.us-west-2.amazonaws.com/attack_data.tar.zstd. This cache is updated whenever new attack data is merged into the master branch.
This archive includes all of the data in the attack_data/datasets for and is only ~215MB. This makes it a significant improvement over the uncompressed size of all attack data today (about 10GB).

To begin using this data during validation or testing, check out the helptext here:
https://github.com/splunk/contentctl/blob/2b633b6207d028f40908b8d7a618544aaa3876ce/contentctl/objects/config.py#L283-L293

Note that running contentctl validate/build/test with the --verbose flag (such as contentctl validate --verbose) enables even more extended validations of your test data during the validation phase, further reducing chances of a failure to find attack data at runtime.

What's Changed

• detection_Type and _entities conf file updates by @pyth0n1c in #404
• Enable Attack Data Download before Test by @pyth0n1c in #392

Full Changelog: v5.4.1...v5.5.0

v5.4.1

This includes a small fix where certain types of validation errors could be clobbered and not appear on the command line. In particular, validation errors to the "tests" section of a Detection could fail to appear.
This issue has now been fixed (however these error lack formatting as elegant as most of the other errors).
Overall, errors are still much easier to read than they were before.

What's Changed

• Update setuptools requirement from >=69.5.1,<79.0.0 to >=69.5.1,<81.0.0 by @dependabot in #401
• New error printout generic case by @pyth0n1c in #403

Full Changelog: v5.4.0...v5.4.1

v5.4.0

Validation output has been significantly improved! In the example below, we see the two big changes that have been introduced:

1. Validation is now color-coded, formatted, and easier to read! Thanks to @MHaggis for these changes!

2. When referencing another piece of content, such as a macro, story, lookup, data source, etc, it is not uncommon to accidentally typo the name. If we fail to find a piece of related content, we now provide suggestions for the content you probably meant to include, sorted from most to least likely.
   

There are also some new validations against the Risk Datamodel to improve internal integration testing

What's Changed

• Update validate output by @MHaggis in #370
• Validation output improvements by @ljstella in #402
• draft for more explicit info by @pyth0n1c in #369
• Adding risk data model validation to integration testing by @cmcginley-splunk in #387

New Contributors

• @MHaggis made their first contribution in #370

Full Changelog: v5.3.2...v5.4.0

v5.3.2

Previously when checking the version field, we only required the version of modified detection be greater than the previous version. We now require it to be exactly previous_version+1. This cuts down on weird gaps in detection versions between releases where a detection's version has been bumped several times between releases.

What's Changed

• New type of version bumping error by @ljstella in #400

Full Changelog: v5.3.1...v5.3.2