RPE-020: Seeing the Forest for the Trees

Events

RPE-020: Seeing the Forest For the Trees

Date: November 15-16, 2022 | Location: Virtual or DreamPort Facility in Columbia, MD

---

Description

Wiktionary describes the idiom to “see the forest for the trees” as discerning an overall pattern from a mass of detail – seeing the big picture. This is precisely the challenge presented by cybersecurity alert analysis: being overwhelmed by details (trees) to the point where they obscure the overall situation (forest). Security Operations Center (SOC) analysts often receive more alerts than they can possibly investigate, so we are seeking innovative techniques for reducing alert volume.

Overview

In this rapid prototyping event, we are seeking techniques and solutions that can aid network defenders (e.g., SOC analysts) in making sense of a flood of alerts. While this challenge is not strictly limited to AI / ML approaches, there are certainly some promising signs that advances in unsupervised machine learning algorithms can be applied to this problem. Such approaches will be viewed favorably. We have $25,000 (USD) in prizes to give away to the best performers as determined by us!

A successful alert volume reduction solution will be capable of answering the following questions:

1. Is this alert part of an ongoing incident?
2. Is this alert part of an incident we’ve seen before?
3. Is this alert part of a new type of incident?
4. In general, can we group alerts into incidents (or other categories) that improve the triage and remediation efforts of an analyst?

This RPE directly addresses the following U.S. Cyber Command technical challenge problems:

• 2020 1.3 CHALLENGE PROBLEM: Identifying Malware
• 2020 2.4 CHALLENGE PROBLEM: User Activity Monitoring
• 2020 3.2 CHALLENGE PROBLEM: Normal and Abnormal Operating Conditions
• 2020 3.3 CHALLENGE PROBLEM: Automated Exploit and Capabilities Discovery

Requirements

Participants are expected to produce a Jupyter notebook, script, or executable that can ingest a file containing alert data in the JSON-based Intrusion Detection Extensible Alert (IDEA) format (link) and produce output that is actionable by SOC analysts. The output format is left to participants; however, the analysis performed and the output itself must be explained interactively to MISI staff during your evaluation session. Consider the benefits of human-readable as well as machine-readable (and searchable) output and/or visualizations. There are no requirements regarding which development language(s) can be used, and participants must provide their own computing equipment.

The event will be based on a publicly-available data set of nearly 12 million alerts collected from the Czech Republic’s SABU alert sharing platform (link). The IDEA file is supplemented by additional files containing enrichment data (e.g., geolocation and reputation of all involved IP addresses).

Participants should begin their analysis immediately, as the event itself consists of evaluation sessions where participants explain their analysis and techniques. Participants must provide their Jupyter notebooks, etc., as well for evaluation, but that information will be held as proprietary within MISI / DreamPort / USCYBERCOM.

The evaluation sessions can be conducted in person at the DreamPort facility (Internet access via WiFi will be provided) or virtually via screen sharing.

Schedule

We have chosen the following schedule of events:

• 27 September 2022: RPE Announced Publicly
• 3 November: Registration Closes
• 4 November: Q&A Session (Hybrid)
• 15-16 November: Main Event (Hybrid)

Participants are welcome to attend the Q&A session either virtually or in person at DreamPort. Similarly, participants may attend their scheduled evaluation time slot either virtually or in person at DreamPort.

Evaluation

Each participant will be assigned a time slot during which they must explain their notebook(s), analysis, and techniques employed to one of our data scientists. Given that the focus is unsupervised machine learning, the evaluation will be mainly qualitative. Evaluation criteria include:

• Explanation of data analysis
• Explanation of modeling approaches and feature extraction
• Demonstration of how your approach makes analysts’ jobs easier
• Description of model quality
• Demonstration or description of how approach would process new incoming alerts
• Explanation of managed embeddings (where applicable)
• Computing requirements, including any external dependencies (e.g., public cloud)

Selection of winner(s) will occur after all participant evaluation sessions have concluded. Participants will be notified of the results as soon as possible.

Suggested Skills

We recommend the following skills for RPE participants:

• Machine learning algorithms for alert volume reduction
• Security alerting systems
• Threat information sharing
• Incident response

Prizes

DreamPort will award $25,000 in cash prizes to the best participants, as determined by MISI based on post-event evaluation.