RPE-020: Seeing the Forest for the Trees
Events
RPE-020: Seeing the Forest For the Trees
Date: November 15-16, 2022 | Location: Virtual or DreamPort Facility in Columbia, MD
Description
Wiktionary describes the idiom to “see the forest for the trees” as discerning an overall pattern from a mass of detail – seeing the big picture. This is precisely the challenge presented by cybersecurity alert analysis: being overwhelmed by details (trees) to the point where they obscure the overall situation (forest). Security Operations Center (SOC) analysts often receive more alerts than they can possibly investigate, so we are seeking innovative techniques for reducing alert volume.
Overview
In this rapid prototyping event, we are seeking techniques and solutions that can aid network defenders (e.g., SOC analysts) in making sense of a flood of alerts. While this challenge is not strictly limited to AI / ML approaches, there are certainly some promising signs that advances in unsupervised machine learning algorithms can be applied to this problem. Such approaches will be viewed favorably. We have $25,000 (USD) in prizes to give away to the best performers as determined by us!
A successful alert volume reduction solution will be capable of answering the following questions:
- Is this alert part of an ongoing incident?
- Is this alert part of an incident we’ve seen before?
- Is this alert part of a new type of incident?
- In general, can we group alerts into incidents (or other categories) that improve the triage and remediation efforts of an analyst?
This RPE directly addresses the following U.S. Cyber Command technical challenge problems:
- 2020 1.3 CHALLENGE PROBLEM: Identifying Malware
- 2020 2.4 CHALLENGE PROBLEM: User Activity Monitoring
- 2020 3.2 CHALLENGE PROBLEM: Normal and Abnormal Operating Conditions
- 2020 3.3 CHALLENGE PROBLEM: Automated Exploit and Capabilities Discovery
Requirements
Participants are expected to produce a Jupyter notebook, script, or executable that can ingest a file containing alert data in the JSON-based Intrusion Detection Extensible Alert (IDEA) format (link) and produce output that is actionable by SOC analysts. The output format is left to participants; however, the analysis performed and the output itself must be explained interactively to MISI staff during your evaluation session. Consider the benefits of human-readable as well as machine-readable (and searchable) output and/or visualizations. There are no requirements regarding which development language(s) can be used, and participants must provide their own computing equipment.
The event will be based on a publicly-available data set of nearly 12 million alerts collected from the Czech Republic’s SABU alert sharing platform (link). The IDEA file is supplemented by additional files containing enrichment data (e.g., geolocation and reputation of all involved IP addresses).
Participants should begin their analysis immediately, as the event itself consists of evaluation sessions where participants explain their analysis and techniques. Participants must provide their Jupyter notebooks, etc., as well for evaluation, but that information will be held as proprietary within MISI / DreamPort / USCYBERCOM.
The evaluation sessions can be conducted in person at the DreamPort facility (Internet access via WiFi will be provided) or virtually via screen sharing.
Schedule
We have chosen the following schedule of events:
- 27 September 2022: RPE Announced Publicly
- 3 November: Registration Closes
- 4 November: Q&A Session (Hybrid)
- 15-16 November: Main Event (Hybrid)
Participants are welcome to attend the Q&A session either virtually or in person at DreamPort. Similarly, participants may attend their scheduled evaluation time slot either virtually or in person at DreamPort.
Evaluation
Each participant will be assigned a time slot during which they must explain their notebook(s), analysis, and techniques employed to one of our data scientists. Given that the focus is unsupervised machine learning, the evaluation will be mainly qualitative. Evaluation criteria include:
- Explanation of data analysis
- Explanation of modeling approaches and feature extraction
- Demonstration of how your approach makes analysts’ jobs easier
- Description of model quality
- Demonstration or description of how approach would process new incoming alerts
- Explanation of managed embeddings (where applicable)
- Computing requirements, including any external dependencies (e.g., public cloud)
Selection of winner(s) will occur after all participant evaluation sessions have concluded. Participants will be notified of the results as soon as possible.
Suggested Skills
We recommend the following skills for RPE participants:
- Machine learning algorithms for alert volume reduction
- Security alerting systems
- Threat information sharing
- Incident response
Prizes
DreamPort will award $25,000 in cash prizes to the best participants, as determined by MISI based on post-event evaluation.

Congratulations to Graphistry for winning RPE-020: Seeing the Forest for the Trees!
First, we want to thank all who participated! Second, we are happy to announce that Graphistry was the winner of RPE-020 with their GrAI-Vi solution. Graphistry truly embraced the RPE's Alert Volume Reduction challenge, demonstrating robust AI/ML approaches to achieve a 97% reduction in RPE data set alert volume while highlighting patterns that could also support triage and threat hunting cases.
Prizes
DreamPort will award $25,000 in cash prizes to the best participants, as determined by MISI based on post-event evaluation.


