Event RPE-003: The Wolf in Sheep's Clothing

Events

RPE-003: The Wolf in Sheep's Clothing

Date: January 29-31, 2019 & February 12-14, 2019 | Location: DreamPort Facility in Columbia MD


Overview

The Fables of Aesop by Joseph Jacobs published in 1894 includes the tale of the wolf in sheep's clothing. Just as in this tale, DreamPort is playing the Sheperd trying to spot the wolf in the flock. DreamPort plans to execute an RPE competition where participants design, implement or enhance user activity monitoring (UAM) solutions for detecting live and recent insider threat attacks or unauthorized activities. DreamPort is interested in identifying UAM solutions that employ advanced real-time analysis of multiple data sources for detecting unauthorized activities. DreamPort will again use HQ (from our first RPE) but this time setup a we will use live portions of the HQ environment consisting of Windows and Linux machines (endpoints and servers) and use human actors to interact with this environment in real-time. During the RPE, one or more of these actors will be 'bad' and will execute unauthorized actions at any time. Participants must install their solutions within HQ and attempt to detect these bad actors as quickly as possible. The solution(s) which detect all of the unauthorized actions will be deemed a winner'.

DreamPort acknowledges multiple solutions are available today for UAM, but we are explicitly interested in solutions which offer predictive monitoring (configuration-less) features and not just policy-based (ALLOW/DENY) monitoring features.

Participant solutions should offer alerts for overtly prohibited activities and activities which fall outside normal operating behavior.

Evaluation
Judges will evaluate participant solutions during judging rounds of one hour each. During this time, multiple actors will be using HQ resources performing daily activities while participants will be alerted that at least one prohibited and at least three suspicious activities will take but place possibly more of each. No visual or audible alerts will be given when a suspicious or malicious activity is about to take place or after it has occurred. Solutions which can identify the activity, characterize properly and tie back to a user the fastest will be considered the winner.

Outside of judging rounds, DreamPort actors will continue to use HQ resources but no more than three actors will be active during these times. A malicious event may occur outside of judging rounds and participant solutions that are able to identify those events will receive extra credit scores. HQ will be connected to the Internet and consist of at least the following elements:

  • One Active Directory Domain Controller also offering
    • DHCP
    • DNS
    • NTP
    • File Share with documents (which will be marked with sensitive keywords)
  • One HTTP proxy server with HTTPS monitoring
  • At least ten accounts, including two administrators, a minimum of five which will be active during judging times.
  • Between three to five Linux servers offering HTTP, SSH, FTP and application layer services
  • One Source Code control server
  • One unmanaged gigabit switch
  • One 'Production' database server
  • One Administrator workstation
  • Five end-user workstations
  • Central Syslog Server
  • One Firewall (ASA 5505)

DreamPort will publish Syslog and Windows Event log data from HQ servers and endpoints for all these resources which will be guaranteed to consist of at least one month of limited usage but will not contain any malicious or suspicious activities. DreamPort will consider requests for publishing additional UAM training data prior to the start of the RPE but these requests will be shared with all parties. No one participant will have sole access to UAM data.

Suggested Skills
This RPE requires participants have the at least intermediate the following skills:

  • Text Processing (log files)

The following skills are suggested:

  • Machine Learning

Expected Solution
A participant should plan to bring a solution that can ingest UAM data from suggested sources such as Windows Event Viewer, Syslog files or live remote server(s), UNIX-style configuration files, Windows Registry, Web Server log files and process the data to identify machines and user accounts that have, are actively performing or may have performed a suspicious activity. There are no limitations on the implementation of the participant solution save for the fact that the must produce some type of alert, HTML page, file on disk, or log message(s) that will indicate details about the events they suspect are suspicious or malicious. Participants should plan on operating their own solution to avoid having to train any DreamPort or US Government personnel on the operation of their solution or interpretation of the data. Judges will be looking for some unique identifier in the output from a participant solution that ties back to a user account, MAC or IP address of an malicious party within HQ.

Participants must bring their own machine(s) to plug into the HQ network, however they may request software installation (e.g. log forwarding) on an HQ asset. Participants are responsible for the configuration of their own solution, no DreamPort person will configure a participant solution. A participant will be given time to ensure their solution(s) are receiving UAM data properly before any judging round officially begins. Depending on the number of participants, we will limit the number of UAM solutions monitoring HQ at any one time reserving the right to restrict active participants in any single round to a single party. Aside from interested participants the final limit on active monitoring systems on HQ depends on the methods by which said participant can ingest available UAM source data files. Subsequent groups will then connect to HQ and monitor the exact same malicious events as all previous participants. At the completion of a round, certain log files will be archived to disk and made available for all parties to consume for additional training.

DreamPort defines the following activities as malicious:

  • Elevating privilege level to Administrator or root (including using an exploit)
  • Installing keylogging or malicious software on machines for sniffing or dumping passwords
  • Installing malicious hardware for keystroke sniffing
  • Copying files from file share to external device or attempting to exfil via Internet connection(s)
  • Brute-Force password guessing (manual or automated)
  • MiTM attacks using malicious software (e.g. Cain & Abel, Ettercap, Scapy)
  • Executing remote access software (e.g. PowerShell Empire) 'smuggled' in via USB
  • Performing denial of service attacks

DreamPort defines the following activities as suspicious:

  • Installing software for concealing web traffic
  • Google searches for malicious software
  • Visiting malicious software websites
  • Printing sensitive data
  • Installing remote-access software (e.g. TeamViewer)
  • Enabling PS-Remoting with no encryption
  • Inserting USB on machines that do not normally have any inserts
  • Authenticating to domain controller as Administrator from non-standard machine(s)

Results

  • Best Performance with Agent: Jazz Networks
  • Best Performance without Agent: IBM
  • Highest # of Detections: Jazz Networks
  • Lowest False Positive Rate: Booz Allen
  • Best Performance by Category:
    • Obvious plain text threats: Jazz Networks
    • Malicious/suspicious USBs added to network: Jazz Networks
    • Foreign computer(s) on network or added to network: Jazz Networks
    • External connections: LogicHub
    • Incidents after loss of log data: Jazz Networks
  • Best Overall Performance:
    Jazz Networks

Congratulations to all of the participants!