Skip to end of metadata
Go to start of metadata

-

C2 Protocol Introduction

Command and control, herein C2, consists of a C2 server and C2 agents. MiNIFi agents must adhere to the C2 protocols to have successful communications. C2 communications occur over a variety of protocols. Currently an HTTP/HTTPS RESTFul paradigm exists to support C2 capabilities to MiNiFi C2 agents. In the future additional protocols may become available for use. Note that when the phrasing "C2 designator" is used, this implies the C2 server, server, or agent that is designated as a responder to the C2 agent(s). All protocols must support the following operations:

 

Operation NameDescription

ACKNOWLEDGE

Operation used by MiNiFi C2 agents to acknowledge the receipt and execution of a C2 server requested operation

CLEAR

Clears flow connection queues or repositories on the C2 agent
CONSUME

Consumes a heartbeat with an agent to avoid sending from other agents. Allows multiplexing responses from a

condensed agent response.

DESCRIBE

Currently Unused
EXECUTEExecutes commands on the agent's operating system. This feature may be disabled for any agent.

HEARTBEAT

Heartbeat provides status and operational capabilities to C2 server(s)

UPDATE

Updates components of the C2 agent or the flow configuration.
REPLICATEReplicates agent state between agents, with the ability to place agents in standby mode until they are needed.

RESTART

Restarts C2 agents

START

Starts components within the C2 agents
STOP Stops components within the C2 agent
SUBSCRIBEAllows servers or agents to subscribe to an agent's heartbeat , requesting specific information for the next
heartbeat.
TRANSFERTransfers an object between the C2 agent and C2 designator.

C2 Requirements

The requirements are an evolving list that have grown organically from an implementation

C2 RequirementJustification and Purpose
C2 agents shall report C2 status at regular intervals through a heartbeat messageAgents must employ heartbeat messages that follow an interval that is favorable to the agent ( power )
C2 agents shall report the flow version within the heartbeat message at regular intervalsAgents must report flow version to the C2 server
C2 agents shall report queue status within the heartbeat message at regular intervalsAgents must report queue status to the C2 server at regular intervals
C2 agents shall execute acknowledge commands sent via a heartbeat responseAgents must execute and acknowledge commands from the C2 server
C2 agents shall apply requested changes and inform the C2 server of success or failureAgents must apply and acknowledge updates from the C2 server, responding with a success or failure
C2 agents shall implement clear, update, restart, start, stop, and transfer commandsAgents must implement the prescribed commands.

C2 Messages 


Heartbeats

   Primary communications are carried over a C2 heartbeat. The heartbeat contains operational information about the C2 agent and can occur a configurable frequency. The heartbeat provides status information to the C2 server. The response from the heartbeat contains requested operations from the C2 server. These operations are then acknowledged if/when they are completed. This means that the heartbeat is the only operation initiated by the C2 agent and the C2 server responds directly to these heartbeats. Version four of the heartbeat will allow a subscription model to be used for heartbeats to avoid sending unnecessary information. Though Describe can provide parts of the same information the aggregate produced for a heartbeat allows the agent to flush messaging queues to ensure subscribed heartbeats have the most up to date information.

Protocols

HTTP/S Protocol

  The HTTP/S protocol supports a url for heartbeating and acknowledging operations. These endpoints support the JSON structures defined below. C2 agents must send a heartbeat, defined above, to update the C2 server of its status and to receive operations. The frequency of these calls are up to the C2 agent to define. 

Heartbeat structure

Heartbeats consist of a POST of the following Schema to the C2 heartbeat url. Metrics is a configurable list of metrics that can be returned, so the entirety of that object is optional.

Update: current work uses a flow identifier to help identify the currently running flow within DeviceInfo. 

Key Value
AgentInformation

Key Value
AvailableClasses

SupportsDynamicProperties Class_name Properties
false org::apache::nifi::minifi::processors::AppendHostInfo

 
property

false org::apache::nifi::minifi::processors::AppendHostInfo

 
property

BuildInformation

Key Value
Build_date date
Build_rev string
Build_version string
Compiler string
Compiler_flags string
Device_id string

Extensions

 
extension1
extension2
extensionn

NetworkInfo

Key Value
Deviceid string
Flowid string
Hostname string
Ip string

SystemInformation

Key Value
Machinearch string
Physicalmem string
Vcores string

Components

Key Value
FlowController enabled
ProcessorName enabled/disabled

Metrics

Key Value
ProcessMetrics

Key Value
CpuMetrics

Key Value
Involcs string

MemoryMetrics

Key Value
Maxrss string

QueueMetrics

Key Value
Connection

Key Value
Datasize string
Datasizemax string
Queued string
Queuedmax string

RepositoryMetrics

Key Value
Flowfile

Key Value
Full 1/0
Running 1/0
Size string

Provenance

Key Value
Full 1/0
Running 1/0
Size string

Operation heartbeat
State

Key Value
Running true/false
Uptime string

Responses to the heartbeats have the following structure
Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation Content
string string string

String
string


 

Operation schemas

The following are the schema definitions for each operation that is contained within the requested operations of a heartbeat response. It is expected that C2 agents adhere to this structure
 
Clear
 
 The clear operation uses name of connection or repositories to clear either the connections or the repositories. In the case of a connection the content contains the operation arguments, in which the value defines the connection name to clear.


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation Content
connection string clear

Unique_map_key
connection_name

Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation
repositories string clear


 
Update
 
Update allows the C2 server to update either the c2 agent or provide a URI from which we download the new flow configuration through a GET request. 
 


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation Content
configuration string update

Location
HTTP or HTTPS URL


 

The following activity diagram depicts the flow of updating an agent from failure to success. 

Update Agent Activity Diagram

 
 


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation Content
c2 string update

Option_name
option_value


 
Start
 Start starts a previously stopped command. If a start is called on a component that is already started, nothing should occur other than an acknowledgement. Name defines the component to start. 
 


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation
component name string start



Stop
 Stop stops a component that is started. Components can be the FlowController, processors, or RPGs
 


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation
component name string stop


Transfer
 Transfer will download an object from the C2 designator ( server or other location ). The location URI will be provided by the update JSON. This transfer will not be a JSON
response but will instead be the binary object. The hash of the object will be the acknowledgement ID for the transfer. 


Key Value
Operation acknowledge
Operationid hash of object

Restart
  Attempts to restart the component defined within name


Key Value
Operation heartbeat
Requested_operations

Name Operationid Operation
component name string restart

 


Acknowledgements.

 Acknowledgements occur through a separate URL. This URL will receive a POST that contains the following payload, which acknowledges that the operation ID was received and executed.

Key Value
Operation acknowledge
Operationid string


MQTT Protocol

MQTT can be used as a connecting protocol in lieu of a RESTFul Service. Additionally, MQTT can be used within an enclave and then as conversion to RESTFul to support MQTT → HTTP comms. 

Operations and versioning

Apache NIFi MiNiFI C++ automatically supports versions 4, 3, and 2. Version one is not supported by the agent. Agents are backwards compatible and forward compatible. 

C2 role delegation

Since C2  defines operations for agents and a conversion protocol, agents can be used for role delegation. This means that Agents can act as a C2 server. This design is intentional, and supports the eventual automation of capabilities. Version four

begins implementing facilities that will be used for autonomous control. Agents can control each other and execute commands based on defined policies. Policies are controller services and configurations that enable activities to be that allow

autonomous and near-autonomous control of subnets of agents. 

C2 Policies

This sections defines the sub pages for C2 policies defined as controller services. 

C2 Failure Policies

C2 Update Policies

Operations and their operands for agents (Version 4* Not released)

Operation NameDescriptionoperand/namecontent/args

ACKNOWLEDGE

Operation used by MiNiFi C2 agents to acknowledge the receipt and execution of a C2 server requested operation N/A
CLEARClear repositoriesrepositoriesN/A

CLEAR

Clears the connection queuesconnection

connection1=<connection name>, connection2=<connection 2>  ...

 

Will also accept a list

<connection name1>,<connection name2>, ...

CONSUMEConsumes a heartbeat within an agent to avoid sending from other agentsN/AN/A

DESCRIBE

Return metricsmetricsmetricsClass=<metric class to obtain>
DESCRIBEconfigurationN/AN/A
DESCRIBEmanifestN/AN/A
DESCRIBEpolicy events – Based on the defined policies

EXECUTEExecutes commands per the agent's defined policiescommandarguments
HEARTBEATheartbeat operation – may contain embedded heartbeats.

HEARTBEATnonce of combined heartbeats

UPDATE

Update flowconfigurationlocation=<URL to updated flow file>
UPDATEUpdate c2 propertiesc2

configkey1=configvalue1, configkey2=configvalue2 ...

*configkey1 is a configuration option that is updated and its new value

UPDATEUpdate configuration options defined within agent policies

UPDATEUpdate agentagent

location=<URL to agent binary or diff>

partial=true/false ( optional)

RESTART

Restarts C2 agentsN/AN/A
REPLICATEReplicates an Agent's state to another agent; with standby true the replicant is paused and awaits restart.agentstandby=true/false
REPLICATETells agents to replicate state to nearby agentsserver

START

Starts components within the C2 agents<name of component to start>N/A
STOPStops components within the C2 agent<name of component to stop>N/A
SUBSCRIBE

Subscripts a C2 server to internal respondables ( Metrics , configuration, and policy/audit events ) .

These will be placed into the heartbeat

enable/disablesubscribe=metrics, subscribe=configuration, subscribe=auditevents
TRANSFERTransfers an object between the C2 agent and C2 designator.N/AN/A

Operations and their operands for agents (Version 3)

Operation NameDescriptionoperand/namecontent/args

ACKNOWLEDGE

Operation used by MiNiFi C2 agents to acknowledge the receipt and execution of a C2 server requested operation N/A
CLEARClear repositoriesrepositoriesN/A

CLEAR

Clears the connection queues
connection

connection1=<connection name>, connection2=<connection 2>  ...

 

Will also accept a list

<connection name1>,<connection name2>, ...

DESCRIBE

Return metrics
metricsmetricsClass=<metric class to obtain>
DESCRIBEconfigurationN/AN/A
DESCRIBEmanifestN/AN/A

UPDATE

Update flow
configurationlocation=<URL to updated flow file>
UPDATEUpdate c2 propertiesc2

configkey1=configvalue1, configkey2=configvalue2 ...

*configkey1 is a configuration option that is updated and its new value

UPDATEUpdate agentagentlocation=<URL to agent binary or diff>

RESTART

Restarts C2 agentsN/AN/A

START

Starts components within the C2 agents<name of component to start>N/A
STOP Stops components within the C2 agent<name of component to stop>N/A
TRANSFERTransfers an object between the C2 agent and C2 designator.N/AN/A

Operations and their operands for agents (Version 2)

Operation NameDescriptionoperand/namecontent/args

ACKNOWLEDGE

Operation used by MiNiFi C2 agents to acknowledge the receipt and execution of a C2 server requested operation N/A
CLEARClear repositoriesrepositoriesN/A

CLEAR

Clears the connection queuesconnection

connection1=<connection name>, connection2=<connection 2>  ...

 

Will also accept a list

<connection name1>,<connection name2>, ...

DESCRIBE

Return metricsmetricsmetricsClass=<metric class to obtain>
DESCRIBEconfigurationN/AN/A

UPDATE

Update flowconfigurationlocation=<URL to updated flow file>
UPDATEUpdate c2 propertiesc2

configkey1=configvalue1, configkey2=configvalue2 ...

*configkey1 is a configuration option that is updated and its new value

UPDATEUpdate agentagentlocation=<URL to agent binary or diff>

RESTART

Restarts C2 agentsN/AN/A

START

Starts components within the C2 agents<name of component to start>N/A
STOPStops components within the C2 agent<name of component to stop>N/A

Operations and their operands for agents (Version 1)

Operation NameDescriptionoperand/namecontent/args

UPDATE

Update flowconfigurationlocation=<URL to updated flow file>

Future Work

Future architecture of C2 should be open to the discussion of distributed architectures and multiple heads ( i.e. in a client server multiple client/servers in the case where we can talk to geographically distributed agents ). 

 

  • GPS based C2 server – using location information to identify and locate C2 servers.
    • C2 specific DNS responses.  could we use DNS to simply provide us the closest C2 server?
  • Multiple C2 servers providing separate keys to act as security arbiters.

 

 

 

  • No labels

1 Comment

  1. Example response:

    {
        "operation": "heartbeat",
        "metrics": {
            "ProcessMetrics": {
                "CpuMetrics": {
                    "involcs": "1609"
                },
                "MemoryMetrics": {
                    "maxrss": "19337216"
                }
            },
            "QueueMetrics": {
                "GetFile/success/LogAttribute": {
                    "datasize": "0",
                    "datasizemax": "1073741824",
                    "queued": "0",
                    "queuedmax": "10000"
                }
            },
            "RepositoryMetrics": {
                "flowfilerepository": {
                    "full": "0",
                    "running": "1",
                    "size": "0"
                },
                "provenancerepository": {
                    "full": "0",
                    "running": "1",
                    "size": "0"
                }
            }
        },
        "AgentInformation": {
            "AvailableClasses": [
                {
                    "properties": [
                        "Hostname Attribute",
                        "IP Attribute",
                        "Network Interface Name"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::AppendHostInfo",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Max Bin Age",
                        "Maximum Group Size",
                        "Maximum Number of Entries",
                        "Maximum number of Bins",
                        "Minimum Group Size",
                        "Minimum Number of Entries"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::BinFiles",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Compression Format",
                        "Compression Level",
                        "Mode",
                        "Update Filename"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::CompressContent",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Batch Duration",
                        "Command",
                        "Command Arguments",
                        "Redirect Error Stream",
                        "Working Directory"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ExecuteProcess",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Module Directory",
                        "Script Body",
                        "Script Engine",
                        "Script File"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ExecuteScript",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Attribute"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ExtractText",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Path"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::FocusArchiveEntry",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Batch Size",
                        "Data Format",
                        "File Size",
                        "Unique FlowFiles"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::GenerateFlowFile",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Batch Size",
                        "File Filter",
                        "Ignore Hidden Files",
                        "Input Directory",
                        "Keep Source File",
                        "Maximum File Age",
                        "Maximum File Size",
                        "Minimum File Age",
                        "Minimum File Size",
                        "Polling Interval",
                        "Recurse Subdirectories"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::GetFile",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "SSL Context Service",
                        "Stay Connected",
                        "concurrent-handler-count",
                        "connection-attempt-timeout",
                        "end-of-message-byte",
                        "endpoint-list",
                        "receive-buffer-size"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::GetTCP",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Always Output Response",
                        "Attributes to Send",
                        "Connection Timeout",
                        "Content-type",
                        "Disable Peer Verification",
                        "HTTP Method",
                        "Include Date Header",
                        "Proxy Host",
                        "Proxy Port",
                        "Read Timeout",
                        "Remote URL",
                        "SSL Context Service",
                        "Use Chunked Encoding",
                        "invokehttp-proxy-password",
                        "invokehttp-proxy-user",
                        "send-message-body"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::InvokeHTTP",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Battery Capacity Path",
                        "Battery Status Path",
                        "Low Battery Threshold",
                        "Trigger Threshold",
                        "Wait Period"
                    ],
                    "class_name": "org::apache::nifi::minifi::controllers::LinuxPowerManagerService",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Authorized DN Pattern",
                        "Base Path",
                        "HTTP Headers to receive as Attributes (Regex)",
                        "Listening Port",
                        "SSL Certificate",
                        "SSL Certificate Authority",
                        "SSL Minimum Version",
                        "SSL Verify Peer"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ListenHTTP",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Max Batch Size",
                        "Max Number of TCP Connections",
                        "Max Size of Socket Buffer",
                        "Message Delimiter",
                        "Parse Messages",
                        "Port",
                        "Protocol",
                        "Receive Buffer Size"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ListenSyslog",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Attributes to Ignore",
                        "Attributes to Log",
                        "Log Level",
                        "Log Payload",
                        "Log prefix"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::LogAttribute",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "After",
                        "Before",
                        "Destination",
                        "Operation",
                        "Target"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::ManipulateArchive",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Correlation Attribute Name",
                        "Delimiter Strategy",
                        "Demarcator File",
                        "Footer File",
                        "Header File",
                        "Keep Path",
                        "Max Bin Age",
                        "Maximum Group Size",
                        "Maximum Number of Entries",
                        "Maximum number of Bins",
                        "Merge Format",
                        "Merge Strategy",
                        "Minimum Group Size",
                        "Minimum Number of Entries"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::MergeContent",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "Conflict Resolution Strategy",
                        "Create Missing Directories",
                        "Directory",
                        "Maximum File Count"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::PutFile",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "CA Certificate",
                        "Client Certificate",
                        "Passphrase",
                        "Private Key"
                    ],
                    "class_name": "org::apache::nifi::minifi::controllers::SSLContextService",
                    "supportsDynamicProperties": "false"
                },
                {
                    "properties": [
                        "File to Tail",
                        "Input Delimiter",
                        "State File"
                    ],
                    "class_name": "org::apache::nifi::minifi::processors::TailFile",
                    "supportsDynamicProperties": "false"
                },
                {
                    "class_name": "org::apache::nifi::minifi::processors::UnfocusArchiveEntry",
                    "supportsDynamicProperties": "false"
                },
                {
                    "class_name": "org::apache::nifi::minifi::processors::UpdateAttribute",
                    "supportsDynamicProperties": "true"
                }
            ],
            "BuildInformation": {
                "build_date": "Tue Mar  6 09:20:53 EST 2018",
                "build_rev": "9b01c508e53c3108d4f9273b2751c7311648aece",
                "build_version": "0.5.0",
                "compiler": "/Library/Developer/CommandLineTools/usr/bin/c++",
                "compiler_flags": "  -std=c++11 -Wall",
                "device_id": "classone"
            },
            "Extensions": [
                "minifi-archive-extensions",
                "minifi-civet-extensions",
                "minifi-expression-language-extensions",
                "minifi-http-curl",
                "minifi-rocksdb-repos",
                "minifi-script-extensions"
            ],
            "NetworkInfo": {
                "deviceid": "2166229772469413744",
                "hostname": "MachineName",
                "ip": "IP"
            },
            "SystemInformation": {
                "machinearch": "x86_64",
                "physicalmem": "17179869184",
                "vcores": "8"
            }
        },
        "Components": {
            "FlowController": "enabled",
            "GetFile": "enabled",
            "LogAttribute": "enabled"
        },
        "state": {
            "running": "true",
            "uptime": "2889"
        }
    }