About active responses in OSSEC

I already wrote about OSSEC's active response feature, and I said that I'm going to write a bit more after I study a bit more thorougly how it works. After I did analysis of log collecting feature of OSSEC, I decided to finally look at this too. So, here is the essey about what I found out by analysing code of OSSEC 2.6 with respect to active response. This essay will intergrate also the previous post, so, you don't have to read that previous one if you didn't already. Since active responses are bound to rules - triggering rule triggers active response - I'll touch also on rules, but not in a lot of details. Only as much is necessary to explaing active response.
I'll start with the purpose of active response, then I'll continue to configuration of active response, and I'll finish with implementation details and conclusions.

Purpose of active response

The idea behind active response in OSSEC's is to activelly block potential offenders and in that way to allow system to defend itself instead of passively monitoring what's happening. In some way this classifies OSSEC also as an IPS. As with all other IPSes, the feature is great but can bite you in case you are not careful, or, if you are using it without realising how it actually works. In the current implementation of OSSEC, active response is restricted to act on offending IP addresses and users.

Configuring active response

Active response configuration consists of two parts. In the first part you define all the commands, and in the second part - active response - you define how and when they are called. When writing configuration you should first list all of you commands, and then active responses that reference them. The reason is the way how the code was written, i.e., you will get undefined command error in case you don't list commands before they are used.
Note that final component, and important, connection between rules and active responses. Namely, rules are those that get triggered, and if they are triggered then active responses bound to them will be activated. In this post, we'll use manual method of activation, but I'll also describe how active responses are connected to rules.
It is interesting, and important, that when the configuration files are analyzed, the code writes all enabled active responses into the file shared/ar.conf, which is placed underneath the main OSSEC directory (by default it is /var/ossec). We'll come to exact format and purpose of that file later.
The following text is based on the analysis of config/active-response.[ch], analysisd/active-response.[ch] files, util/agent_control.c and some others.

Defining commands

All the commands that will be run as the part of the active response should be defined using <command> element placed directly beneath <ossec_config> top-level element. The structure should look something like this:

<ossec_config>
    ...
    <!-- Definition of command 1 -->
    <command>
    </command>
    ...
    <!-- Definition of command N -->
    <command>
    </command>
    ...
</ossec_config>

Each <command> element can have the following subelements:

• name - This will be identifier by which the command will be referenced in active response definitions, i.e. the second part. Obviously, this is a mandatory subelement. You shouldn't use dashes here because the way parsing is done throughout OSSEC and that will most likely confuse it! It is also advasible to avoid using spaces for the same reasons!
• expect - Mandatory to specify. The value can be either user or srcip. Due to the use of OS_Regex, and the way it calls OSRegex_Compile, those two strings are case insensitive, i.e. you can capitalize them as you wish, the meaning is the same.
• executable - Mandatory element, defines the exact name of the executable. The path where the executable is placed is predefined during compilation time. Default value is active-response/bin.
• timeout_allowed - Allowed values are yes and no. Basically, this tells OSSEC if the effect of the command (after it is called) should be reversed, i.e., it's effect have to be cleared by new invocation. This is optional and default value is no. To signal to a script if it is called for the first time, or as a part of a reversal process, the first argument to the script will be add or delete.

Defining active responses

The second part of the configuration defines active responses themselves. Active responses are configured (again) in OSSEC's configuration file using <active-response> elements that are placed directly within <ossec_config> top-level element, like this:

<ossec_config>
    ...
    <!-- Definition of active response 1 -->
    <active-response>
    </active-response>
    ...
    <!-- Definition of active response M -->
    <active-response>
    </active-response>
    ...
</ossec_config>

Under each <active-response> element the following subelements can be used to more specifically define active response:

• command - string value that references one <command> element. This is mandatory to specify.
• location - one of the following values: AS, analysisd, analysis-server, server, local, defined-agent, all, or any. AS, analysisd, analysis-server, server are synonyms, and so are all and any. You must specify this.
• agent_id - in case the value of the location is defined-agent, then this defines which agent and is mandatory to specify.
• rules_id - comma separated list of rule IDs to which this active response should be bound. This is an optional element without a default value.
• rules_group - name of a rule group to which this active response will be bound.
• level - Integer between 0 and 20, inclusive. If some rule has level equal to or grater than this value than it is bound to active response.
• timeout - Numerical value without constraints.
• disabled - Can have values yes or no.
• repeated_offenders - This element and its value are ignored.

Note one interesting thing. Namely, there is no ID bound to a specific active response. This means that the relationship between command and active response is 1:1. It seems somehow meaningless, beacuse, it is not possible to have some command (i.e. one <command> element) bind to multiple active responses commands each with a different set of parameters. Or, I'm missing something important?
When OSSEC agent starts it reads and parses <command> and <active-response> elements and writes shared/ar.conf file (actually, when any component of ossec starts if the previous paragraph is true). In that file each line defines one active response. Lines have the following printf like format:

%s - %s - %d

The first string is name of the active response and of the command (becaues active response doesn't have it's own name, as I explained). The second string is the executable. Finally, the third is a timeout value if it is defined. If not, 0 will be written in that field.

Manually triggering active responses - execution flow

To manually test how active response works you can use the utility agent_control whose source can be found in util/ top-level directory of OSSEC source. Note that only responses that require IP address can be activated. This tool should be run on a manager node and it actually connects to local ossec-remoted instance and sends it the appropriate commands that will trigger requested active response.
To trigger active reponse, using the utility, you have to specify three things:

1. Offending IP address (option -b).
2. ID of an agent where active response will be triggered (option -u).
3. Which active response to run (option -f).

For example, to tell an agent whose ID is 003 that it should run firewall-drop600 active response and pass it IP adress 192.168.1.1, you would invoke agent_control in the following way:

agent_control -u 003 -f firewall-drop600 -b 192.168.1.1

What will happen than is that agent_control will connect to the local ossec-remoted instance and send it the following command:

(msg_to_agent) [] NNS 002 firewall-drop600 - 192.168.1.1 (from_the_server) (no_rule_id)

The interesting thing here is that there is possibility to run active response on all nodes but this functionality is not enabled in agent_control utility. It boils down not to specify agent ID, in which case all agents are assumed and the previous command would look now like this:

(msg_to_agent) [] ANN (null) firewall-drop600 - 192.168.1.1 (from_the_server) (no_rule_id)

ossec-remoted accepts message from agent_control utility and based on that message sends a message over the network to agent. This message is encrypted, but in our case (running firewall-drop600 with IP address 192.168.1.1 as an argument) it will have the following format:

#!-execd firewall-drop600 - 192.168.1.1 (from_the_server) (no_rule_id)

The three characters at the beginning (#!-) are message header, and "execd " is a command. More commands can be seen in headers/rc.h file. Note that there is no agent ID in the message. That is because this information is used only by ossec-remoted so that it knows to whom it should send the message. So, no matter if you sent the message to some specific agent, or all of them, the message will look the same.
The message is received and preparsed by ossec-clientd in function receive_msg (in file client-agent/receiver.c) and then sent to ossec-execd (in top-level directory os_execd) via local message queue starting with a letter f, i.e. message header and command are stripped off by ossec-clientd.
It is interesting that ossec-execd, when started, reads standard configuration file (in XML) and, based on active response configuration, creates shared/ar.conf (unless changed during compilation). Then, it enters main loop, and there when active response is triggered for the first time, it reads shared/ar.conf instead of main XML configuration file. It has also one additional interesting behavior. Namely, if unknown active response is triggered it will first reread shared/ar.conf and if the active response isn't found then it will signal error. This, I suppose, can be used to dynamically update a set of active responses.
But, back to our example. ossec-execd receives message and assumes that everything to the first space is the command name. Using that token it then searches for a command. There are two interesting thing here:

1. The list of commands is taken from shared/ar.conf file that is dynamically created, not from ossec.conf or some similar.
2. If the command isn't found, then the shared/ar.conf is read again, and the search is peformed again. Only now, if there is no command, an error is reported and processing of a message is stopped. Note that no return message is sent to manager!

When the command is found, standard execve(2) structures are created, but twice. One for the first execution and the other for timeout value. Only later it is checked if timeout is supported/requested by command, and if it is not then memory is released. The arguments are copied from the message (spaces are used to separate arguments) with additional argument inserted first. Namely, add is inserted if the command is called first time, while delete is inserted in case it is called as a consequence of a timeout.
Finally, before executing command, one more check is performed. Namely, if the command was run, and timeout is hanging waiting to trigger, then command will not be called again, nor new timeout value will be called. The message from the manager will be ignored!

Automatic triggering of active responses

An active response is tirggered when the rule it is bound to is triggered. A single rule can have multiple active responses attached to it. Note that there are three ways in which you can bind rule and active response:

1. Using level element in active response. Namely, if the rule has attached level that is greater than or equal to the level of active response then the active response is attached to the rule.
2. If a rule ID is listed in the rules_id element of the active response, then the active response is attached to the rule.
3. If the group name to which rule belongs is listed in the rules_group element of active response, than they are bound together. The check is done very simply, namely it only checks that the value specified in rules_group occurs within group name. For example, if group name is "syslog,pam" then rules_group value "syslog" will match, as well as "pam", but "apache" will not. Note that even the value "syslog,pam" will match!

Rules and active responses are bound together (the code in analysisd/rules.c) when the ossec-analysisd is started.

Some implementation details

Configuration reading and parsing is implemented in the config/active-response.c file.
The data contained in each <active-response> element is kept in a structure active_response defined in src/config/active-response.h file. The elements of that structure quite closely mimic the configuration layout, i.e. this structure is defined as follows:

/** Active response data **/
typedef struct _ar{
    int timeout;
    int location;
    int level;
    char *name;
    char *command;
    char *agent_id;
    char *rules_id;
    char *rules_group;
    ar_command *ar_cmd;
}active_response;

That's it, not much. :)

Conclusion

Ok, I think I managed to document active response in OSSEC at least better then the majority of the existing documents. Tell me what you think. :)
Next, let me say that I'm impressed with XML parsers shipped with OSSEC. It seems to be very featerfull even though it is written in C, which is usually not used for XML processing.
One of the interesting things of OSSEC is that it reads the same configuration files multiple times. For example, in ossec-analysisd deamon, the configuration file is first read by active response initialization code (active-response.c) and then by the deamon itself. The reason why is this so interesting is that, during parsing of element <active-response> the file shared/ar.conf is written each time. In other words, the operation isn't idempotent. This is also problematic from the performance reasons standpoint, but probably less so.
While reading the code I sumbled on several interesting things, not directly related to the topic of this post. But since they are important, I'll mention them here and maybe I devote some future post to some of them:

• Top level directory addagent is misleading. It actually contains binary manage-agents which is much more encompassing then that. So, it should be renamed.
• In the top-level directory util/ there are some tools that should be moved or integrated with manage-agents utility.