OSSEC active response...

This is still work in progress (I need to add more about configuration part). But since OSSEC is so badly documented and I don't know when this will be finished I'm publishing it now.

Prompted by the problem caused by the OSSEC active response, I decided to try to debug why an error is occurring in logs and fine tune it. But in order to do that I had first to understand how it works. There is a section in OSSEC manual about active response, but what wasn't immediately clear to me is who runs active response and where this is configured. But soon I found out that the active response is run on the client via agent (actually this part wasn't problematic) but that active response is configured on the server and the server is the one that instructs agents to run active response.

On server you'll find active response configuration in $OSSEC/etc/ossec.conf. In that file you have several parts of the configuration:

1. Set of <command> blocks. Each one defines a command for active response and the arguments it expects. Note that those commands have to exist on agents.
2. Set of <active-response> blocks that define circumstances under which there will be active response and which command will be executed.

Scripts for active response

Scripts for active response receive five arguments. The first argument is either add or delete. If add is specified, given IP address should be added to a ban list, in case delete is specified address should be removed from the ban list. The second argument is a user name. The third argument is offending IP address. Fourth argument is Unix timestamp (microsecond resolution) when the script was called. Final argument is rule number that triggered active response.

firewall-drop.sh script

This script ships with OSSEC and it adds or removes IP addresses from firewall. It is a relatively simple shell script that accepts command line arguments as specified in the introduction of this section and installs IP address in the ban list (or removes it from there, depending on the command line arguments). The script could be run on Linux, FreeBSD, NetBSD, Solaris and AIX. The following description is specific to Linux behavior even though some things will be common across different platforms.

This script logs its activity into active-responses.log file (in my case in /var/ossec/logs directory). For each invocation one line is emitted into that file. Here is an example of one log entry:

Thu Jan 19 13:52:29 CET 2012 /var/ossec/active-response/bin/firewall-drop.sh add - 193.41.36.141 1326977549.1358625 3301

First group of fields is timestamp when the log entry was generated. Next is a full path and name of the script itself. Finally, all five arguments given to script are also recorded.

Ocasionally, you'll also see error messages like the following one:

Thu Jan 19 06:17:19 CET 2012 Unable to run (iptables returning != 1): 1 - /var/ossec/active-response/bin/firewall-drop.sh delete - 208.115.236.82 1326949601.551727 3301

This log entry is a bit misleading. What happened is that iptables command returned exit code 1 (judging from the log entry it could be interpreted as if it returned something else and 1 was expected, but that's not true). What is important to note is that you'll usually see multiple log entries like the previous one grouped together and the only thing that will differ between them is the number 1 (shown in bold above). What basically happens is that in case of an error returned by iptables command the script tries to run it five times, so, you'll usually see five records and each of those records is numbered.

There are two places where this error might occur. The first one is when the IP address is removed from INPUT chain, while the other is when it is removed from FORWARD chain.

The only reason this error can occur is because someone or something already removed IP address (or added it). But, this should not happen. Still, it happened to em but I don't know the reason for that.

Looking into this script it was clear that it could be improved from the logging perspective. Currently, if you manually run this command from the command line it will write part of error messages to stdout and some other to log file.

Manual control of scripts

Scripts for active response can be started from the server using agent_control tool. Note that the help message of this tool isn't updated to reflect real arguments (bug?) so I had to look into source to infer how to call it. Let me give you several examples of its use.

To list all available agent use this command in the following way:

# ./agent_control -l
   ID: 000, Name: agent0.somedomain.local (server), IP: 127.0.0.1, Active/Local
   ID: 001, Name: agent1.somedomain.local, IP: 192.168.1.2, Active
   ID: 002, Name: agent2.somedomain.local, IP: 192.168.1.3, Active
   ID: 003, Name: agent3.somedomain.local, IP: 192.168.1.4, Disconnected

The output of the command is a list of known agents, either active or non-active. In case you want only active agents use -lc option instead.

Next, if you want to find out some information about a certain agent, you can query it in the following way:

# ./agent_control -i 002

OSSEC HIDS agent_control. Agent information:
   Agent ID:   002
   Agent Name: agent2.somedomain.local
   IP address: 192.168.1.3
   Status:     Active

   Operating system:    Linux agent2.somedomain.local 2.6.32-131.17.1.el6.x86_64..
   Client version:      OSSEC HIDS v2.5-SNP-100907
   Last keep alive:     Thu Jan 19 13:26:01 2012

   Syscheck last started  at: Thu Jan 19 12:10:16 2012
   Rootcheck last started at: Thu Jan 19 06:33:06 2012

To activate active response on a certain agent use the following form of the agent_control command:

./agent_control -b 1.1.1.1 -f firewall-drop600 -u 002

Here, the IP address to be blocked is argument of the -b option (in this case 1.1.1.1). There could be more responses available (defined in ossec.conf on server) and the option -f selects which one to run. To see which responses are available use the option -L, like this:

# ./agent_control -L

OSSEC HIDS agent_control. Available active responses:

   Response name: host-deny600, command: host-deny.sh
   Response name: firewall-drop600, command: firewall-drop.sh

Agent on which this command should initiate active response is specified via ID given as the parameter to option -u. Note that, if you look into help output of the agent_control, this option is not listed, at least not in the version 2.6.0. There is a bit of inconsistency here, as some commands use agent ID as a parameter, while this one requires separate parameter. It would be more uniform if all command would instead use -u option.

Note that when you manually initiate active response then fourth argument to the script will be '(from_the_server)' and the fifth argument will be '(no_rule_id)'.

Also, the rule that was added can not be removed with agent_control, you have to wait for it to timeout when it will be automatically removed.