Examples of ovaldi checks: sysctl variables

After I described how to compile ovaldi on CentOS and a simple test to verify it is working, in this post I'm going to describe how to use ovaldi to check the values of sysctl variables. More precisely, I'm going to check that IPv4 forwarding is turned off. The general idea behind this example is to give you a starting example on which you can build more complicated checks. Note that there is even more general idea. Namely, you can create your own security benchmarks that can check if certain security criteria are met, and if not, you can be alarmed by automatic monitoring process that is based on ovaldi.

In the text that follows I'm referencing the following file. That is a simple and complete file that will check the value of net.ipv4.ip_foward sysctl variable. After you've downloaded this file, and assuming that you have your environment properly configured (see posts I referenced at the beginning) then you can run ovaldi to do the check:

ovaldi -m -o sysctl-test.xml -a /opt/oval/share/ovaldi/xml

ovaldi will create the usual output files after running this command: ovaldi.log, results.xml, results.html and system-characteristics.xml. Each one of them you can open in a brower. You should open them and check their content. system-characteristics.xml is interesting because there you can find what information was collected about the system. Those values can be checked for in oval definitions XML file. Note that ovaldi collects only referenced data, not everything it could possibly collect.

Let us now dissect a bit oval definitions file, sysctl-test.xml. The basic structure of this file is:

<oval_definitions ...>
    <definitions>...</definitions>
    <tests>...</tests>
    <objects>...</objects>
    <states>...</states>
</oval_definitions>

Basically, what oval definition does is to define a series of test, each one describing what expected state of certain object is. Test themselves can be combined in many different ways using AND and OR operators and nesting.

In our simple example object whose state we are interested in is ip_forwarding variable. So, if you look into XML file, inside element, you'll find that we define object of interest:

<sysctl_test id="oval:hr.sistemnet.oval:tst:1"
   version="1"
   comment="forwarding is disabled"
   check="at least one"
   xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
  <object object_ref="oval:hr.sistemnet.oval:obj:1" />
  <state state_ref="oval:hr.sistemnet.oval:ste:1" />
</sysctl_test>

xmlns attribute is important. I had some problems with undefined element until I got that one correctly. In other words, all the objects, their state and tests are defined in XML documents in /opt/oval/share/ovaldi/xml directories. But, when using those be certain to correctly define namespace where they are defined, or otherwise ovaldi will complain that you are using unknown test, objects and/or states.

This particular test reference object that has to be checked and the state in which this object has to be. Object itself is defined in <objects> element as follows:

<sysctl_object id="oval:hr.sistemnet.oval:obj:1" version="1"
         xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
    <name>net.ipv4.ip_forward</name>
</sysctl_object>

As you can see, within name element you specify which sysctl variable you wish to check. The second part of the test is the state in which object has to be. We want our object to have value 0, meaning forwarding is disabled. That check is performed using the following within states element:

<sysctl_state id="oval:hr.sistemnet.oval:ste:1"  version="1"
      xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
    <value>0</value>
</sysctl_state>

Obviously, you just place desired value within value element.

So, we saw that test consists of an object that has to be in particular state. The test itself is referenced in definition element (which is placed within definitions element). For that purpose you are using criterion element:

<criteria operator="AND">
    <criterion test_ref="oval:hr.sistemnet.oval:tst:1" comment="forwarding is disabled" />
</criteria>

As you can probably guess, multiple criterions can be specified and in this case they will be bound with AND operator. You can nest criteria and criterion elements to get very complex tests.

Installing and testing ovaldi on Windows 7...

When you are dealing with a single computer with a particular operating system, it is relatively easy to keep it safe. But, as the number of machines grows and becomes more heterogeneous, keeping them safe becomes very donating task. You may have automated updates and such, but they have to be checked from time to time in order to see if they function correctly. Still, if those computers are used (and by definition they are, more or less frequently), then they are like living organisms, they change. No matter if you are tweaking particular installation because user requested some new functionality or he requested removal of something that annoys him, or you are trying to diagnose why something worked and now it doesn't work, you will change something. After you are finished, you might think that changes you've made won't influence anything and leave for some later time to reverse them, and eventually you'll forget about them. But, any unintended change might bring system into a risk. So, it is important to perform regular checks in order to spot changes. Since such checks are time consuming and error prone, it is a good practice to use some tool that will do it for you. That tool could be OVAL.

But even if you are not an system administrator, but e.g. auditor, you can also benefit from OVAL since checks that you have to perform could be in some way prescribed and automated. In that way you can check larger sample of systems and achieve better accuracy and confidence in obtained results than by manual checks.

OVAL is basically a language that describes checks to be made, more concretely, it's an application of XML. Those checks could be conditional (i.e. depend on a system under audit, or if a particular component is installed or not), and they can be grouped with operators like AND, OR and NOT. There are many existing checks defined, for example, here are latest additions and updates, while here are complete databases for download. The tests are provided by some vendors (like RedHat) and also by community. Finally, you can add your own checks customized to your particular environment.

In themselves, those checks are worthless without a proper tool that will execute them. And here we have open source reference implementation, Ovaldi. Some security vendors have their own versions, which of course cost money. Ovaldi, on the other hand, is free, but you are forced to use command line. Ovaldi interprets (in a way) given database and produces reports, in XML and HTML formats. HTML is great for viewing results, while XML for parsing and automating scans.

I was testing ovaldi on Linux before with mixed success, but now I decided to try it on Windows 7. The reason that I believe that its use on workstations and servers on a periodic basis will make those computers more secure, and, by extension, the whole system more secure. In the text that follows I'm going to describe a process of installing, manually running the tool and analyzing the results. Automated testings I'll leave for some future post.

Download and install Ovaldi

Download page for Ovaldi is here. Note that this will take you to the latest version at the time this post was written, i.e. 5.10.1. So, before downloading check if there is a newer version, and if is, use that one. Don't forget to change all the references from version 5.10.1 to your version in the text that follows.

Anyway, you'll find there EXE versions for Windows, so select one that suits your environment. In my case that was 32-bit version, but if you have 64-bit version of Windows, download that one instead.

The file you've downloaded isn't regular installation file, so to install it you have to follow a bit different procedure. After download finishes, left click on file you downloaded and select option Run as administrator. Winzip dialog will appear which will ask you where to unzip (i.e. install) the files. Enter C:\Program Files\OVAL, or anything you wish but don't forget to change reference to that directory in later text to the one you've entered. Click Unzip button, and that's it. Ovaldi is installed.

Environment setup

To be able to run ovaldi without typing the whole path to it, add it to the PATH environment variable. To do that, click on Windows menu (left bottom corner) and then do left click on Computer item. Select Properties item and in window that appears select Advanced System Settings (option on the left). New window appears and there you'll notice Environment Variables... button on the right bottom. Click on it and new window appears. In this window there is System variables pane. Find there variable PATH and click on Edit. At the end of the line add the following text:

;C:\Program Files\OVAL\ovaldi-5.10.1\

Be carefull not to erase existing values! Close all the windows by clicking on OK, and close final window (the one opened with Properties on Computer) by clicking on X in upper right corner. Now, open command prompt and enter ovaldi followed by return. If you get help message then everything is OK and you can proceed to the next step. Otherwise, review previous steps.

Download file definitions

Now you have interpreter and you need definitions that will be run by interpreter. Go to the following page. There you'll see section Downloads by Version and Namespace. You need to select class to download based on the version of oval interpreter you have. The following classes are available:

• compliance - checks that the installation is compliant with good security practices.
• inventory - checks that produce results of what is installed.
• miscellaneous
• patch
• vulnerability - test that verify if there is a vulnerability present on the machine.

When you click on one of those classes you are presented with a new page that gives you a list of available definitions grouped by different criteria. For example, by clicking on vulnerability class (probably the largest one) you can select download by platform, family or all. There are pros and cons of each one. If you select by family (or all) you don't have to think which platform you have, you get everything and oval interpreter will not be confused that, e.g. there are Windows XP specific checks and you are running on Windows 7. But, this commodity goes at the expense of the execution time.

For the purpose of initial testing of oval, I went to download by platform/vulnerabilities, and there I downloaded file microsoft.windows.7.xml which I renamed into microsoft.windows.7.vulnerabilities.xml. I also downloaded equivalent files from compliance and inventory classes naming them microsoft.windows.compliance.xml and microsoft.windows.inventory.xml, respectively. All those files I placed into working directory that, from now on, I'll reference by WORK_DIR identifier. So, whenever you see that string, replace it with the full directory path of your working directory.

Running Ovaldi and viewing results

Ok, lets do the first scan to see what we are going to get. To start scan, open terminal windows, go to your working directory, and run the following command (this is a single line!):

ovaldi -m -a "c:\program files\oval\ovaldi-5.10.1\xml" -o microsoft.windows.7.vulnerability.xml -r 20111025-result.xml -x 20111025-result.html -d 20111025-system-characteristics.xml

This command will check vulnerabilities that are present on the system it runs on. Of course, only vulnerabilities defined in the database (microsoft.windows.7.vulnerability.xml) will checked. If the tool reports that there are no vulnerabilities, it only means there are no known vulnerabilities! The other options are:

• Option -m. Don't check md5 sum of oval definitions file (in this case that is microsoft.windows.7.vulnerability.xml).
• Option -a specifies where all the auxiliary files necessary for interpreter are. For example, default style sheet file is there, also, XML definitions and tests are also there. The default value of this option assumes that you are running ovaldi in its base directory (i.e. where it is installed) so it has to be specified in order for everything to work.
• Option -o specifies oval definition file to use. 
• Option -r specifies XML result file. The default value is results.xml and in the case of multiple runs, default file name will be overwritten. So, using this option prevents that from happening.
• Option -x specifies HTML result file. This file is generated from XML result file by applying style sheet (XSL) file. Default file is used if none is specified on the command line.
• Option -d specifies in which file will be saved system characteristics, i.e. installed options, existing files, etc. used during interpreter run of oval definition file.

After this command finishes you'll have three new files in the directory in which you run it (provided no errors occured). All of the files can be viewed by Web browser (e.g. Mozilla Firefox) but only the file specified as the argument to -x option is specifically meant to be viewed in such way. XML files are primarily used for automated processing.

When you open results file (20111025-results.html if you used the command given above) then you'll see four section named OVAL Results Generator Information, System Information, OVAL System Characteristics Generator Information and OVAL Definition Results.

The largest one will be OVAL Definition Results, which is a table with 5 columns. This first column is ID of a test performed, the second is result of a test, either it is positive (true) or negative (false). Then there is a class of a test, either inventory (i.e. something is installed or not), vulnerability checked, reference ID that links you to the description of that particular item on the Internet, and finally title that gives a short description of item.

OVAL definitions for Fedora

Note: This post has been transferred from my old home page on January 4th, 2020 and the date of the publication is only approximate.

... or to be honest, a start of OVAL definitions for F9F14 and CentOS, i.e. work in (very long) progress. I'm just learning how to use this technology, and if it's of any use at all! :)
In short, OVAL is a XML based specification language for vulnerability assessment. It is already used by different vendors and it will probably see wider adoption as NIST is pushing OVAL as a part of automated solution for vulnerability assessment and management. For example, RedHat publishes vulnerability advisories in OVAL format.
I have to stress that I didn't yet fully learned the idea behind OVAL as well as the technology used so there is a high probability of errors in the following text. If you spot an error please mail me the correction!

What is OVAL and how to use it

As I already said, OVAL is a language that describes checks to be performed on a system in order to determine if any vulnerability is present on it, either to a software bug or to a configuration setting. This is performed via tool in a package called ovaldi, which is available in Fedora's RPM repository. So, you should install it as usual using the yum command. After installation process finishes, you'll have command line tool called, surprisingly, ovaldi! :) The next thing you need in order to use this tool are definitions of vulnerabilities. Unfortunately, there are no vulnerability descriptions for Fedora in OVAL form. I'll try to make few, for Fedora and CentOS. In case I missed some repository, please notify me via e-mail message! While we are at CentOS, it's possible that RedHat's OVAL definitions could be used with a little bit of hacking, but I didn't try it so it could prove false!
Anyway, in order to try ovaldi tool you need OVAL definitions. You can use RedHat's but all the results will be false which is expected as you are not running RedHat on your computer. So, in order for you to try oval, download this definition file I prepared. It is very simple OVAL definition and only checks which version of Fedora is installed on the computer, 8 or 9.

Running ovaldi

To run ovaldi it has to have schema definitions. Now, this is interesting as those are placed in the /usr/share/ovaldi directory but the tool looks for them in the current directory.
Even more interesting is that I can't seem to identify option that would allow me to change schema path. To get around this problem, copy all the content from the /usr/share/ovaldi directory into current directory. Be carefull to create separate working directory for this or otherwise you'll have a mess on your disk!
There are two ways to define where schema files will be searched. The first one is using the option -a. The default value for this option is /usr/share/ovaldi but for some reason this value is not used, i.e. ovaldi tool can not find schema files. The other, and not so good(!), approach is to encode path in the XML file itself. This approach will be described later.
Now, run the ovaldi tool as follows:

$ ovaldi -o fedora.9.oval.xml -m

Note that in real situations is could be possible that you'll have to run ovaldi as root since it could try to access data not accessible to ordinary users. In this case it is not necessary as the tests are very simple.
The output from the command on the Fedora 9 will be:

----------------------------------------------------
OVAL Definition Interpreter
Version: 5.4 Build: 2
Build date: Jun  7 2008 15:06:57
Copyright (c) 2002-2008 - The MITRE Corporation
----------------------------------------------------

Tue Jul  8 17:00:52 2008

 ** parsing fedora.9.oval.xml file.
    - validating xml schema.
 ** checking schema version
     - Schema version - 5.4
 ** skipping Schematron validation
 ** creating a new OVAL System Charateristics file.
 ** gathering data for the OVAL definitions.
      Collecting object:  FINISHED                        
 ** saving data model to system-characteristics.xml.
 ** running the OVAL Definition analysis.
      Analyzing definition:  FINISHED                        
 ** OVAL definition results.

    OVAL Id                                 Result
    -------------------------------------------------------
    oval:org.fedoraproject.oval:def:1       true           
    oval:org.fedoraproject.oval:def:2       false          
    -------------------------------------------------------


 ** finished evaluating OVAL definitions.

 ** saving OVAL results to results.xml.
 ** running OVAL Results xsl: results_to_html.xsl.

----------------------------------------------------

The part that is in bold shows the results of two tests. The one that is true is a test for Fedora 9, while the other one is the test for Fedora 8. Apart from the output on stdout there are few files created along the way, those are:

• results.html is HTML version of the results. I slightly modified this file in order to remove IP addresses, but otherwise it is untouched!
• results.xml is XML version of the previous file.
• results_to_html.xsl
• system-characteristics.xml is where you'll find some data that the tests run against. It is usefull for debugging purposes!
• ovaldi.log is basically what was seen on the stdout.

What's in the OVAL file

The simple OVAL file I provided checks if Fedora 8 or 9 is running on the computer where ovaldi is started. So, before going further open it in some text or XML editor. Few notes to bare in mind while we step through this file:

• I wrote it based on RedHat's definition so there are some references on RedHat left in the file. I think they are harmless, and also, I don't (yet) know what to place there.
• As an ID for all the stuff in the file I used org.fedoraproject.oval namespace!

General structure of OVAL definition file

The file has the following general structure:
First there is XML PI element that defines it's XML version 1.0 as well as that UTF-8 coding is used.
Top level element is oval_definitions and it has attributes with schemas that I just c/p!
The first element is generator. I didn't changed it, but I suppose it's for metadata about file itself, e.g. who created it, with what tool, etc.
Then there are the following important four parts:

1. definitions that define checks to be performed.
2. tests defines basic tests to be performed.
3. objects are the elements on which tests are performed. For example, if version of some package is checked, then the object is the package.
4. states are states that are checked on objects. For example, certain package is an object, version is a state. Probably it could be more complicated than that, but this is enough to get and idea.

Example OVAL definition file

So what we have in the example file? We are determining the exact version of Fedora running on the test computer and this is done by looking what the version (state) of the package fedora-release is present. Thus, the object is the package, and the state is either version 8 or 9.
In the example file, the object is specified in the objects part of the definition file as follows:

<rpminfo_object id="oval:org.fedoraproject.oval:obj:1"
		version="1" comment="the fedora-release rpm"
		xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#linux">
  <name>fedora-release</name>
</rpminfo_object>

rpminfo_object element is predefined in OVAL library and it's used to query RPM objects, i.e. packages. In our case, we are querying for package with the name fedora-release. The attribute id is used for referencing this definition in other parts of the file!
The other part of the equation, states, are defined within the states element of the OVAL definition file as follows:

  <rpminfo_state id="oval:org.fedoraproject.oval:ste:1"
	version="1"
	xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#linux">
    <version operation="pattern match">9<version>
  <rpminfo_state>

  <rpminfo_state id="oval:org.fedoraproject.oval:ste:2"
	version="1"
	xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#linux">
    <version operation="pattern match">8<version>
  <rpminfo_state>

The state simply matches the version variable (identified by the version attribute) with number 8 (for Fedora 8) or 9 (for Fedora 9). Also, id attributes are used for referencing those states in other parts of the file.
Now, we have two tests. One that checks for Fedora 8 and another one for Fedora 9. Those go within tests element.

<rpminfo_test id="oval:org.fedoraproject.oval:tst:1" version="1"
	comment="Fedora 9 is installed"
	check_existence="at_least_one_exists" check="at least one"
	xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#linux">
  <object object_ref="oval:org.fedoraproject.oval:obj:1"/>
  <state state_ref="oval:org.fedoraproject.oval:ste:1"/>
<rpminfo_test>

<rpminfo_test id="oval:org.fedoraproject.oval:tst:2" version="1"
	comment="Fedora 8 is installed"
	check_existence="at_least_one_exists" check="at least one"
	xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#linux">
  <object object_ref="oval:org.fedoraproject.oval:obj:1"/>
  <state state_ref="oval:org.fedoraproject.oval:ste:2"/>
<rpminfo_test>

Note that each of those two tests simply says that object referenced by the object element has to be in the state referenced by the state element. So, what we have here are two tests that check if a single object is in the one state and then in the other.
Finally, beacuse both tests have to be executed, and the results of each one of them has to be printed, there are two definitions. If we are interested in some logical combination of the two tests we could write them in a single definition. So, the definition that checks for Fedora 9 is within definitions element and has the following structure:

<definition id="oval:org.fedoraproject.oval:def:1" version="1" class="inventory">
  <metadata>
    <title>The operating system installed on the system is Fedora 9<title>
    <affected family="unix">
          <platform>Fedora 9<platform>
    <affected>
    <reference source="CPE" ref_id="cpe:/o:redhat:enterprise_linux:3::ix86"/>
    <description>The operating system installed on the system is Fedora 9<description>
    <oval_repository>
      <dates>
        <submitted date="2008-01-12T14:07:00">
          <contributor organization="University of Zagreb, FER">Stjepan Groš<contributor>
        <submitted>
        <status_change date="2008-07-08T13:56:57.725+02:00">DRAFT<status_change>
      <dates>
      <status>DRAFT<status>
    <oval_repository>
  <metadata>
  <criteria>
    <criterion comment="Fedora 9 is installed" test_ref="oval:org.fedoraproject.oval:tst:1"/>
  <criteria>
<definition>

Esentially, the part in criterion element references tests that have to be performed in order to determine whether vulnerability is present or not.

Implementing sysctl checks in OVAL

On one occasion I had to do a security analysis of a CentOS server. In order to do that as best as I can, I found document Guide to the Secure Configuration of Red Hat Enterprise Linux 5 that I took as a starting point in doing security analysis. Then, I realised that by manually checking what's done isn't going to be enough for two reasons:

1. There are another servers that I want also to check and it's going to be too much work so I have to automate somehow this whole process.
2. Also, once the things are configured it has to be regularity verified, which is also to be problematic if done manually.

So I decided to write OVAL checks that will be customized for each server and that will be periodically run in order to very if security settings are in place.
Of course, it want' be easy as there is lot to learn in order for me to be able to write OVAL security checks. So, I'm going to write here what and how I did.

Checking sysctl variables

In section 2.5.1.1 of the Guide to the Secure Configuration of Red Hat Enterprise Linux 5 there is recommendation for the following sysctl values:

net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

It turns out that in the version 5.9 of OVAL there is sysctl test available. So until it is available in Fedora I'll have to wait.