Compiling OVALDI 5.10.1.6 on CentOS 6.5

Some time ago I wrote about compiling Ovaldi on CentOS 6. Now, I tried to compile it again, and I found out that some things changed. Most importantly, there is no need to compile old Xalan/Xerces libraries any more. But, there are still problems with RPM. To make the story short, I managed to compile it and create RPM. Here are the files:

• patch you need to be able to compile ovaldi
• SRPM file you can use to recompile ovaldi; it contains patch
• RPM file if you don't want to compile it yourself (and you trust me ;))

Note that I didn't do any testing at all! So, it might happen that rpm based stuff doesn't work. If that's the case leave a comment and I'll take a look when I find time.

New developments in CentOS world...

Suddenly, there is a flurry of news about CentOS. First, there was some lengthy discussion on CentOS user's mailing list about the current state of the 6.1 release and about the RedHat versus CentOS. Also, on a development mailing list there is discussion about moving CR repository back to main repository. Finally, there was a question on Slashdot about why would someone use RedHat when there is CentOS. So, let me give you a digest of some of the topics that were discussed.

RedHat Company vs. CentOS Project

Relationship between CentOS and RedHat often comes under discussion and suspicion. And with recent changes that RedHat made, some of which made harder for CentOS to follow RedHat, discussions become even more energetic.

It is true that it bacome hard for CentOS project to follow RedHat releases. The reason is that RedHat on purpose made this process more complicated than necessary. That's why people are bashing RedHat assuming that it is complicating things because of a CentOS. But, there is high probability that RedHat is doing this because of Oracle and Novel. Oracle is building it's own distribution based on RHEL and then it is selling services for it. Both, Novel and Oracle, are selling services for RHEL too, and with lower prices than RedHat itself. And Novel and Oracle have much more resources on their disposal than CentOS will ever have. So it seems logical that RadHat is not concerned with CentOS, but with Oracle and Novel. If it were concerned with CentOS than it would be a major tactical error.

Not only that, but CentOS developers are repeatedly stressing that CentOS without RHEL, and RedHat, wouldn't exist, and if you need, or can, that you should buy support from RedHat. This is not done by either Oracle nor Novell. It turnes out that buying RHEL service isn't so expensive, after all, as I'm going to discuss shortly a bit later.

Also, there is information that someone from the core team said  that RedHat helped them, but these claims are unverifiable, i.e. I can not find mails where this is written, or anything that could be used as a proof. But, one member of the core team told on a mailing list that they were contacted by RedHat to verify that they (the CentOS team) are aware of Acceptable Use Policy (AUP). It is interesting to follow the tone of this particular discussion. It started as an information that RedHat contacted them and they asked CentOS if they are aware of new AUP. Quickly, this tread turns into bashing RedHat. Finally, the developer in question responded himself. And the conclusion was that it was not hostile neither there was any threatening! The developer again explicitly states that CentOS tries to obey AUP as much as possible.

Finally, from some informal discussions that some users had it turns out that RedHat is looking at CentOS beneficially. But I somehow doubt that RedHat can or will help CentOS, at least not publicly. And may I say that whoever expects from RedHat to do so, isn't expecting the right thing to happen. Instead, those should expect CentOS project to change itself.

As a side note, I didn't know that Microsoft officially supports CentOS, but only as a virtual guest. This is quite interesting.

CentOS Project

It seems to me that the main problem to CentOS is CentOS itself. First, for such a large project more people should be included in the core development team. Second, the lack of communication from core team is also very seriously hurting CentOS. At the time this post was written there was an announcement from Sempember, 1st that only 16 packages have to be built and 6.1 will be released. But it was two months ago, and in the mean time there wasn't any announcement or status update. People that use CentOS are those that need some stability and predictability, and both of those seems to be missed by CentOS. All this makes people look for alternatives, and more frequently than not Debian and Ubuntu popup. I hope that I don't have to explain how this hurts RedHat in longer term.

And while I'm at alternatives, it was rather interesting to find out that there is also additional alternative to CentOS and Scientific Linux based on RHEL, the distribution neatly named PUIAS.

RHEL Pricing model
Part of the discussion were concerned about licensing models. Someone wrongly calculated that you have to pay for a RHEL license $2000/year for 2 sockets (two places for CPU). But then it turned out that there are cheaper options, namely, buy only OS without support services. In that case you are paying $50 for a workstation per year, and 350$ for a server per year. These options are actually very attractive for people that use CentOS for a simple reason that in both cases you don't have support, BUT you do have latest security updates.

Also, there is an option of paying $2000/year for virtualization platform with unlimited number of guests. Furthermore, this options covers RHEL licences in all guests. Obviously, in case you plan to virtualize some other OS, then you'll have to buy licenses for that OS separately.

CR Repository

Apparently, there are also discussion to change how transitions between point releases is done. CR (or Continuous Repository) has been introduced with the idea that security updates are faster released. Even though some are happy with CR, while others are not, it seems that it isn't good enough (TM) solution. First, there are many older installations that don't have installed CR repository. They do have installed updates repository, and they require manual install. This is definitely problematic. Second, constantly some question or something like that pops out in some mailing list. The new idea is that point releases are fixed (e.g. 6.1) while generic release (e.g. 6) is a moving target. All in all, it seems that the change is inevitable but we'll see what exactly is going to happen.

Testing ovaldi on CentOS 6...

In the previous post I described how to compile ovaldi tool for CentOS. In the mean time I tested that installation and found few more bugs in rpm handling code:

• query format was wrong, i.e. the tag used was %{SIGGPG:pgpsig} but actually it should be %{SIGPGP:pgpsig}. I tested this on CentOS 6 and Fedora 15 and on both the second form is right. The first form returns (none).
• After obtaining signature key from rpm, the code wrongly calculated starting offset  of the key, so you ended up with space before and last digit cutt of. (NOTE: This has been fixed in Ovaldi 5.10.1.1 so I removed that part from my patch!)

Both of those I corrected and the changes are included in the provided patch. If you downloaded that patch (or binaries) before this post was published, then download them again.

Since I had problems with rpm I extracted problematic part of the code into separate program and used it to test its functionality. You can obtain the test program here. If you compile it and start it, you'll note that it functions exactly as the following rpm query command:

rpm -q --qf '%{SIGPGP:pgpsig}' <packagename>

To compile it, use the following command:

gcc -o rpmq rpmq.c -lrpm

On Fedora you'll also need -lpopt option added on the end.

Oval definitions for RedHat's security advisories can be found on the following address. I downloaded rhsa.tar.bz2 which includes all the advisories, unpacked it and then modified OVAL description com.redhat.rhsa-20111409.xml. This particular description checks for a vulnerable openssl. To see if the check will detect vulnerability I downgraded openssl to the original version shipped with CentOS, i.e. openssl-1.0.0-4.el6.x86_64. Furthermore, I also had to heavily modify aforementioned OVAL description because CentOS doesn't have packages like RedHat, nor it is using the same signing key. So, the version I ended up can be obtained here (hope RedHat won't be mad on me for this! :))

Running that description within ovaldi on a vulnerable system produces the following output:

$ ovaldi -m -o org.centos.cesa-20111409.xml

----------------------------------------------------
OVAL Definition Interpreter
Version: 5.10 Build: 1
Build date: Oct 30 2011 21:40:11
Copyright (c) 2002-2011 - The MITRE Corporation
----------------------------------------------------

Start Time: Mon Oct 31 00:14:16 2011

 ** parsing org.centos.cesa-20111409.xml file.
    - validating xml schema.
 ** checking schema version
     - Schema version - 5.3
 ** skipping Schematron validation
 ** creating a new OVAL System Characteristics file.
 ** gathering data for the OVAL definitions.
      Collecting object:  FINISHED                         
 ** saving data model to system-characteristics.xml.
 ** running the OVAL Definition analysis.
      Analyzing definition:  FINISHED                      
 ** applying directives to OVAL results.
 ** OVAL definition results.

    OVAL Id                                 Result
    -------------------------------------------------------
    oval:org.centos.cesa:def:20111409        true          
    -------------------------------------------------------


 ** finished evaluating OVAL definitions.

 ** saving OVAL results to results.xml.
 ** running OVAL Results xsl: /opt/oval/share/ovaldi/xml/results_to_html.xsl.

----------------------------------------------------

Basically, it detects that there is the vulnerability present (clearly indicated by the result field which I set to bold to be more visible!). After performing an update to CentOS and running test again produces negative results, as expected, i.e.

$ ovaldi -m -o org.centos.cesa-20111409.xml

----------------------------------------------------
OVAL Definition Interpreter
Version: 5.10 Build: 1
Build date: Oct 30 2011 21:40:11
Copyright (c) 2002-2011 - The MITRE Corporation
----------------------------------------------------

Start Time: Mon Oct 31 00:16:55 2011

 ** parsing org.centos.cesa-20111409.xml file.
    - validating xml schema.
 ** checking schema version
     - Schema version - 5.3
 ** skipping Schematron validation
 ** creating a new OVAL System Characteristics file.
 ** gathering data for the OVAL definitions.
      Collecting object:  FINISHED                         
 ** saving data model to system-characteristics.xml.
 ** running the OVAL Definition analysis.
      Analyzing definition:  FINISHED                      
 ** applying directives to OVAL results.
 ** OVAL definition results.

    OVAL Id                                 Result
    -------------------------------------------------------
    oval:org.centos.cesa:def:20111409        false         
    -------------------------------------------------------


 ** finished evaluating OVAL definitions.

 ** saving OVAL results to results.xml.
 ** running OVAL Results xsl: /opt/oval/share/ovaldi/xml/results_to_html.xsl.

----------------------------------------------------

This time ovaldi produced the following files ovladi.log, results.xml, results.html and system-characteristics.xml.

With this I'm now pretty sure that ovaldi works on centos. Still, more extensive testing is absolutely necessary, but for the time being this, I think, is a great step forward.

So, here are some conclusions from this exercise:

• CentOS doesn't have assigned CPE values past version 5. So, some procedure has to be initiated in that respect.
• RedHat's oval descriptions can not be used for two reasons. First, the legality is questionable, and second, the change is not straightforward.
• Editing of OVAL XML description files is very hard and error prone. Furthermore, ovaldi itself is not very helpful. For example, if you don't get IDs and references right, it will complain but the diagnostic information is basically useless.

Just as a note, when I had a problem that some test, or object or something else, is referenced but not defined, I used the following quick hack to find the offending ID:

for i in `grep _ref org.centos.cesa-20111409.xml | cut -f2 -d\"`do grep -q id=\"$i org.centos.cesa-20111409.xml || echo $i ; done

Which printed the offending ID.

That concludes this post. In some future post I'll describe in more detail the structure of OVAL description, and in the mean time you can find some old information on my homepage.

Compiling OVALDI for CentOS 6

Note: Take a look at the newer version of this post. Things are simpler now.

I described in one earlier post the purpose of OVAL and the benefits it gives to a user. Here I'm going to describe how to setup OVAL interpreter on CentOS 6. The problem is that there is no prepackaged Oval interpreter for CentOS 6. Actually, there is but it's only for 32 bit version of CentOS 4 and 5, an it is an older version, not the latest one. So, here I'm going to describe how to build it from source. The build process consists of building XML processor Xalan, then XSLT processor Xerces and finally in building interpreter itself. There are certain prerequisites you need to have in order for Oval to build, I'll mention those also.

I'll assume that you created working directory for this purpose and that you run all the following commands within that directory. When necessary, I'll reference that directory as $WORKDIR and when you see that string replace it with full path of your working directory. Also, I'm going to install oval interpreter into directory /opt/oval. The reason I'm not placing it into some of the "system" directories like /usr/bin, /usr/lib and similar is to avoid clash with versions of xalan and xerces shipped with distribution itself.

In case you trust me enough, here is archive of final content of directory /opt/oval, so you can unpack it and skip to the Running ovaldi section.

Installing prerequisites

Xerces
Download version 2.8.0, or whatever is the latest version of Xalan 2. Don't use Xalan 3 because API was changed with respect to version 2 and OVAL won't work with it! In the following text, I'll reference version 2.8.0 and if there is a newer one replace version numbers as necessary.

After downloading some package it is a good practice to check MD5 sum (or SHA1). In this case md5 sum will give the following output:

$ md5sum xerces-c-src_2_8_0.tar.gz
5daf514b73f3e0de9e3fce704387c0d2  xerces-c-src_2_8_0.tar.gz

which matches the one given on the dowload page.

Now, unpack the archive using the following command:

$ tar xzf xerces-c-src_2_8_0.tar.gz

and you'll get directory xerces-c-src_2_8_0/. Go into that directory and then into src/xercesc subdirectory. Before configuring distribution set the environment variable XERCESCROOT to the top level directory of the unpacked archive, i.e.

export XERCESCROOT=$WORKDIR/xerces-c-src_2_8_0

now, start configuration process:

./runConfigure -p linux -c gcc -x c++ -b 64 -P /opt/oval

In that command option p specifies platform on which you are performing build process, option c specifies compiler to use, x specifies c++ compiler, option b determines bit width of the platform (32 or 64 bit) and option P specifies installation directory. All the other options have appropriate default values. Note that you must specify c++ instead of g++! If you specify g++, then while building Xalan, you'll get the following errors:

$XERCESCROOT/lib/libxerces-c.so: undefined reference to `stricmp(char const*, char const*)'
$XERCESCROOT/lib/libxerces-c.so: undefined reference to `strnicmp(char const*, char const*, unsigned int)'

The problem is that the configuration process misidentified that GNU's compiler is used that doesn't have stricmp and strnicmp functions and it didn't include replacement functions!

If everything went without an error, start build process by issuing make command:

make

and finally, install xerces (you should switch to root user to run the following command):

make install

Xalan
Go to the download page and take most recent version of Xalan. I was using 1.10 which was the latest one at the time this post was written. So, after download it, and checking signature(!), unpack it with the following command:

tar xzf Xalan-C_1_10_0-src.tar.gz

This will create new directory, xml-xalan/. Before building Xalan, you should apply a patch to it. The problem is that gcc developers made some changes to header files (removed unnecessary includes) in recent version available on CentOS and that means that some prerequisite includes have to be explicitly specified. The problem is manifested with the following error messages:

home/zavod/sgros/work/xml-xalan/c/src/xalanc/XalanDOM/XalanDOMString.cpp: In member function ‘xalanc_1_10::XalanDOMString& xalanc_1_10::XalanDOMString::assign(const xalanc_1_10::XalanDOMString&, xalanc_1_10::XalanDOMString::size_type, xalanc_1_10::XalanDOMString::size_type)’:
/home/zavod/sgros/work/xml-xalan/c/src/xalanc/XalanDOM/XalanDOMString.cpp:251: error: ‘memmove’ was not declared in this scope
/home/zavod/sgros/work/xml-xalan/c/src/xalanc/XalanDOM/XalanDOMString.cpp: In static member function ‘static xalanc_1_10::XalanDOMString::size_type xalanc_1_10::XalanDOMString::length(const char*)’:
/home/zavod/sgros/work/xml-xalan/c/src/xalanc/XalanDOM/XalanDOMString.cpp:780: error: ‘strlen’ was not declared in this scope

So, download the patch and enter into xml-xalan directory. Then, run the following command:

$ patch -p1 < ../xml-xalan.gcc-4.4.patch
patching file c/src/xalanc/TestXPath/TestXPath.cpp
patching file c/src/xalanc/XalanDOM/XalanDOMString.cpp
patching file c/src/xalanc/XalanExe/XalanExe.cpp
patching file c/src/xalanc/XMLSupport/FormatterToHTML.cpp
patching file c/src/xalanc/XSLT/ElemNumber.cpp

this assumes that you've downloaded patch into the same place where you downloaded Xalan itself (i.e. $WORKDIR).

Now, enter into subdirectory named c/. Before configuring the build process, define the variable XALANCROOT. You should set it to $WORKDIR/xml-xalan/c with the following command:

export XALANCROOT=$WORKDIR/xml-xalan/c

Also, note that Xalan depends on Xerces and to be able for Xalan to find Xerces you need to set the environment variable XERCESROOT, or Xerces has to be in some system directory that is searched by default (e.g. /usr/include and similar directories).  If you followed this post withouth interruption, you probably have it defined already. Now, initiate configure process using runConfigure command:

./runConfigure -p linux -c gcc -x c++ -b 64 -P /opt/oval

the options used are same as for Xerces. Initiate build process using make, and after build finishes, install it using 'make install' command switching before to root user.

Necessary development packages
As a final prerequisite you should check that the following development packages are installed. The simplest way to do that is to initiate install process and yum will react appropriately: pcre-devel, libgcrypt-devel, rpm-devel, openldap-devel, libblkid-devel, and libselinux-devel.

Building and installing Ovaldi
Go now to the download page of Ovaldi and download the latest version. Version 5.10.1.1 is the latest one at the time of writing this post. So download it and upack it. This will create directory ovaldi-5.10.1.1-src. Also, download the following patch. Note that this patch is made so that ovaldi can be compiled on CentOS 6 and it is not applicable for other distributions, neither it will allow ovaldi to be compiled on other platforms (though, very unlikely it might :)).

Now, enter ovaldi-5.10.1.1-src directory and apply patch:

patch -p1 < ../ovaldi-5.10.1.1-centos6.patch

Three changes are in the patch file. The first one is addition of /opt/oval/include and /opt/oval/lib directories in main Makefile. The second are some changes to RPM part of the code since API has changed in recent versions of RPM. More specifically, I introduced compatibility switch (-D_RPM_4_4_COMPAT) and also replaced int_32 with int32_t types.

Third change resolves the following error message already reported on some forums:

Error running rpm query in child process: blah: -q: unknown option

There is also additional patch that isn't always necessary, and that's why I separated it. Namely, I placed ovaldi in /opt/oval directory, while ovaldi expects by default its shared files to be within /usr/share/oval. So, this patch changes this:

patch -p1 < ../ovaldi-5.10.1.1-sharepath.patch

Since for some unknown reason (I didn't have will/time to investigate further) linker can not find libxalanMsg.so.110 library, even though it has appropriate path in -L option, define LD_LIBRARY_PATH using the following command prior to comilation:

export LD_LIBRARY_PATH=/opt/oval/lib/

Finally, enter project/linux subdirectory and initiate build process:

make

When the build process is over, copy ovaldi binary (you'll find it in project/linux/Release subdirectory) to /opt/oval/bin directory. Also, create directory /opt/oval/share/ovaldi and move there directory xml (you'll find it directly beneath ovaldi-5.10.1.1-src directory).

Running ovaldi
Finally, we are ready to run ovaldi interpreter. Before running ovaldi you should define LD_LIBRARY_PATH and optinally PATH variable. In other words, before running ovaldi execute once these commands:

export LD_LIBRARY_PATH=/opt/oval/lib
export PATH=/opt/oval/bin:$PATH

Then, try to run ovaldi, you should get help message.

This concludes this post. In the next one I'm going to try to run ovaldi using RedHat's provided files. Until I do this note that the patches I provided may turn to have errors that would prevent ovaldi from correctly functioning.

Why I think it is in RedHat's interest to help CentOS...

Today I was asked if there are any security implications in selecting particular licensing model from Microsoft. Basically, I know nothing about that particular subject, and as far as I can remember from some previous experiences, this is something that requires specialization in itself. To cut the story short, I don't intend to waste my time in studying Microsoft's licensing models! So, in the end I basically said that any option is valid from my perspective as long as we have access to security updates. No more no less...

But since everything was about selecting the least expensive solution, I mentioned that it might be beneficial to introduce LibreOffice (OpenOffice) instead of Microsoft Office and/or Linux on certain workstations because people don't use all the functionality of Windows and especially Microsoft Office. It is true that LibreOffice isn't quite a match to Microsoft Office, but for people that only write a single page of something and then send this to a printer it is to much to pay for a whole office suite! Or, those that access remote machines and do their work there it is also to much to maintain the whole workstation with a full productivity suite on it. My idea was, unsurprisingly, rejected because of a slew of problems, like compatibility between different office suites, support for equivalent functionality of Outlook, potential problems with user support, etc. Those might or might not be the problems, but in the end, I was asked what distribution I would recommend if there would be (partial) migration?

I said, without almost any thinking, latest version of Ubuntu LTS! Let me first clarify that I'm actually die hard user of Fedora, and also CentOS, and I use them as much as I can. But, I also stand firmly on the ground and I'm aware of problems associated with that route. First, you'll probably ask why I didn't recommend RHEL? Well, the reason is simple, it costs, and price cut wouldn't be large enough to justify such transition. Scientific Linux, as I already blogged about, has a problem with a name. If I'm going to say "Use Scientific Linux!" probably I would be rejected with a comment something in a line with "Wow, we are not scientific institution!". And for CentOS, well, no timely security updates! Period. Ok, to be honest, I do install CentOS on servers in a good hope that things will become better, but it is on a small scale and I'm usually directly in charge of those servers. Note that I didn't mention Fedora as an alternative. Well, the reasons are bitten to death by now, so I won't go into that.

So this leaves me with Ubuntu or Debian. The clear winner is definitely Ubuntu, more specifically, Ubuntu LTS. The reasons in favor are strong. First, quite user friendly, second, long time support (LTS!), third advanced almost like Fedora, but without Fedora's short support timeframe. Finally, there is possibility of obtaining support contract.

And what's the conclusion? The conclusion is that Ubuntu slowly and certainly is introduced into business environments which might or might not pose a threat to RHEL... decide for yourself...

CentOS... something is happening!

I just noticed that RPM packages from RHEL 6.1 appeared on mirrors. Actually, they announced it few days earlier but that was sooner that I was expecting. :)

There is a small catch. In order for yum to be able to catch those packages you'll have to add new repository. Namely, CentOS team decided to go with a mechanism they call Continuous updates. In that way they'll try to be faster, but, time will tell if it will work or not.

The quickest way to do that is to run the following command:

rpm -ivh ftp://ftp.funet.fi/pub/mirrors/centos.org/6/cr/i386/RPMS/centos-release-cr-6-0.el6.centos.i686.rpm

or for 64-bit systems:

rpm -ivh ftp://ftp.funet.fi/pub/mirrors/centos.org/6/cr/x86_64/RPMS/centos-release-cr-6-0.el6.centos.x86_64.rpm

This will install necessary data for yum. Then, just run 'yum update' and that's it!

Still, we'll have to wait a bit more for 6.1, and especially for FreeIPA 2 that I'm waiting for!