Signing XML document using xmlsec1 command line tool

Suppose that you have some XML document you wish to sign. It turns out it's very easy to do so because there is xmlsec library, and in particular xmlsec1 command line tool that's standard part of Fedora Linux distribution. The only problem is that its very picky and not very informative when it comes to error logging, finally, there are a lot of small details that can catch you. Since I had to sign a document I spent some time trying to figure out how to do that. In the end, I managed to do it and I'll write here how to for a future reference. Before I continue you'll need certificate and a key to be used for verification and signing. They are not the topic of this post so I'll just give you them: private key, certificate, and CA certificate.

Ok, let's assume that you have the following XML document you wish to sign:

<?xml version="1.0" encoding="UTF-8"?>
<document>
  <firstelement attr1="attr1">
    Content of first element.
    <secondelement attr2="attr2">
      Content of the second element.
      <thirdelement attr3="attr3">
        And the content of the third element.
      </thirdelement>
    </secondelement>
  </firstelement>
</document>

Basically, you can take any XML document you wish. I'll suppose that this XML document is stored in the file tosign.xml. If you typed yourself XML document, or if you just want to be sure, you can check if XML is well formed. There is xmllint tool that surves that purpose. Just run it like this:

$ xmllint tosign.xml

And if you don't get any error messages, or warnings, that the XML document is well formed. You can also check if the document is valid by providing schema, or DTD, via appropriate command line switches.

In order to sign this document you have to add XML Signature fragment to the XML file. That fragment defines how the document will be signed, what will be signed, and, where the signature, along with certificate, will be placed. The fragment has the following form:

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
    <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
    <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
    <Reference>
      <Transforms>
        <Transform           Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
        <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
      </Transforms>
      <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
      <DigestValue />
    </Reference>
    </SignedInfo>
  <SignatureValue />
  <KeyInfo>
    <X509Data />
  </KeyInfo>
</Signature>

Note that this (quite verbose) fragment has to be placed somewhere within the root element. Now, lets sign this, newly created document. To do so invoke xmlsec1 command like this (this is one line in case it is broken into two due to the formatting):

xmlsec1 --sign --privkey-pem privkey.pem,cert.pem --output signed.xml tosign.xml

After this, the signed XML document will be in the file named signed.xml. Take a look into it, the placeholders within signature fragment are filled up with signature data, and with a certificate who's private key was used to sign the XML document.

Note that signature itself is generated using private key (privkey.pem) which, as its name suggest, has to be private for a signer. Otherwise, anyone can falsify signature.

Now, to verify signed XML document you have to specify trusted CA that will be used to verify signature. It has to be certificate of the certificate authority (CA) that issued signer's certificate. In my case that's cacert.pem, i.e.:

$ xmlsec1 --verify --trusted-pem cacert.pem signed.xml
OK
SignedInfo References (ok/all): 1/1
Manifests References (ok/all): 0/0

As you can see, the signature was verified OK. You can try now to change something in the XML document and see if the verification passes or not.

I'll mentioned one more thing before concluding this post. Namely, in the previous example the whole XML document was signed. But, you can sign only a part. To do so, you have to do two things. The first one is to mark the element that you wish to sign (its content will also be signed) and the second is to tell xmlsec1 to sign only that element.

The first step is accomplished by adding attribute to the element that should be signed. Let's assume that in our case we only want secondelement to be signed. Modify the appropriate opening tag to have the following form:

<secondelement attr2="attr2" id="signonlythis">

Note that I added attribute id, but basically any name can be used (unless you use some predefined schema or DTD).

The second step is to tell xmlsec1 that only this element should be signed. This is accomplished by modifying Reference element to have the following form:

<Reference URI="#signonlythis">

If you now try to sign this modified XML document using command I gave above, you'll receive an error message:

$ xmlsec1 --sign --privkey-pem cert.key,cert.pem --output test_signed.xml tosign.xml
func=xmlSecXPathDataExecute:file=xpath.c:line=273:obj=unknown:subj=xmlXPtrEval:error=5:libxml2 library function failed:expr=xpointer(id('signonlythiselement'))
func=xmlSecXPathDataListExecute:file=xpath.c:line=356:obj=unknown:subj=xmlSecXPathDataExecute:error=1:xmlsec library function failed:
func=xmlSecTransformXPathExecute:file=xpath.c:line=466:obj=xpointer:subj=xmlSecXPathDataExecute:error=1:xmlsec library function failed:
func=xmlSecTransformDefaultPushXml:file=transforms.c:line=2405:obj=xpointer:subj=xmlSecTransformExecute:error=1:xmlsec library function failed:
func=xmlSecTransformCtxXmlExecute:file=transforms.c:line=1236:obj=unknown:subj=xmlSecTransformPushXml:error=1:xmlsec library function failed:transform=xpointer
func=xmlSecTransformCtxExecute:file=transforms.c:line=1296:obj=unknown:subj=xmlSecTransformCtxXmlExecute:error=1:xmlsec library function failed:
func=xmlSecDSigReferenceCtxProcessNode:file=xmldsig.c:line=1571:obj=unknown:subj=xmlSecTransformCtxExecute:error=1:xmlsec library function failed:
func=xmlSecDSigCtxProcessSignedInfoNode:file=xmldsig.c:line=804:obj=unknown:subj=xmlSecDSigReferenceCtxProcessNode:error=1:xmlsec library function failed:node=Reference
func=xmlSecDSigCtxProcessSignatureNode:file=xmldsig.c:line=547:obj=unknown:subj=xmlSecDSigCtxProcessSignedInfoNode:error=1:xmlsec library function failed:
func=xmlSecDSigCtxSign:file=xmldsig.c:line=303:obj=unknown:subj=xmlSecDSigCtxSigantureProcessNode:error=1:xmlsec library function failed:
Error: signature failed
Error: failed to sign file "tosign.xml"

The problem is that URI attribute references ID attribute of an element. But, ID element isn't recognized by name but has to be specified in DTD or in schema, depending what you have. In our case there is neither schema nor DTD and thus ID isn't recognized by xmlsec1. So, we have to tell it what is the name of the ID attribute, and that can be done in two ways. The first one is by using command line switch --id-attr, and so the command to sign this document is:

xmlsec1 --sign --privkey-pem privkey.pem,cert.pem --id-attr:id secondelement --output signed.xml tosign.xml

The name after the column is the attribute name that is ID. Default value is "id", but can be anything else. If it is "id", then it can be omitted. The argument to --id-attr is element whose attribute should be treated as an id. You should also be careful of namespaces. If they are used then the namespace of the element has to be specified too, and not shorthand but the full namespace name. Finally, note that XML is case sensitive!

The other possibility is to create DTD file and to give it as an argument to xmlsec1. In this case, DTD should look like this (I'll assume that this is the content of a file tosign.dtd):

<!ATTLIST secondelement id ID #IMPLIED>

And you would invoke xmlsec1 like this:

xmlsec1 --sign --privkey-pem privkey.pem,cert.pem --dtd-file tosign.dtd --output signed.xml tosign.xml

Note that you'll receive a lot of warnings (DTD is incomplete) but the file will be signed. To check the signature, you again have to specify either --dtd-file or --id-attr options, e.g.

xmlsec1 --verify --trusted-pem cacert.pem --id-attr:id secondelement signed.xml

Now, you can experiment to check that really only secondelement was signed and nothing else.

Final note. You have to put XML signature fragment in XML file you are signing. What can confuse you (and confused me) is that there is an option sign-tmpl that adds this fragment, but it is very specific and used only for testing purposes.

Examples of ovaldi checks: sysctl variables

After I described how to compile ovaldi on CentOS and a simple test to verify it is working, in this post I'm going to describe how to use ovaldi to check the values of sysctl variables. More precisely, I'm going to check that IPv4 forwarding is turned off. The general idea behind this example is to give you a starting example on which you can build more complicated checks. Note that there is even more general idea. Namely, you can create your own security benchmarks that can check if certain security criteria are met, and if not, you can be alarmed by automatic monitoring process that is based on ovaldi.

In the text that follows I'm referencing the following file. That is a simple and complete file that will check the value of net.ipv4.ip_foward sysctl variable. After you've downloaded this file, and assuming that you have your environment properly configured (see posts I referenced at the beginning) then you can run ovaldi to do the check:

ovaldi -m -o sysctl-test.xml -a /opt/oval/share/ovaldi/xml

ovaldi will create the usual output files after running this command: ovaldi.log, results.xml, results.html and system-characteristics.xml. Each one of them you can open in a brower. You should open them and check their content. system-characteristics.xml is interesting because there you can find what information was collected about the system. Those values can be checked for in oval definitions XML file. Note that ovaldi collects only referenced data, not everything it could possibly collect.

Let us now dissect a bit oval definitions file, sysctl-test.xml. The basic structure of this file is:

<oval_definitions ...>
    <definitions>...</definitions>
    <tests>...</tests>
    <objects>...</objects>
    <states>...</states>
</oval_definitions>

Basically, what oval definition does is to define a series of test, each one describing what expected state of certain object is. Test themselves can be combined in many different ways using AND and OR operators and nesting.

In our simple example object whose state we are interested in is ip_forwarding variable. So, if you look into XML file, inside element, you'll find that we define object of interest:

<sysctl_test id="oval:hr.sistemnet.oval:tst:1"
   version="1"
   comment="forwarding is disabled"
   check="at least one"
   xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
  <object object_ref="oval:hr.sistemnet.oval:obj:1" />
  <state state_ref="oval:hr.sistemnet.oval:ste:1" />
</sysctl_test>

xmlns attribute is important. I had some problems with undefined element until I got that one correctly. In other words, all the objects, their state and tests are defined in XML documents in /opt/oval/share/ovaldi/xml directories. But, when using those be certain to correctly define namespace where they are defined, or otherwise ovaldi will complain that you are using unknown test, objects and/or states.

This particular test reference object that has to be checked and the state in which this object has to be. Object itself is defined in <objects> element as follows:

<sysctl_object id="oval:hr.sistemnet.oval:obj:1" version="1"
         xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
    <name>net.ipv4.ip_forward</name>
</sysctl_object>

As you can see, within name element you specify which sysctl variable you wish to check. The second part of the test is the state in which object has to be. We want our object to have value 0, meaning forwarding is disabled. That check is performed using the following within states element:

<sysctl_state id="oval:hr.sistemnet.oval:ste:1"  version="1"
      xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#unix">
    <value>0</value>
</sysctl_state>

Obviously, you just place desired value within value element.

So, we saw that test consists of an object that has to be in particular state. The test itself is referenced in definition element (which is placed within definitions element). For that purpose you are using criterion element:

<criteria operator="AND">
    <criterion test_ref="oval:hr.sistemnet.oval:tst:1" comment="forwarding is disabled" />
</criteria>

As you can probably guess, multiple criterions can be specified and in this case they will be bound with AND operator. You can nest criteria and criterion elements to get very complex tests.