Processing RA messages in NetworkManager

The goal of this post is to analyze processing of RA messages through the NetworkManager code, starting with the initial reception all the way through the assignment of parameters to a device through which the RA was received. As a special case will also take a look what happens when RS is sent, i.e. what is different in comparison to unsolicited RAs. But first, we'll take a look at the relevant code organization and initialization process.

Code to process RAs and initialization phase

For receiving RA and sending RA NetworkManager uses libndp library. This library is used in class NM_TYPE_LNDP_RDISC (defined in the file src/rdisc/nm-lndp-rdisc.c) which is a platform specific class tailored for the Linux OS. This class inherits from class NM_TYPE_RDISC (defined in the file src/rdisc/nm-rdisc.c) which is a platform independent base class. It contains functionality that is platform independent so theoretically NetworkManager can be more easily ported to, e.g. FreeBSD.

To create a new object of the type NM_TYPE_LNDP_RDISC it is necessary to call function nm_lndp_rdisc_new(). This is, for example, done by the class NM_DEVICE_TYPE in function addrconf6_start() for each device for which IPv6 configuration is started.

Now, if NetworkManager will use RAs or not depends on the configuration setting for IPv6 that the user defines. If you go to configuration dialog for some network interface there is a setting for IPv6 configuration which might be ON or OFF. In case it is OFF, no IPv6 configuration will be done. If IPv6 configuration is enabled (switch placed in ON state) then the specific configuration methods should be selected. The selected option is checked in the function src/devices/nm-device.c:act_stage3_ip6_config_start(), where, depending on the option selected, a specific initialization is started:

• Automatic (method NM_SETTING_IP6_CONFIG_METHOD_AUTO)
  
  Start full IPv6 configuration by calling src/devices/nm-device.c:addrconf6_start() function.
   
• Automatic, DHCP only (method NM_SETTING_IP6_CONFIG_METHOD_DHCP)
  
  Only configuration based on DHCP parameters received will be done. This type of configuration is initiated by calling function src/devices/nm-device.c:dhcp_start().
   
• Manual (method NM_SETTING_IP6_CONFIG_METHOD_MANUAL)
  
  Manual configuration using parameters specified in the configuration dialog and nothing else. The configuration of this type is initiated by calling function nm_ip6_config_new() which returns appropriate IPv6 configuration object.
   
• Link-local Only (method NM_SETTING_IP6_CONFIG_METHOD_LINK_LOCAL)
  
  Initiate only a link-local address configuration by calling function src/devices/nm-device.c:linklocal6_start().

Since in this post we are concerned with RA processing than we are obviously interested only in Automatic configuration type, the one that calls addrconf6_start() function. This function, in turn, calls function src/nm-device.c:linklocal6_start() to ensure that link local configuration is present. It might happen that link local address isn't configured and so RA configuration must wait, or link local configuration is still present. In either case, when link local configuration is present RA processing can start. RA processing is kicked off by calling src/nm-device.c:addrconf6_start_with_link_ready() which in turn calls src/nm-rdisc.c:nm_rdisc_start() to kick off RA configuration.

nm_rdisc_start() is called with a pointer to NM_LNDP_RDISC class (defined in src/rdisc/nm_lndp_rdisc.c). Note that a method (nm_rdisc_start()) from a base class (NM_RDISC_TYPE, defined in src/rdisc/nm_rdisc.c) is called with a pointer to a subclass of a NM_RDISC_TYPE! Method in a base class does the following:

1. Checks that there is a subclass which defined virtual method start() (gassert(klass->start)).
2. Initializes timeout for the configuration process. If timeout fires, then rdisc_ra_timeout_cb() will be called that emits NM_RDISC_RA_TIMEOUT signal.
3. Invokes a method start() from a subclass. Subclass is, as already said, NM_LNDP_RDISC and the given method registers a callback src/rdisc/nm-lndp-rdisc.c:receive_ra() with libndp library. The callback is called by libndp library whenever RA is received.
4. Starts solicit process by invoking solicit() method. This method schedules RS to be sent in certain amount of time (variable next) by send_rs() method. This method, actually, invokes send_rs() method from a subclass (src/nm-rdisc/nm-rdisc-linux.c:send_rs()) which sends RS using libndp library. Note that the number of RSes sent is bounded and after certain amount of them sent the process is stopped under the assumption that there is no IPv6 capable router on the network.
5. After RA has been received and processed the application of configuration parameters is done in src/device/nm-device.c:rdisc_config_changed() method. This callback is achieved by registering to NM_RDISC_CONFIG_CHANGED signal that is emitted by src/rdisc/nm-rdisc.c class whenever IPv6 configuration changes.

So, in conclusion, when link local configuration is finished, RA processing is started. The RA processing consists of waiting for RA in src/rdisc/nm-lndp-rdisc.c:receive_ra(). If RA doesn't arrive is certain amount of time then RS is sent in function src/nm-rdisc/nm-rdisc-linux.c:send_rs().

RA processing

When RA is received it is processed by the function src/rdisc/nm-lndp-rdisc.c:receive_ra(). The following configuration options are processed from RA by the given function:

1. DHCP level.
2. Default gateway.
3. Addresses and routes.
4. DNS information (RDNSS option).
5. DNS search list (DNSSL option).
6. Hop limit.
7. MTU.

All the options that were parsed are stored (or removed from) a private attributes of the base object (NMRDisc defined in src/rdisc/nm-rdisc.h).

Finally, the method src/nm-rdisc.c:nm_rdisc_ra_received() is called to cancel all the timeouts. It will also emit signal NM_RDISC_CONFIG_CHANGED that will trigger application of received configuration parameters to a networking device.

Processing RS/RA

The RS/RA processing differs only by the fact that RS is sent after certain amount of time has passed and RA wasn't received, as described in the Code to process RAs and initialization phase section. After RS is sent, the RA processing is the same as it would be without RS being sent.

Applying IPv6 configuration data

Application of received IPv6 configuration data is done in the method src/device/nm-device.c:rdisc_config_changed(). IPv6 configuration is stored in IPv6 configuration object NM_TYPE_IP6_CONFIG defined in src/nm-ip6-config.c.

Note that this isn't the real application of configuration data, but only that the configuration data is stored in the appropriate object.

The function that really applies configuration data is src/devices/nm-device.c:ip6_config_merge_and_apply().

NetworkManager and OpenVPN - How it works?

I spent a lot of time trying to figure out how NetworkManager works. Very early in the process I came to conclusion that NM is a very complex piece of a software while in the same time documentation is lacking. What adds to the complexity is GObject mechanism that tries to fit object oriented programming into C programming language. So, in the end, I decided to write everything I managed to learn. First, in that way I'm leaving notes for myself later. In addition, I hope I'll help someone and thus save someone's time.

NetworkManager

As a first goal to understand NetworkManager I set to understand how NM manages VPN connections. The following sequence of steps is a result of a research to answer the given question. Note that the flow isn't complete, but is enough to understand mechanics of VPN establishment. In addition, note that all the given file pathnames are relative to NetworkManager git repository.

So, everything starts when user activates VPN. At that moment a message is sent via DBus by nm-applet, nmcli or some other mechanism the following happens:

1. Message to activate VPN sent via the DBus ends up in the function src/nm-manager.c:impl_manager_activate_connection().
2. The function src/nm-manager.c:impl_manager_activate_connection() calls function src/nm-manager.c:_new_active_connection().
3. Function _new_active_connection() function creates a new object of a type NM_TYPE_ACTIVE_CONNECTION, i.e. new active connection. To create new active connection object a method src/vpn-manager/nm-vpn-connection.c:nm_vpn_connection_new() in this particular case is used. This triggers a chain of initialization events described later.
4. After the object is created a control returns to impl_manager_activate_connection() where asynchronous authorization check is initiated. Callback function to call when authorization check finishes is src/nm-manager.c:_activation_auth_done().
5. After authorization is done the callback function _activation_auth_done() is called which in turn, if everything is OK, calls function src/nm-manager.c:_internal_activate_generic() which in turn calls function src/nm-manager.c:_internal_activate_vpn().
6. The function _internal_activate_vpn() calls the function src/vpn-manager/nm-vpn-manager.c:nm_vpn_manager_activate_connection() is called.
7. The function src/vpn-manager/nm-vpn-manager.c:nm_vpn_manager_activate_connection() calls function src/vpn-manager/nm-vpn-connection.c:nm_vpn_connection_activate().
8. The function nm_vpn_connection_activate() connects asynchronously to DBus and when the connection is made a callback function on_proxy_acquired() is called.
9. The function on_proxy_acquired() connects to signal "notify::g-name-owner" and then calls src/vpn-manager/nm-vpn-connection.c:nm_vpn_service_daemon_exec(). The purpose of the function nm_vpn_service_daemon_exec() is to start plugin binary (nm-openvpn-service).
   
   The goal of connecting to the signal "notify::g-name-owner" is to receive notification when the service appears, i.e. when it is initialized and available over the DBus so that appropriate signals can be registered. The most important registered signals, in our case, are Ip4Config and Ip6Config. When the function on_proxy_acquired() connects callback functions to DBus signals the process of establishing VPN can be continued so it also calls the function src/vpn-manager/nm-vpn-connection.c:get_secrets().
10. The function get_secrets() calls function nm_settings_connection_get_secrets() and registers a callback src/vpn-manager/nm-vpn-connection.c:get_secrets_cb().
11. The function src/vpn-manager/nm-vpn-connection.c:get_secrets_cb() sends secrets to VPN plugin via DBus. The send process is asynchronous and when finished callback src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() is called.
12. The callback function src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() calls function src/vpn-manager/nm-vpn-connection.c:really_activate().
13. The function src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() calls Connect method on VPN plugin via DBus interface. Since DBus call is asynchronous callback function is registered, src/vpn-manager/nm-vpn-connection.c:connect_cb(). Callback function just calls src/vpn-manager/nm-vpn-connection.c:connect_success() that clears all timers.

The call to Connect method mentioned in the last step above initiates the chain of events in the VPN plugin described in the next section that, in the end, results in real connection establishment. When VPN connection is established, the VPN plugin will send three signals: Config, Ip4Config and Ip6Config. Those signals are caught by callbacks config_cb(), ip4_config_cb() and ip6_config_cb() respectively. Those signals, when activated, will be caught in nm-vpn-connection.c (look at step 9 in the previous list).

OpenVPN plugin

VPN plugins for NetworkManager are written so that they inherit some base classes from Network Manager and they only have to implement code specific to a particular VPN, while generic parts are implemented by NetworkManager  and placed in the base class. For example, DBus interface that each plugin has to implement is common to all of them and thus implemented in the base class. This is classic OO programming paradigm. But, in this case OO paradigm is emulated in C so when you start to study the code of some specific plugin, at first sight nothing will make sense and it will be hard to grasp what happens, where and when. So, before I describe sequence of events, I'll describe a code structure first as it aids a lot in understanding the code.

Static code structure and plugin initialization

The main part of the NetworkManager OpenVPN plugin is in the file src/nm-openvpn-service.c. This file inherits a class defined in libnm/nm-vpn-service-plugin.c from NetworkManager. Additionally, VPN DBus interface is defined in NetworkManager source in file introspection/nm-vpn-plugin.xml.

So, when you build OpenVPN plugin, a new binary is created, nm-openvpn-service. When NetworkManager executes this plugin its main method is invoked. In main method, the most important line is the following one:

plugin = nm_openvpn_plugin_new (bus_name);

That line causes new object of type NM_TYPE_OPENVPN_PLUGIN to be instantiated. Looking at the code of the function nm_openvpn_plugin_new() nothing special can be seen at a first glance. Basically, there is only the following line:

plugin = (NMOpenvpnPlugin *)
    g_initable_new (NM_TYPE_OPENVPN_PLUGIN,
                    NULL, &error,
                    NM_VPN_SERVICE_PLUGIN_DBUS_SERVICE_NAME,
                    bus_name,
                    NULL);

But, because GObject type system is in background, actually a lot of things happen. First, initialization methods/constructors of OpenVPN plugin class and objects are called (nm_openvpn_plugin_init() and nm_openvpn_plugin_class_init()). Also, initialization methods/constructors of base class are called (nm_vpn_service_plugin_class_init() and nm_vpn_service_plugin_init()).

In addition, base class nm-vpn-service-plugin defines an interface that is initialized separately from class and object. The mechanism used is described here.

Note that after the plugin is initialized, NetworkManager receives information about this via a signal notify::g-name-owner in src/vpn-manager/nm-vpn-connection.c. This causes src/vpn-manager/nm-vpn-connection.c to connect to DBus interface and starts the sequence of steps described in the following subsection.

VPN activation sequence of steps

In the following code, when I write base class I mean on the code in the file libnm/nm-vpn-plugin-service.c in NetworkManager. When I write OpenVPN class or OpenVPN service then I'm thinking on file src/nm-openvpn-service in network-manager-openvpn code.

The following sequence of steps is initiated by calling method Connect on VPN plugin via DBus:

1. DBus signal initiates a function impl_vpn_service_plugin_connect() in base class. This function is registered as a handler for DBus Connect method serviced by the plugin in the function init_sync() in the base class. If you look at the code of the given function you'll notice that it only transfers control to the function _connect_generic() in the base class.
2. The function _connect_generic() does some checks and transfers function to OpenVPN class, i.e. the method real_connect() is called in the file src/nm-openvpn-service.c.
3. The method real_connect() just redirects control to the function _connect_common().
4. The method _connect_common() does some sanity check, obtains some parameters, and calls method nm_openvpn_start_openvpn_binary().
5. The method nm_openvpn_start_openvpn_binary() searches for openvpn binary on a local file system, constructs command line options based on preferences set for the VPN connection and from environment variables, and finally starts openvpn binary (call to function g_spawn_async()). One important part of the command line construction is that openvpn binary is told not to assign parameters itself, but to call helper scripts and pass it all the parameters. This helper script is nm-openvpn-service-openvpn-helper contained in the src directory of networkmanager-openvpn-plugin. So, when OpenVPN binary establishes VPN connection it calls helper script.
   
   It also registers two callbacks. The first one monitors process using a g-child-watch-source-new() function. A callback function is openvpn_watch_cb() that, if called, assumes an error occurred or openvpn binary just finished. The second callback is timer that, when fires, calls function nm_openvpn_connect_timer_cb(). It only tries to connect to OpenVPN's management socket. So, in other words, notification about successful VPN establishment to OpenVPN plugin isn't done using GLib or DBus notification system, but just by waiting and checking. As a final note, timer isn't used in every case. Actually, it seems that it is used rarely, only when authentication type is static key.
6. The nm-openvpn-service-openvpn-helper script, in its main function, collects all network related data from the environment (set by openvpn binary) and sends it via DBus to nm-openvpn-service using methods SetConfig, SetIp4Config and SetIp6Config.
7.  The configuration is caught by functions impl_vpn_service_plugin_set_config(), impl_vpn_service_plugin_set_ip4_config(), and impl_vpn_service_plugin_set_ip6_config() in VPN plugin base class. Those function, in turn, call functions nm_vpn_service_plugin_set_config(), nm_vpn_service_plugin_set_ip4_config() and nm_vpn_service_plugin_set_ip6_config(). They also call finish DBus proxies so that helper script can continue executing after each call.
8. Each of the functions nm_vpn_service_plugin_set_config(), nm_vpn_service_plugin_set_ip4_config() and nm_vpn_service_plugin_set_ip6_config() does two things. It emits signal ("config", "ip4-config" and "ip6-config") from base class and OpenVPN class. [Note: I don't fully understand this mechanism yet]

Literature

It seems to me that it is very hard to come up with a good documenation that describes this topic. So, here are some better references I used:

1. For signals the best reference I could find was from Maemo 4 documentation, available on the following link: http://maemo.org/maemo_training_material/maemo4.x/html/maemo_Platform_Development_Chinook/Chapter_04_Implementing_and_using_DBus_signals.html