NetworkManager and OpenVPN - How it works?

I spent a lot of time trying to figure out how NetworkManager works. Very early in the process I came to conclusion that NM is a very complex piece of a software while in the same time documentation is lacking. What adds to the complexity is GObject mechanism that tries to fit object oriented programming into C programming language. So, in the end, I decided to write everything I managed to learn. First, in that way I'm leaving notes for myself later. In addition, I hope I'll help someone and thus save someone's time.

NetworkManager

As a first goal to understand NetworkManager I set to understand how NM manages VPN connections. The following sequence of steps is a result of a research to answer the given question. Note that the flow isn't complete, but is enough to understand mechanics of VPN establishment. In addition, note that all the given file pathnames are relative to NetworkManager git repository.

So, everything starts when user activates VPN. At that moment a message is sent via DBus by nm-applet, nmcli or some other mechanism the following happens:

1. Message to activate VPN sent via the DBus ends up in the function src/nm-manager.c:impl_manager_activate_connection().
2. The function src/nm-manager.c:impl_manager_activate_connection() calls function src/nm-manager.c:_new_active_connection().
3. Function _new_active_connection() function creates a new object of a type NM_TYPE_ACTIVE_CONNECTION, i.e. new active connection. To create new active connection object a method src/vpn-manager/nm-vpn-connection.c:nm_vpn_connection_new() in this particular case is used. This triggers a chain of initialization events described later.
4. After the object is created a control returns to impl_manager_activate_connection() where asynchronous authorization check is initiated. Callback function to call when authorization check finishes is src/nm-manager.c:_activation_auth_done().
5. After authorization is done the callback function _activation_auth_done() is called which in turn, if everything is OK, calls function src/nm-manager.c:_internal_activate_generic() which in turn calls function src/nm-manager.c:_internal_activate_vpn().
6. The function _internal_activate_vpn() calls the function src/vpn-manager/nm-vpn-manager.c:nm_vpn_manager_activate_connection() is called.
7. The function src/vpn-manager/nm-vpn-manager.c:nm_vpn_manager_activate_connection() calls function src/vpn-manager/nm-vpn-connection.c:nm_vpn_connection_activate().
8. The function nm_vpn_connection_activate() connects asynchronously to DBus and when the connection is made a callback function on_proxy_acquired() is called.
9. The function on_proxy_acquired() connects to signal "notify::g-name-owner" and then calls src/vpn-manager/nm-vpn-connection.c:nm_vpn_service_daemon_exec(). The purpose of the function nm_vpn_service_daemon_exec() is to start plugin binary (nm-openvpn-service).
   
   The goal of connecting to the signal "notify::g-name-owner" is to receive notification when the service appears, i.e. when it is initialized and available over the DBus so that appropriate signals can be registered. The most important registered signals, in our case, are Ip4Config and Ip6Config. When the function on_proxy_acquired() connects callback functions to DBus signals the process of establishing VPN can be continued so it also calls the function src/vpn-manager/nm-vpn-connection.c:get_secrets().
10. The function get_secrets() calls function nm_settings_connection_get_secrets() and registers a callback src/vpn-manager/nm-vpn-connection.c:get_secrets_cb().
11. The function src/vpn-manager/nm-vpn-connection.c:get_secrets_cb() sends secrets to VPN plugin via DBus. The send process is asynchronous and when finished callback src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() is called.
12. The callback function src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() calls function src/vpn-manager/nm-vpn-connection.c:really_activate().
13. The function src/vpn-manager/nm-vpn-connection.c:plugin_need_secrets_cb() calls Connect method on VPN plugin via DBus interface. Since DBus call is asynchronous callback function is registered, src/vpn-manager/nm-vpn-connection.c:connect_cb(). Callback function just calls src/vpn-manager/nm-vpn-connection.c:connect_success() that clears all timers.

The call to Connect method mentioned in the last step above initiates the chain of events in the VPN plugin described in the next section that, in the end, results in real connection establishment. When VPN connection is established, the VPN plugin will send three signals: Config, Ip4Config and Ip6Config. Those signals are caught by callbacks config_cb(), ip4_config_cb() and ip6_config_cb() respectively. Those signals, when activated, will be caught in nm-vpn-connection.c (look at step 9 in the previous list).

OpenVPN plugin

VPN plugins for NetworkManager are written so that they inherit some base classes from Network Manager and they only have to implement code specific to a particular VPN, while generic parts are implemented by NetworkManager  and placed in the base class. For example, DBus interface that each plugin has to implement is common to all of them and thus implemented in the base class. This is classic OO programming paradigm. But, in this case OO paradigm is emulated in C so when you start to study the code of some specific plugin, at first sight nothing will make sense and it will be hard to grasp what happens, where and when. So, before I describe sequence of events, I'll describe a code structure first as it aids a lot in understanding the code.

Static code structure and plugin initialization

The main part of the NetworkManager OpenVPN plugin is in the file src/nm-openvpn-service.c. This file inherits a class defined in libnm/nm-vpn-service-plugin.c from NetworkManager. Additionally, VPN DBus interface is defined in NetworkManager source in file introspection/nm-vpn-plugin.xml.

So, when you build OpenVPN plugin, a new binary is created, nm-openvpn-service. When NetworkManager executes this plugin its main method is invoked. In main method, the most important line is the following one:

plugin = nm_openvpn_plugin_new (bus_name);

That line causes new object of type NM_TYPE_OPENVPN_PLUGIN to be instantiated. Looking at the code of the function nm_openvpn_plugin_new() nothing special can be seen at a first glance. Basically, there is only the following line:

plugin = (NMOpenvpnPlugin *)
    g_initable_new (NM_TYPE_OPENVPN_PLUGIN,
                    NULL, &error,
                    NM_VPN_SERVICE_PLUGIN_DBUS_SERVICE_NAME,
                    bus_name,
                    NULL);

But, because GObject type system is in background, actually a lot of things happen. First, initialization methods/constructors of OpenVPN plugin class and objects are called (nm_openvpn_plugin_init() and nm_openvpn_plugin_class_init()). Also, initialization methods/constructors of base class are called (nm_vpn_service_plugin_class_init() and nm_vpn_service_plugin_init()).

In addition, base class nm-vpn-service-plugin defines an interface that is initialized separately from class and object. The mechanism used is described here.

Note that after the plugin is initialized, NetworkManager receives information about this via a signal notify::g-name-owner in src/vpn-manager/nm-vpn-connection.c. This causes src/vpn-manager/nm-vpn-connection.c to connect to DBus interface and starts the sequence of steps described in the following subsection.

VPN activation sequence of steps

In the following code, when I write base class I mean on the code in the file libnm/nm-vpn-plugin-service.c in NetworkManager. When I write OpenVPN class or OpenVPN service then I'm thinking on file src/nm-openvpn-service in network-manager-openvpn code.

The following sequence of steps is initiated by calling method Connect on VPN plugin via DBus:

1. DBus signal initiates a function impl_vpn_service_plugin_connect() in base class. This function is registered as a handler for DBus Connect method serviced by the plugin in the function init_sync() in the base class. If you look at the code of the given function you'll notice that it only transfers control to the function _connect_generic() in the base class.
2. The function _connect_generic() does some checks and transfers function to OpenVPN class, i.e. the method real_connect() is called in the file src/nm-openvpn-service.c.
3. The method real_connect() just redirects control to the function _connect_common().
4. The method _connect_common() does some sanity check, obtains some parameters, and calls method nm_openvpn_start_openvpn_binary().
5. The method nm_openvpn_start_openvpn_binary() searches for openvpn binary on a local file system, constructs command line options based on preferences set for the VPN connection and from environment variables, and finally starts openvpn binary (call to function g_spawn_async()). One important part of the command line construction is that openvpn binary is told not to assign parameters itself, but to call helper scripts and pass it all the parameters. This helper script is nm-openvpn-service-openvpn-helper contained in the src directory of networkmanager-openvpn-plugin. So, when OpenVPN binary establishes VPN connection it calls helper script.
   
   It also registers two callbacks. The first one monitors process using a g-child-watch-source-new() function. A callback function is openvpn_watch_cb() that, if called, assumes an error occurred or openvpn binary just finished. The second callback is timer that, when fires, calls function nm_openvpn_connect_timer_cb(). It only tries to connect to OpenVPN's management socket. So, in other words, notification about successful VPN establishment to OpenVPN plugin isn't done using GLib or DBus notification system, but just by waiting and checking. As a final note, timer isn't used in every case. Actually, it seems that it is used rarely, only when authentication type is static key.
6. The nm-openvpn-service-openvpn-helper script, in its main function, collects all network related data from the environment (set by openvpn binary) and sends it via DBus to nm-openvpn-service using methods SetConfig, SetIp4Config and SetIp6Config.
7.  The configuration is caught by functions impl_vpn_service_plugin_set_config(), impl_vpn_service_plugin_set_ip4_config(), and impl_vpn_service_plugin_set_ip6_config() in VPN plugin base class. Those function, in turn, call functions nm_vpn_service_plugin_set_config(), nm_vpn_service_plugin_set_ip4_config() and nm_vpn_service_plugin_set_ip6_config(). They also call finish DBus proxies so that helper script can continue executing after each call.
8. Each of the functions nm_vpn_service_plugin_set_config(), nm_vpn_service_plugin_set_ip4_config() and nm_vpn_service_plugin_set_ip6_config() does two things. It emits signal ("config", "ip4-config" and "ip6-config") from base class and OpenVPN class. [Note: I don't fully understand this mechanism yet]

Literature

It seems to me that it is very hard to come up with a good documenation that describes this topic. So, here are some better references I used:

1. For signals the best reference I could find was from Maemo 4 documentation, available on the following link: http://maemo.org/maemo_training_material/maemo4.x/html/maemo_Platform_Development_Chinook/Chapter_04_Implementing_and_using_DBus_signals.html