Retro computing resources

From time to time I like to search on the Internet stuff related to retro and vintage computing. This time my interest was (again) resolving around a wish to try Xenix. Namely, Xenix was a Unix variant written by Microsoft that could be run on IBM PC XT with 8086 microprocessor. While searching where to download Xenix, I stumbled on a WinWorld site which has many other vintage operating systems available for download. So, a lot of stuff to try. Later, I discovered that there is also software shipped as part of a MAME ROM package. You can browse it here.

Obviously, to try them it is necessary to have an emulator. Note that virtualization solutions like QEmu, VirtualBox, VMWare and similar don't fit here because they emulate modern hardware, or they only virtualize existing hardware. So, I tried to find emulator that emulates PC XT/AT, but also emulators that would allow me to run software for other platforms. After half day search, I decided to write this post with everything I found so far.

One site with some kind of a database of a different emulators is Zophar's Domain. It isn't particularly up to date, but nevertheless it lists a lot of emulators with different categorizations so I think it's very useful. Later, I found several other similar sites:

• Emulation General Wiki
• Wikipedia list of computer system emulators 
• DMOZ directory with emulators

Zophar's site helped me to find emulator for IBM PC XT/AT, namely PCE. This particular emulator emulates a lot more hardware than just IBM's, and what's more interesting is that the author already prepared different configurations for you to download and try. Specifically, there is Xenix 2.13 to download and try. Just one note, these archives are for Windows machines, so if you are - like me - on Linux, download first PCE's source, compile it and then use it instead of provided EXE files. Anyway, I tried it and it worked flawlessly.

As it turns out, there are some other IBM PC XT/AT emulators:

• The first one is 8086tiny. It claims to be the smallest emulator there is. But, I haven't had a lot of success in trying it.
• In case you want something that runs within a Web browser, then you'll be interested in PCjs, i.e. JavaScript emulator. Source is available here. This one seems to be actively developed at the time this post was written.
• The MAME emulator.

And this last one is interesting enough for me to devote few paragraphs to it. MAME is emulator that emulates multiple different machines. What was interesting to me is to try several ones:

• Orao
• Amstrad PC1512

In each case I had to find appropriate ROM files. In the end, the most useful site was Emuparadise. After downloading zip files, I unpacked them into a directory named after the machine as identified in MAME itself and then started the emulation like this:

mame -rp . orao

or in case of Amstrad

mame -rp . pc1512

In each case the option -rp specifies that the current directory should be searched for rome files, while the other argument is driver/machine to be emulated. Note that ROM files will be searched for in the directory named as driver/machine. It happened in some cases that not all ROM files were present, but I managed to find them in other archives on the same site.

For the end, here are some additional interesting links I stumbled on:

• Blog with some posts related to XT
• Documentation about IBM 51xx PC Family Computers including ROM files. There is a lot more information there.
• The Old School Emulation Center (TOSEC) and mirror

Short Tip: Find files with non-printable ASCII characters

I have a directory full of different files obtained from the Internet and it turned out that some of them contain UTF-8 characters because of which indexing didn't work. So, I had to find all files that contain such characters. The solution I found was the following one:

LC_ALL=C find . -name '*[! -~]*'

This command will print all filenames with embedded unicode characters represented as question marks. Few facts about this command:

1. Assignment (LC_ALL=C) temporarily switches to C locale during the execution of find(1) command. The effect of this is that find(1) will not interpret multibyte utf8 characters, but strictly byte-per-byte input.
2. find(1) will then search for file name that don't contain printable ASCII characters. To see this, take a look at a glob pattern. First and last star mean that the square brackets can be anywhere within the file name. Square bracket, on the other hand, specifies class of characters outside (exclamation negates range) of a range from space (ASCII code 32) to tilde character (ASCII code 126).

The output of find(1) command will include question marks in places where byte (ASCII char) has a value below 32 or above 126. In order to see what unicode character is in the particular place, you can pipe output to, e.g. cat(1) command, like this:

LC_ALL=C find . -name '*[! -~]*' | cat

This will work because cat(1) command will have unicode encoding selected (the value of the variable LC_ALL isn't changed for it) and will properly interpret and output multibyte sequences used in utf8 coding. Actually, if you want to nitpick, cat isn't going to interpret anything but will initialize terminal to properly handle utf8 characters which will do actual interpretation. 

A bit about RSS feed readers on Linux

I'm monitoring lot of sites using RSS so having a good RSS feed reader is mandatory for me. Once upon a time, I used Liferea but since I have a lots of RSS feeds with lots of posts I want to keep around, turned out that Liferea wasn't designed with scalability in mind. So, I decided to find another one. Web based readers are out of question, because I prefer desktop applications. Not to mention that locally I have lot of disk storage that I don't have to pay, while storage in the cloud I would have to pay due to my heavy use of it.

After a search I settled on QuiteRSS. In the process I tried RSSOwl but I wasn't able to start it due to different XULRunner version on my Fedora. Besides, it turns out the last version of RSSOwl was released in December 2013, and isn't maintained any more. QuiteRSS was very good, but it turned out that the bug in Webkit started to annoy me. So, I started to explore RSS feed readers again. Note that I have the following requirements:

• No Web application! I want desktop RSS feed reader with GUI interface. It would be nice, though, that I can synchronize it with a reader on a mobile phone!
• I have a large number of feeds and keep a lot of new (that is unread :D) posts around. So, scalability is of paramount importance.
• And last, but not least, nice looking and usable GUI. 

This brought me to three candidates: QuiteRSS, FeedReader and RSSGuard. I'll describe each of them in a bit more details below. But before that, note that this is a live post, i.e. I'll still try all the mentioned readers and update it with new experience. Also, I would like to hear you comments/sugestions, so if you have any, please leave a comment.

QuiteRSS

QuiteRSS is quite good and I'm using it all the time. There is a homepage and GitHub development page. It has the ability to tag posts, mark them as a read, etc.

It is interesting to look at QuiteRSS GitHub page. From there, the following conclusions can be inferred:

1. QuiteRSS is quite popular, 33 watches, 180 stars and 28 forks.
2. QuiteRSS is basically in maintenance mode since there is no substantial activity since 2014. From 2012 to 2014 development was very intensive.
3. There are 212 open issues and 719 closed ones. I think that there are a lot of open issues but more thorough statistics has to be performed to know for certain.

The problems are the following ones, from the most important to the least important ones:

• You have to disable JavaScript because QuiteRss often freezes on some feeds while loading. It still freezes with some RSS feeds and if that happens some history is lost (read feeds, marked/tagged feds, etc).
• If you accidentally click on a link to PDF file, QuiteRSS freezes!
• Once I mistakenly selected the option "Mark all news read" which is irreversible. There is no confirmation dialog for such cases.
• Some posts on GitHub are in Russian. That's a problem because not everyone is speaking Russion. ;)
• It depends on Qt4 and Webkit4 that are not maintained any  more.

FeedReader

FeedReader is a interesting because it has two components, daemon and a front end. This is uniqe to other readers that bundle those functions together into a single binary. You can read more about this reader on its homepage, and there is also GitHub development page. Looking at the Web page, it has a lots of features but I'm using only a few, if any at this stage. Take this into account while reading this review. Looking at the GitHub page of FeedReader, the following conclusions can be inferred:

1. FeedReader is somehow less popular than QuiteRss. It has 26 watches (against 33 for QuiteRss), 152 stars (against 180) and 6 forks (against 28).
2. FeedReader is in active development, and all the activity is concentrated in 2016 with some additional in 2015.
3. There are 27 open issues and 197 closed ones. This is better ratio than for QuiteRss, but again more research has to be done!

First problem I had was while removing feeds. It was painful because it doesn't allow selection of multiple feeds or feed groups at once.

The next problem was that only two level hierarchy supported, while in QuiteRSS I have three level. So, importing OPML file with multiple levels will result in transforming everything into two layers.

While removing certain feed folders, some of them kept coming back! Maybe the problem was that I right-clicked on a feed and selected delete but it was necessary to first left-click and then right-click. Who will know...

RSS Guard

RSS Guard, as all the other feed readers mentioned above, has its GitHub development page. As for the homepage, it uses Wiki on GitHub. Again, by looking into GitHub page, the following conclusions can be made:

1. RSSGuard has 6 watches, 21 stars and 6 forks. This makes it the lowest ranked by popularity of the three RSS readers reviewed here.
2. RSSGuard is in development since 2013 with evenly spread development efforts. This probably means it isn't going to be finished soon.
3. It has 11 open issues reported and 51 closed. Which isn't that bad.

So, some shortcomings from the personal experience. It is a bit non-intuitive. It took me some time to realize that in order to import OPML file, first I have to create account. Another non-intuitive task was the process of importing itself. When you select OPML file and all the feeds appear, you click OK, but then you have to click Close. First time I clicked twice OK and got all the feeds imported twice!

It support multilevel feed organization, but it is not possible to fold certain feed groups, i.e. they are always unfolded! I finally realized that it is possible to fold a folder, you just need to click twice in order to fold/unfold it. But, this isn't something particularly intuitive, nor visible. Namely, if the folder is folded there is no indication nor there is indication that that the folder can be folded.

When I click "Update all items" button in a toolbar, I expected that all feeds will be updated. But for some reason, that didn't happen.

Conclusion

Comparing development of each of the proposed readers, it turns out that each one of them basically depends on a single developer and has its own pros and cons. In the end, I think that despite its shortcomings, QuiteRSS is still the best feed reader closely followed by FeedReader. If development activity of FeedReader continues with the same intensity, expect that it will become the best RSS among the three.

ChangeLog

• 20161101 - Initial version

Fedora 24, kernel 4.7 and VMWare Workstation 12.1.

As usual, when Fedora upgrades kernel to a new major version VMWare stops working. In this particular case my Fedora was upgraded to Linux kernel version 4.7.2. The fix I found is the following one:

# cd /usr/lib/vmware/modules/source
# tar xf vmnet.tar
# mv vmnet.tar vmnet.old.tar
# sed -i -e 's/dev->trans_start = jiffies/netif_trans_update\(dev\)/g' vmnet-only/netif.c
# tar cf vmnet.tar vmnet-only
# vmware-modconfig --console --install-all

And that was it!

Note that this works if you upgraded from linux kernel version 4.6. In case you've upgraded from some earlier version, you'll have to patch it first for those version. Here is how to patch VMWare Workstation to work with Linux kernel version 4.6.

A little bit about ZXDSL 931VII

These are short notes about ZTE's home router ZXDSL 931VII modified and distributed by Croatian Telecom (abbreviated as HT from Croatian name Hrvatski Telekom). I had some problems while trying to access CLI of this router and Google didn't return anything usefull. Finally I managed to solve those problems and this is a log of what I did so that I have a reminder and also with a hope that this will be useful to someone else.

So, the key to everything is a configuration file that can be downloaded from the router itself by the ordinary user. Basically, you should click on Administration menu option, then select User Configuration Management and in the main screen you'll have Backup Configuration button. Click on it and you'll get a file config.bin. This is a binary file with a full configuration of the router, and luckily, it is not encrypted. Now, download Python script from the Pastebin page. This script will convert binary file into a text form and you'll have access to a lot of goodies inside. :) In the following subsection I wrote some interesting stuff I managed to obtain from the given file.

CLI Access

The holly grail of any advanced user is, of course, command line interface. So, the question is how to do it. It used to be simple in previous models of the HT's, just do telnet, use administrator user name and that's it. In this model, you'll have to do it slightly differently:

1. Telnet to the device, but use username tech. The password you'll find in the configuration file. Just search for tech username.
2. Now, you'll get prompt "CLI>" in which you should type command enable. Note that you can use question mark (?) to get a list of available commands.
3. After typing enable, you'll be requested to provide password. Password is zte which can also be found in configuration file. You'll spot easily, it doesn't have associated username.
4. Now, type command shell . You'll be asked to provide username and password. Type root for username, and again root for password.

Now, you a presented with a greeting message from busybox as well as a prompt:

BusyBox v1.01 (2013.09.12-09:36+0000) Built-in shell (ash)

Enter 'help' for a list of built-in commands.

#

Usernames and password

You'll also find in the configuration file all the usernames and the related passwords, for DSL, VoIP, etc.

Quick note about Fedora 24 and VMWare Workstation 12.1

I just updated Fedora 24 from update-testing repository and that pulled Linux kernel 4.6. Well, as usual, VMWare Workstation needed some patching in order to work. Luckily, I quickly found a fix on VMWare forums. Note that at the end of the thread there is a script you can use to automatically patch necessary files. But, be careful, I didn't try it!

Anyway, after patching, run:

vmware-modconfig --console --install-all

and that should be it!

Just as a sidenote, turns out that the same info I found on Arch Wiki page devoted to VMWare. And that page is full of information so you should bookmark it and whenever you have a problem with VMWare check there.

Edit 20160707: The script mentioned above had some errors in it. Here is the fixed version.

Network namespaces and NetworkManager

This post documents a process to implement support for network namespaces in the NetworkManager. The code described in the post can be found on GitHub. While my personal motivation to add namespace support to NetworkManager was to be able to add support for provisioning domains as specified by IETF MIF WG, it also brings benefits to existing users by allowing isolation of different applications into different network namespaces. The best example is a VPN connection isolation. When network namespaces are used then certain applications can be started in namespace in which only network connectivity is via VPN. Those applications can access VPN resources while all the other applications, that are in different network namespaces, will not see VPN connection and thus couldn't use them. The additional benefit would be from using multiple connections, as described by MIF Architecture RFC.

Note that after I started to work on this, Thomas Haller implemented basic support for namespaces in NetworkManager that was a bit different in some implementation details from mine.

The idea

The intention of implementing support for network namespaces was to allow applications to be isolated so that they use specific network connections. Network namespaces in Linux kernel have some properties that anyone that wants to use them must be aware of. This is documented in a separate post.

So, an application can not be moved between different network namespace by some other application, i.e. only application itself can change network namespaces, and only if it has appropriate permissions.

So, the idea is the following one. Application is started in some network namespace. This can be done easily (e.g. see 'ip netns exec' command). Then, this network namespace is manipulated by either the application itself - it is aware of NM's support for network namespaces, or by third party application (that application could be nmcli). The manipulation means that requests are sent to NetworkManager via D-Bus to make changes to network namespace. The changes can be activation and deactivation of certain connections. NetworkManager, based on those requests and on specifics of connections and devices those connections are bound to, determines what to do. For example, it can create virtual device in the network namespace, or can move physical device. Basically, this part isn't important to the application itself, the only thing that is important is that the application is assigned requested connections.

Implementation

The following changes were made to NetworkManager in order to introduce support for network namespaces:

1. Created new object NMNetnsController whose purpose is to allow management of all network namespaces controlled by NetworkManager. Via the interface org.freedesktop.NetworkManager.NetworkNamespacesController it is possible to create a new network namespace, or to remove the existing one. It is also possible to obtain a list of existing network namespaces.
    
2. Created new object NMNetns that represents a single network namespace. So, when new network namespace is created a new object NMNetns is created and exposed on D-Bus. This object allows manipulation with network namespace via the interface org.freedesktop.NetworkManager.NetNsInstance. So, it is possible to get a list of all devices within the network namespace, take certain device from some other network namespace and to activate some connection.
    
3. NMSettings is now singleton object. This wasn't so significant change because there was also one object of this type before, but now it is more explicitly exposed as such.
    
4. NMPlatform, NMDefaultRouteManager and NMRouteManager aren't singleton objects any more. They are now instantiated for each new network namespace that is created.

VPN isolation

VPN isolation was done as a first user of network namespaces implementation. It was easier then other connections because the assumption was that VPN connection should live only in single network namespace and it should be the only available connection.

At the beginning, there was doubt on where to place the knowledge of network namespace and two places were candidates, in NMActiveConnection and NMVPNConnection classes. NMActiveConnection is actually a base class of NMVPNConnection class. The modification of NMVPNConnection approach is better for the following reason because the idea was to introduce new configuration parameter in the configuration file of a VPN connection that will specify that isolation is necessary and also some additional behaviors:

• netns-isolate
  
  Boolean parameter (yes/no) which defines weather VPN connection should be isolated within a network namespace or not. For backwards compatibility reasons
   
• netns-persistent
  
  Should network namespace be persistant (yes) or not (no). Persistant namespace will be retained when VPN connection is terminated, while non-persistant will be removed.
   
• netns-name
  
  Network namespace name. Special value is uuid which means connection's UUID should be used, also name is special value that requests connection's name to be used. Finally, any other string is taken as-is and used as a network namespace name.
   
• netns-timeout
  
  How much time to wait (in milliseconds) for device to appear in target namespace.

Basically, the implementation is such that when device appears in root network namespace it is taken from there (using TakeDevice method, but called directly instead via D-Bus).  When device appears in the target network namespace network parameters are assigned to the interface. This was tested with OpenVPN type of VPN.

The implementation has two problems. First, the case of VPN connections that don't create virtual devices but instead just modify packet processing rules in the Linux kernel (i.e. XFRM). Secondly, hostname and name resolution parameters aren't assigned because the infrastructure is lacking in that respect.

Conclusion

The initial goal of having network namespaces support in NetworkManager was achieved. There are functionalities missing like isolation of any connection, hostname handling and DNS resolution handling. Those are things that will have to be resolved in the future.

NetworkManager and multiple provisioning domains

The goal of this post is to list different options on how to introduce PvDs into the NetworkManager, i.e. what should be changed in NetworkManager and how it should handle explicit and implicit PvDs. But first we'll start with the definition of Provisioning Domain and object that could potentially be used to store/represent Provisioning Domains. The implementation this post refers to can be found on GitHub.

The term Provisioning Domain (PvD) is defined and clarified in RFC7556 as:

A consistent set of network configuration information. Classically, all of the configuration information available on a single interface is provided by a single source (such as a network administrator) and can therefore be treated as a single provisioning domain.  In modern IPv6 networks, multihoming can result in more than one provisioning domain being present on a single link.  In some scenarios, it is also possible for elements of the same PvD to be present on multiple links.

Basically it is a set of configuration information that should be treated as a single unit. Here are some examples of such units of configuration data:

1. Static IPv4 configuration provided by a user for a server or for a network without DHCP.
2. Data handed over to a client by DHCP server.
3. On an IPv6 enabled local network with a single router which sends configuration data in RA to nodes attached to the network.
4. Configuration data sent by VPN gateway upon successful connection of a client. 

In all these cases we have implicit PvDs, meaning that the sets of configuration data are implicitly bound together and there was no indication whatsoever that they should be treated as a single unit. This is in contrast to explicit PvDs which are sets of configuration data bound together by some explicit mechanism and associated with some kind of a PvD identifier sent to a client in some way. Explicit PvDs, as of time this post was written, don't exist yet, and the IETF MIF working group is trying to define necessary mechanisms to support them as well as how exactly IDs should look like.

Note that apart from explicit PvD and implicit Pvd we also differentiate between PvD and PvD instance. The difference is that PvD consists of a set of PvD instances thate are the same on some local network, while PvD instance is valid for only a single host on a given local network. In other words, PvD will include network prefix and mask, while PvD instance will include host addresses too. It is interesting to note that router advertisements communicate PvDs while DHCP communicates PvD instaces.

How to implement PvDs in the NetworkManager

As always, the same goal can be achieved in multiple ways, so here are the options on how PvDs can be implemented within NM. Basically, there are two main approaches: first, existing objects can be enhanced so that they can represent PvDs or a completely new object can be introduced.

Using NMSettingsConnection object to store PvD and PvD instance

Each network connection (which is not the same as PvD or PvD instance) is stored in NMSettingsConnection object. Those objects are generated from static files or dynamically during NetworkManager's execution. NMSettingsConnection objects are initialized from the following sources:

1. Distribution configuration files. System dependent network configuration files (e.g. /etc/sysconfig/network-scripts for RHEL based systems) are read by NM via plugins and NMSettingsConnection objects are created as a result.
    
2. Network manager specific configuration. NetworkManager has its own configuration files that are stored in /etc/NetworkManager/system-connections/.
   
3. Dynamically created configurations. While running, NetworkManager allows new configurations to be created via D-Bus interface.

Note that NetworkManager has a concept of profiles that are used in the case of Wired networks. Basically, those are settings which are not bound to any specific network interface. Profiles can have 802.1x type of credentials assigned to them.

So, the idea of integrating PvDs into NetworkManager is for each new PvD or PvD instance to create a new NMSettingsConnection object. The modification to NMSettingsConnection should be extended with PvD ID parameter.

There are several potential problems with this approach:

1. There is a difference between NMSettingsConnection on the one hand, and PvD and PvD instance on the other hand. For example, some NMSettingsConnection defines a network connection that should be configured using DHCP and in that case the NMSettingsConnection isn't PvD nor PvD instance. On the other hand, NMSettingsConnection can be the same as PvD instance. This is the case with static IPv4 configurations when a user specifies concrete IP addresses. Finally, NMSettingsConnection can be PvD only in the case of IPv6 when host part is generated from MAC address.
   
2. When PvDs and PvD instances are received they are valid only for the interface on which they are received. But, a user can request any NMSettingsConnection object to be activated on any interface which isn't possible.
   
3. Also, this can create confusion. Take for example preconfigured NMSettingsConnection which is now treated as PvD with a specific PvD ID, and it is defined to use DHCP for the configuration. Obviously, this PvD ID is expected to be valid on a certain interface on a specific attachment point. But due to the way the interface is configured (DHCP) it can actually be activated on any interface on any network that supports DHCP. Thus, it might easily happen that a user by mistake activated this particular NMSettingsConnection on a "wrong" network and so makes a user believe the network is active while in the reality it is not.
   
   Note that even NMSettingsConnection objects that contain credential information aren't guaranteed to retrieve the same PvD every time the connection is made. Namely, there are AAA servers and infrastructure that allow clients with a same credentials to connect to multiple networks, and thus to potentially receive multiple PvDs.
    
4. Finally, the problem is that on a single network interface only one NMSettingsObject might be activated and so this prevents having multiple PvDs on a single interface.

Those problems are not unsolvable, i.e. they could be solved by modifying certain aspects of the NetworkManager in general, and NMSettingsObject in particular.

Treating NMActiveConnection object as PvD instance and PvD

Whenever a connection is made in NetworkManager an object is created. Basically, there are two classes for the object, both of which inherit from NMActiveConnection base class. Which class is used depends on the type of the connection. Basically,  the only distinction is made between VPN connections that are represented by NMVPNConnection objects and other connections that are represented by NMActRequest objects. The main task of NMActiveConnection is to bind NMSettingsConnection with NMDevice objects.

The idea in this case is to treat NMActiveConnection as a PvD or a PvD instance, i.e. on each new PvD or PvD instance received new NMActiveConnection is created.

But, there are still some problems:

1. Since NMActiveConnection objects are transient that means that there would be no history of PvDs used. This might, or might not be a problem, depending on whether we need this history or not.
   
   The cases when the history would be necessary is if we cache some information for the next time we connect to the given PvD. The second case is if there are processes still using PvD through API and thus the information about PvD must live until the process dies. Note that this letter problem could be solved with delayed removal of NMActiveConnections or by some asynchronous mechanism informing applications that specific NMActiveConnection isn't available any more.
    
2. The second problem is the question if there could exist two ActiveConnection objects that were created from the same NMSettingsConnection object, i.e. can NMSettingsConnections be shared.
    
3. The third problem is that it will happen from within a single NMActiveConnection that two or more PvDs are received and this requires that NMActiveConnection is a factory for itself.

Using NMIP4Config and NMIP6Config objects for PvDs and PvD instances

NetworkManager has object/classes for storing IPv4 (libnm-core/nm-setting-ip4-config.c) and IPv6 (libnm-core/nm-setting-ip6-config.c) settings. More precisely, those objects are used to expose network settings of devices to the rest of the NetworkManager. So, in some way they are PvDs in a sense that each of them contains enough information to allow connection to the network.

The problem is that internally NetworkManager keeps a single IPv4/IPv6 configuration object per device and in addition it merges all received configuration data on a single interface.

Specifically, in case of configuration data received in RAs everything is kept in the object NMRdisc defined in src/rdisc/nm-rdisc.h. There you'll find arrays of received configuration data. NetworkManager assumes that a single router sends all the configuration data. This assumption is not valid on a multihomed network, or a network that can send multiple provisioning domains within each RA. What would be necessary is to change this structure so that configuration data is kept separate for each router and provisioning domain.

The problems in this case are:

1. NMIPxConfig objects were not intended to keep information about available IPv4 and IPv6 addresses but to make available addresses configured on device. So, it reverses the purpose of those objects which isn't accepted so well.
   
2. Again, those are transient objects and thus there is no history. It is possible to keep every object alive, but NM isn't designed to behave in such way.
   
3. It seems that in libnm there is no way to obtain a list of IPv4 and IPv6 objects.

Having separate PvD structures

This is the final alternative and the most intrusive one. The idea is that settings, active connections and IPv6 and IPv6 objects/classes stay as is, but instead, when each new connection is established a new PvD data structure is created. PvD is inferred from configuration settings or the NetworkManager received explicit PvD.

This would solve the problem that some settings might be used to obtain different PvDs which isn't known until connection is established. For example, if we are using DHCP to configure the interface, then, PvD received depends on the PoA.

It would also solve the problem that the user might try to instantiate one PvD, while some other is actually in use. This way, after the connection is established, appropriate PvD is searched for, or new one is created.

This is most intrusive change that would require change in APIs and thus break compatibility with the existing applications (or require a completely new API).

Current PvD Support Implementation

The first implementation of PvDs was done using NMIP6Config as a PvD container. Before describing the implementation we have to state that the only mechanism currently able to carry PvDs is RA messages. NMIP6Config objects are extended with PvD ID field. At first, there was support for different types of PvD IDs and the first implemented type was UUID stored in ASCII format. Later in the development process PvD ID types were removed and the only possible type is UUID. It seems that this doesn't make implementations less flexible and in the same time substantially reduces complexity.

When RA is received, and after it is processed as usual, a new implicit PvD is created from data in RA. If there are two or more routers on the local network, each sending its own configuration data, then a separate PvD is created for each RA. Also, in case there are PVD container option in RA it is parsed and additional PvD is created from that data.

This information is then handled to NMDevice object which merges data from implicit PvDs (as it does in the unmodified version) but now there is also a hash table with set of PvDs received on the given interface. This information is then exposed through NMActiveConnection object.

Few notes about network namespaces in Linux

For some time I'm working with network namespaces as implemented in the Linux kernel. Here I'll collect some notes about the implementation, behavior, usage and anything else I learn while using network namespaces.

Kernel API for NETNS

Kernel offers two system calls that allow management of network namespaces. The first one is for creating a new network namespace, unshare(2). Actually, this system call allows other types of namespaces to be created, but here we are interested only in network namespaces. So, to create a new network namespace you should call the function like this:

#include <sched.h>
...
    unshare(CLONE_NEWNET);
...

And that would define a new network namespace.

There are two ways other processes can now use that network namespace. The first approach is for the process that created new network namespace to fork other processes and each forked process would share and inherit the parent's process network namespace. The same is true if exec is used.

The second system call kernel offers is setns(2). To use this system call you have to have a file descriptor that is somehow related to the network namespace you want to use. There are two approaches how to obtain the file descriptor.

The first approach is to know the process that lives currently in the required network namespace. Let's say that the PID of the given process is $PID. So, to obtain file descriptor you should open the file /proc/$PID/ns/net file and that's it, pass file descriptor to setns(2) system call to switch network namespace. This approach always works.

The second approach works only for iproute2 compatible tools. Namely, ip command when creating new network namespace creates a file in /var/run/netns directory and bind mounts new network namespace to this file. So, if you know a name of network namespace you want to access (let's say the name is NAME), to obtain file descriptor you just need to open(2) related file, i.e. /var/run/netns/NAME.

Note that there is no system call that would allow you to remove some existing network namespace. Each network namespace exists as long as there is at least one process that uses it, or there is a mount point.

Two remarks for the end of this section. First, there is no system call that would allow one process to move some other process into another network namespace! And second, you need appropriate privileges to use the mentioned system calls, i.e. regular user processes can't switch namespaces.

Socket API behavior

The next question is how Socket API behaves when network namespaces are used, and things here are quite interesting.

First, each socket handle you create is bound to whatever network namespace was active at the time the socket was created. That means that you can set one network namespace to be active (say NS1) create socket and then immediately set another network namespace to be active (NS2). The socket created is bound to NS1 no matter which network namespace is active and socket can be used normally. In other words, when doing some operation with the socket (let's say bind, connect, anything) you don't need to activate socket's own network namespace before that!

Also, to note is that network namespace is per-thread setting, meaning if you set certain network namespace in one thread, this won't have any impact on other threads in the process.

Command line tools

There are two command line tools available to manipulate network namespaces. The first one is nsenter(1) which isn't specific to networking. It allows one to start some process within predefined network namespace. The second tool is ip command from iproute2 package. It allows management of network namespaces and also allows network interfaces to be switched between different namespaces.

NETLINK behavior

To change device from one network namespace to another one, NETLINK must be used. I found somewhere references to /sys files, but at least on my system they don't appear to exist.

One interesting fact is that interface ID is global across all network namespaces - except for loopback interface, i.e. if you create interface in one network namespace and it gets ID N, and then you move it to another network namespace, it will keep ID N.

TBD.

NetworkManager architecture

After figuring out how VPN establishment works in NetworkManager the next big question is about the architecture of NetworkManager and its run-time behavior. This post tries to provide answer to the given question. This post consists of two parts. The first part, Source code organization, gives information about the source tree organization of NetworkManager. The second part, Run-time initialization behavior has a goal to describe initialization of NetworkManager and its key parts. Note that I'll update this post as I learn about NetworkManager.

The last update time of this post was: February 2nd, 2016.

Source code organization

So, let us start with a source tree organization. If you take a look at the top level directory of the NetworkManager git tree, you'll find there some files and the set of directories. The files are build scripts and configuration files, there are also some documentation files. But, in the end all the interesting functionality is placed within the top level directories. There are the following top level directories:

• callouts
  
  TBD
   
• clients
  
  Clients to control NM like nmcli or nmtui.
   
• introspection
  
  contains XML description of DBus interfaces provided by NetworkManager and its components. For example, VPN plugins are external to NetworkManager but they use DBus to communicate with NetworkManager and there is an XML file that describes VPN plugin's interfaces. All the descriptions are converted into C code stubs and skeletons using dbus-binding-tool. There is a lot of magic staff happening here.
   
• libnm
  
  Libraries intended to be used by clients accessing and using services provided by NetworkManager. This also includes application that control NetworkManager itself (like nmcli and nmtui)
   
• libnm-core
  
  Core NM libraries used by NetworkManager itself and all the other components, like command line utilities.
   
• libnm-glib, libnm-util
  
  Old NM libraries that are deprecated starting with 1.0 release of NetworkManager. Still, it seems that at least some functionality is used from those libraries so probably until all the pieces of NetworkManager are converted to the new libraries they are to stay. Note that in my posts I ignore the content of those libraries as much as possible and I concentrate on the previous two libraries, libnm and libnm-core.
   
• src
  
  The directory with the code specific to the NetworkManager unlike the previous directories that contain code used by different programs. The src/ directory contains file and  a number of subdirectories:
   
• devices/
  
  Classes and sub classes used to represent different networking devices, real and virtual, that can be managed by NetworkManager.
   
• dhcp-manager/
  
  Manager class that manages and interfaces DHCP clients to NetworkManager.
   
• dns-manager/
  
  
   
• dnsmasq-manager/
  
  
   
• platform/
  
  Platform specific code that interfaces NetworkManager to particular operating system it is running on. For the moment there is a base class and derived Linux specific class.
   
• ppp-manager/
  
  
   
• rdisc/
  
  Code to monitor and parse IPv6 router advertisement messages.
   
• settings/
  
  Code to keep track of connections. Since connections are read from configuration files that depend on particular distributions this directory also contains plugins, one plugin for each supported distribution.
   
• supplicant-manager/
  
  
   
• systemd/
  
  
   
• vpn-manager/
  
  VPN specific code. Here is a manager class that is used to manage all the VPN connections and also class that is used to represent each active VPN connection.
   

Run-time initialization behavior

When NM is started execution starts from main() function that can be found in src/main.c. The goal of the main function is to parse command line options, check some prerequisites for running NM, do initialization steps and start glib main loop. The most interesting are initialization steps which are:

• Initialize logging.
   
• Create configuration object singleton, NM_TYPE_CONFIG (defined in file src/nm-config.c).
   
• Creates a singleton for managing DBus connections and communication, NM_TYPE_BUS_MANAGER (defined in file src/nm-bus-manager.c).
   
• Creates a singleton that is platform dependent used for management of network subsystem, NM_TYPE_LINUX_PLATFORM (defined in file src/platform/nm-linux-platform.c).
   
• Creates a main singleton object of class NM_TYPE_MANAGER (defined in file src/nm-manager.c).
   
• Initializes loopback interface.

So, while NetworkManager is running it basically consists of a set of singleton objects that communicate and make what is named NetworkManager.

References

• https://wiki.gnome.org/Projects/NetworkManager/libnm
• http://events.linuxfoundation.org/sites/events/files/slides/networking.pdf
• https://blogs.gnome.org/dcbw/2009/06/25/networkmanager-and-connman/

Connections in NetworkManager

Connections, as defined and used by NM, are very close to PvDs. The goal of this post is to analyse data structures/functions for connections within NetworkManager and so that plan of integrating PvDs into NM can be developed. This is done in a separate post.

A definition of connection in NetworkManager can be found in the comment within the libnm-core/nm-connection.c file:

An #NMConnection describes all the settings and configuration values that are necessary to configure network devices for operation on a specific network. Connections are the fundamental operating object for NetworkManager; no device is connected without a #NMConnection, or disconnected without having been connected with a #NMConnection.

Each #NMConnection contains a list of #NMSetting objects usually referenced by name (using nm_connection_get_setting_by_name()) or by type (with nm_connection_get_setting()). The settings describe the actual parameters with which the network devices are configured, including device-specific parameters (MTU, SSID, APN, channel, rate, etc) and IP-level parameters (addresses, routes, addressing methods, etc).

In the following text we'll see how connections are implemented in the NM code, how they are initialized and how they are accessed over the DBus. Note that there are some parts related to connections that are specific for a system/distribution on which NM is running. In that case I concentrate on how things are done on Fedora (and very likely all RHEL derivatives).

Class and interface hierarchy

The base for all connection objects in NetworkManager is the interface defined in the files libnm-core/nm-connection.[ch]. The interface is implemented by the following classes:

• Class NMSettingsConnectionClass defined in the files  src/settings/nm-settings-connection.[ch].
  
  This class is used by NetworkManager daemon and it is exported via DBus interface.
   
• Class NMRemoteConnectionClass defined in the files libnm/nm-remote-connection.[ch].
  
  Used by clients in clients subdirectory.
   
• Class NMSimpleConnectionClass defined in the files libnm-core/nm-simple-connection.[ch].
  
  This is the object passed via DBus so it is for communicating connections from and to NetworkManager and its clients.

Accessing individual connection data stored in NetworkManager

Each connection, active or not, known to the NetworkManager is exposed through DBus on path /org/freedesktop/NetworkManager/Settings/%u where %u is a sequence number assigned to each connection. Interface implemented by each connection is org.freedesktop.NetworkManager.Settings.Connection. The interface is described in introspection/nm-settings-connection.xml file.

To invoke a method defined in interface on the given object you can use dbus-send command line tool like this:

dbus-send --print-reply --system \
    --dest=org.freedesktop.NetworkManager \
    /org/freedesktop/NetworkManager/Settings/0 \
    org.freedesktop.NetworkManager.Settings.Connection.GetSettings

In this particular case we are invoking GetSettings method on object /org/freedesktop/NetworkManager/Settings/0 which will return us its configuration parameters. Note that invoking this particular method is easy since there are no arguments to the method.

The class that represents connection, and that is answering to DBus messages, is declared in src/settings/nm-settings-connection.[ch] files. This class implements interface NM_TYPE_CONNECTION and also subclasses NM_TYPE_EXPORTED_OBJECT class.  The NM_TYPE_EXPORTED_OBJECT class has all the methods necessary to expose the object on DBus.

To see what functions are called when DBus methods are called take a look at the end of the source file nm-settings-connection.c. There, you'll find the following code:

nm_exported_object_class_add_interface (NM_EXPORTED_OBJECT_CLASS (class),
    NMDBUS_TYPE_SETTINGS_CONNECTION_SKELETON,
    "Update", impl_settings_connection_update,
    "UpdateUnsaved", impl_settings_connection_update_unsaved,
    "Delete", impl_settings_connection_delete,
    "GetSettings", impl_settings_connection_get_settings,
    "GetSecrets", impl_settings_connection_get_secrets,
    "ClearSecrets", impl_settings_connection_clear_secrets,
    "Save", impl_settings_connection_save,
    NULL);

What this code does is that it binds GBus methods to function that should be called. When we called GetSettings method, obviously that ended up in the function impl_settings_get_settings().

The first step done when processing GetSettings method is authorization check. After authorization check has succeeded, the return message is constructed in get_settings_auth_cb() method.

Accessing and manipulating all connections in NetworkManager

NetworkManager exposes interface org.freedesktop.NetworkManager.Setting on object org.freedesktop.NetworkManager.Setting that, among other things, allows the user to retrieve list of all the known connections to the NetworkManager.  To get all connections you would could use the following dbus-send command:

dbus-send --print-reply --type=method_call --system \
        --dest=org.freedesktop.NetworkManager \
        /org/freedesktop/NetworkManager/Settings \
        org.freedesktop.DBus.Properties.Get \
        string:org.freedesktop.NetworkManager.Settings \
        string:"Connections"

This would get you something like the following output:

variant array [
   object path "/org/freedesktop/NetworkManager/Settings/10"
   object path "/org/freedesktop/NetworkManager/Settings/11"
   object path "/org/freedesktop/NetworkManager/Settings/12"
   object path "/org/freedesktop/NetworkManager/Settings/13"
   object path "/org/freedesktop/NetworkManager/Settings/14"
   object path "/org/freedesktop/NetworkManager/Settings/15"
   object path "/org/freedesktop/NetworkManager/Settings/0"
   object path "/org/freedesktop/NetworkManager/Settings/1"
   object path "/org/freedesktop/NetworkManager/Settings/2"
   object path "/org/freedesktop/NetworkManager/Settings/3"
   object path "/org/freedesktop/NetworkManager/Settings/4"
   object path "/org/freedesktop/NetworkManager/Settings/5"
   object path "/org/freedesktop/NetworkManager/Settings/6"
   object path "/org/freedesktop/NetworkManager/Settings/7"
   object path "/org/freedesktop/NetworkManager/Settings/8"
   object path "/org/freedesktop/NetworkManager/Settings/9"
   object path "/org/freedesktop/NetworkManager/Settings/16"
]

The exact output will depend on your particular setup and usage.

The given interface and property is implemented by class NMSettingsClass (defined in src/settings/nm-settings.[ch]). This class implements interface NM_TYPE_CONNECTION_PROVIDED (defined in src/nm-connection-provider.[ch]). There is only one object of this class in NetworkManager and it is instantiated when NetworkManager is starting.

Looking in the file src/settings/nm-settings.c you can see at the end registration of function to be called when DBus messages are received. DBus interface of this module is defined in introspection/nm-settings.xml file. Here is the relevant code that binds DBus methos to the functions that implement them:

nm_exported_object_class_add_interface (
        NM_EXPORTED_OBJECT_CLASS (class),
        NMDBUS_TYPE_SETTINGS_SKELETON,
        "ListConnections", impl_settings_list_connections,
        "GetConnectionByUuid", impl_settings_get_connection_by_uuid,
        "AddConnection", impl_settings_add_connection,
        "AddConnectionUnsaved", impl_settings_add_connection_unsaved,
        "LoadConnections", impl_settings_load_connections,
        "ReloadConnections", impl_settings_reload_connections,
        "SaveHostname", impl_settings_save_hostname,
        NULL);

So, when we called ListConnections method, obviously that ended up in the function impl_settings_list_connections(). Here, we'll emphasize one more method, LoadConnections. This DBus method, implemented in impl_settings_load_connections(), load all connections defined in the system. We'll take a look now at that method.

Initializing connections

All network connections are loaded and initialized from two sources: system dependent network configuration and VPN configuration scripts.

System dependent network configuration

There are several types of distributions with different network configuration mechanisms. Since that part is obviously system dependent, NetworkManager has a plugin system that isolates the majority of NetworkManager code from those system dependent parts. Plugins can be found in the directory src/settings/plugins. Additionally, all the plugins are based on the src/settings/plugin.[ch] base class. In the case of Fedora Linux (as well as RHEL, CentOS and other derivatives) network configuration is recorded in scripts in the directory /etc/sysconfig/network-scripts. and the plugin that handles those configuration files is stored in the directory src/settings/plugins/ifcfg-rh.

The initialization of connections stored in the directory /etc/sysconfig/network-scripts is done when NetworkManager bootstraps and instantiates object NMSettings of type NMSettingsClass. This is performed in the function nm_manager_start() in the file src/nm-manager.c. There, the method nm_settings_start() is called on the NMSettings object which in turn first initializes all plugins (as, by default, found in the directory /usr/lib64/NetworkManager/). It then calls private method load_connections() to actually load all connections. Note that in the directory  /usr/lib64/NetworkManager/ there are plugins of other types too, but only plugins that have prefix libnm-settings-plugin- are loaded. Which plugins should be loaded are can be defined in three places (lowest to highest priority):

1. Compile time defaults, as given to configure.sh script, or, by default for RHEL type systems "ifcfg-rh,ibft" plugins.
2. In configuration file /etc/NetworkManager/NetworkManager.conf.
3. As specified in the command line.

Method load_connections() iterates over every defined plugin and asks each plugin for all registered connections it knows by calling a method get_connections() within a specific plugin. For RedHat type of distributions the the plugin that handles all connections is src/nm-settings/plugins/ifcfg-rh/plugin.c and in that file function get_connections() is called. Now, if called for the first time, this function will in turn call read_connections() within the same plugin/file that will read all available connections. Basically, it opens directory /etc/sysconfig/network-scripts and builds a list of all files in the directory. Than, it tries to open each file and only those that were parsed properly are left as valid connections. Each found connection is stored in object NMIfcfgConnection of type NMIfcfgConnectionClass. These objects are defined in files src/nm-settings/plugins/ifcfg-rh/nm-ifcfg-connection.[ch].

When all the connections were loaded, read_connections() returns a list of all known connections to the plugin. The function load_connections() then, for each connection reported by each plugin, calls claim_connection() method in nm-connection.c. This function, among other tasks, exports the connection via DBus in a form described above, in the section Accessing individual connection data stored in NetworkManager.

VPN configuration scripts

Details about VPN connections are stored in /etc/NetworkManager/system-connections directory, one subdirectory per VPN. Those files are read by src/vpn-manager/nm-vpn-manager.c when the object is initialized and as such initialized when VPN manager is initialized. VPN manager also also monitors changes in the VPN configuration directory and acts appropriately.

Properties of a connection

Each connection has a set of properties attached to it in a form of a key-value pairs.

Activating a connection

A connection is activated by calling ActivateConnection DBus method. This method is implemented in the NetworkManager's main class/object, NMManager. This class/object is a singleton object who's impementation is in src/nm-manager.[ch] files. Looking at the code that binds DBus methods to the functions that implement them we can see that ActivateConnection is implemented by the function impl_manager_activate_connection(). The ActivateConnection method, and its implementation function, accept several parameters:

1. Connection that should be activated identified by its connection path (e.g. "/org/freedesktop/NetworkManager/Settings/2").
2. Device on which connection should be activated identified by its path (e.g. "/org/freedesktop/NetworkManager/Devices/2").
3. Specific object?

Some of the input argument can be unspecified. To make them unspecified in DBus call they should be set to "/" and this will be translated into NULL pointer in the impl_manager_activate_connection() function. Of all combinations of parameters (with respect to being NULL or non-NULL) the following ones are allowed:

1. When connection path is not specified device must be given. In that case all the connections for that device will be retrieved and the best one will be selected. "The best one" is defined as the most recently used one.
2. If connection path is specified, then device might or might not be specified. In case it is not defined the best device for the given connection will be selected. To determine "the best device" first list of all devices is retrieved and then for each device status is checked (must be managed, available, compatible with the requested connection). Note that "compatible with the requested connection" means, for example, you can not start wireless connection on a wired connection.

There are "software only", or virtual, connections. Those are checked in the function nm_connection_is_virtual() which is implemented in the file libnm-core/nm-connection.c. When this post was written, the following connections were defined as virtual, or software only:

• Bond
• Team
• Bridge
• VLAN
• TUN
• IPtunnel
• MACVLAN
• VXLAN

Finally, there are also VPN connections that also don't have associated devices.

When all the checks are performed, devices and connections are found, then an object of type NMActiveConnection is created in the function _new_active_connection(). Here, in case VPN connection is started, VPN establishment is initiated and you can read more about that process in another post.

Few tips about GObject for C++ programmers

While studying NetworkManager code I got more and more comfortable with OO programming retrofitted into C programming language using GObject library. At first I was confused because it is quite complex and I didn't well understand how it works. Furthermore, the documentation for beginners is lacking and scare. But, the more I learned the more logical it seemed to me. Since I'm well acquainted with C programming language and to some extent to C++ I decided to write about OO model in GObject, NetworkManager and C based on what someone might expect from C++. As usual, this is for my later reference, but also for everyone else wanting to learn how to use or understand GObject. I'll write in a form of tips, or details you should know in order to better understand how it works.

Declaring a class

In C++ language there are two parts of class implementation. First, there is a class declaration and then there is a class definition. Class declaration goes into a header file (e.g. Point.h) and there you'll find something like this:

class Point {
}

Now, the question is how to do this in C using GObject? The process is a bit more involved due to the characteristics of a C programming language. Anyway, the recipe is the following one. In the header file (e.g. point.h) that declares class method you would put the following code:

#include <glib-object.h>

#define TYPE_POINT (point_get_type())

typedef struct _PointClass {
    GObjectClass parent_class;
} PointClass;

typedef struct _Point {
    GObject parent_instance;
} Point;

GType point_get_type(void);

This code declares two structures, one that will be used by a class (PointClass) and the another one for each object of the given class type (Point). You would also need to define class and object initialization functions as follows:

#include "point.h"

typedef struct _PointPrivate PointPrivate;

G_DEFINE_TYPE (Point, point, G_TYPE_OBJECT);

static void point_class_init (PointClass* klass)
{
}

static void point_init (Point* self)
{
}

Finally, you can write a main function (in e.g. main.c file) which doesn't do anything, but, nevertheless the class is here. That's how you would declare a class.

The code for this example can be found on GitHub.

Final and derivable classes

The distinction between the two is that you can not subclass final class, while on the other hand, derivable classes can be sub classed. I don't want to go into discussions why final classes, but only into technical details related to their use in GObject. GObject documentation recommends to use final classes if you can, and only then derivable classes. If you take a look at some other sources, especially for C++, you'll find doubts about benefits of final classes.

So, in C++ (at least from C++11) you would declare final class like this:

class Point final {
};

Now, if you try to subclass it:

class Point2 : public Point {
};

You'll receive compiler error due to the final keyword prohibiting class Point to be subclassed:

g++ -Wall -std=c++11 main.cpp -o main
In file included from main.cpp:1:0:
Point.h:6:7: error: cannot derive from ‘final’ base ‘Point’ in derived type ‘Point2’
 class Point2 : public Point {
       ^

Note that nothing special has to be done in order for a class to be derivable. Only if you want a class to be final you have to explicitly say so. So, what about GObject and final and derivable classes?

Two macros are used in GObject to create classes for the purpose of creating final and derivable classes. To define final class use G_DECLARE_FINAL_TYPE macro. You should modify header file from the previous example like this:

#include <glib-object.h>

#define TYPE_POINT (point_get_type())
G_DECLARE_FINAL_TYPE(Point, point, , POINT, GObject)

typedef struct _Point {
    GObject parent_instance;
} Point;

Note that there is no definition of type PointClass! The reason is that the macro G_DECLARE_FINAL_TYPE declares it. Also, point_get_type() isn't declared because G_DECLARE_FINAL_TYPE declares it. That's basically it for final class.

To declare derivable class, macro G_DECLARE_DERIVABLE_TYPE is used. In this case, as in the previous, you should only change header file which should look like this:

#include <glib-object.h>

#define TYPE_POINT (point_get_type())
G_DECLARE_DERIVABLE_TYPE(Point, point, , POINT, GObject)

typedef struct _PointClass {
    GObjectClass parent_instance;
} PointClass;

This time, PointClass type is defined explicitly while Point is defined by G_DECLARE_DERIVABLE_TYPE macro.

The code for this example can be found on GitHub.

Instantiate the object of a given class

The next thing is how would you instantiate an object of some class. For sample, to instantiate an object of a class written in C++ from the previous section in the main function you would do the following:

Point* point = new Point();

To do the same thing in C/GObject combo, you again have to do a bit more work. Actually, you have to define allocator for a class, i.e. something equivalent to a new keyword in C++. The way to create a new object of a given type is:

Point *point = someclass_new();

So, you need to define the function someclass_new(). Our will be very simple one:

Point* point_new(void)
{
        return g_object_new(TYPE_POINT, NULL);
}

Place it at the end of the file point.c and in the main function create an object of the Point class by calling the function point_new(). As a final note, instantiating final or derivable class is the same.

The code for this example can be found on GitHub.

Instantiate a singleton object

One feature used by NetworkManager a lot are singleton objects. Namely, some objects control system wide resources and thus there is a need to have a single object control a single resource. For example, main component of a NetworkManager is the object NM_TYPE_MANAGER and it is necessary to have only one such object. There is also a single object for a configuration held in the object/class NM_TYPE_CONFIG.

So, how is singleton object created? NetworkManager has some infrastructure that makes this task really simple. You start with a regular class/object. Then in the C file with an object implementation you should call the following two macros/functions:

NM_DEFINE_SINGLETON_INSTANCE (NMNetnsController);
NM_DEFINE_SINGLETON_REGISTER (NMNetnsController);

Then, in function that allocates an object, you have a variable singleton_instance that should be NULL until the object is created and then it should contain a pointer to a singleton object. Additionally, after creating object, you should call function nm_singleton_instance_register().

You are tasked with taking care that no new object of the given type is created, i.e. you can shoot yourself in the foot in case you don't take care to check the content of the variable singleton_instance. singleton_instance is a file global variable declared by the macros shown above.

Private attributes

The next step is to add private attributes to our class. Let's suppose that we need to add x and y coordinates. In C++ we would modify the class declaration in the following way:

class Point {
private:
        int x, y;
}

And that's basically it for C++ version. In case of C/GObject, you would define struct for private data in C file (point.c) with the following content:

typedef struct _PointPrivate PointPrivate;

struct _PointPrivate {
        int x, y;
};

First, observe that the structure is placed in C file, not in the header file. The reason is that this is private/internal structure so no users of a class should know the content of the given structure. Furthermore, keep in mode that this structure isn't required to be called as shown in the example by the GObject system, you can call it whatever you want, but it is a good practice and strongly suggested to add Private suffix to the object name for the readability reasons.

Private part for the object should be allocated when the class is initialized. To achieve that add the following line in the class initialization function, i.e. in point_class_init() function:

g_type_class_add_private (klass, sizeof (PointPrivate));

Before finishing with private attributes, there is one more thing we need to discuss, and that is how to access private part of the object. For that purpose it is good to declare the following macro:

#define POINT_GET_PRIVATE(object)      \
          (G_TYPE_INSTANCE_GET_PRIVATE((object), TYPE_POINT, PointPrivate))

This macro takes the pointer to an object and returns pointer to the private data of a given object. Note that the last argument to G_TYPE_INSTANCE_GET_PRIVATE is type (structure definition) of a private data. So, when you have a method that accesses the object's private data you would, at the start of the method, call the mentioned macro to obtain pointer to private data structure and then you would access it as usual.

The code for this example can be found on GitHub.

Public methods

After we added private attributes we want to add public methods that will allow us to get and set the values for the given private attributes. So, in C++ we would modify Point class definition as follows:

class Point {
private:
        int x, y;
public:
        void setValue(int x, int y);
        int getx(void);
        int gety(void);
};

The public methods should be defined in a C++ file (i.e. Point.cpp):

#include "Point.h"

void Point::setValue(int x, int y)
{
        this->x = x;
        this->y = y;
}

int Point::getx(void)
{
        return this->x;
}

int Point::gety(void)
{
        return this->y;
}

And, obviously, the methods would be used in a following way in the main function:

point->setValue(1, 1);
std::cout << "x=" << point->getx() << ", y=" << point->gety() << std::endl;

Now, to achieve the same with GObject system several changes to the existing C code are necessary. First, we'll define a helper macro in a C file (point.c) that we will use to obtain private data of an object of class Point:

#define POINT_GET_PRIVATE(object)      \
          (G_TYPE_INSTANCE_GET_PRIVATE((object), TYPE_POINT, PointPrivate))

Next, public methods are just global functions with a prefix of a object name and with the first argument being the pointer to an object of a given class. In our case, we have three public methods each with the following prototype:

void point_set_value(Point *point, int x, int y);
int point_get_x(Point *point);
int point_get_y(Point *point);

Those declaration should go into the header file (point.h) because they are accessible to any user of object Point, while their definitions should go into the C file (point.c):

void point_set_value(Point* point, int x, int y)
{
        PointPrivate* priv = POINT_GET_PRIVATE(point);

        priv->x = x;
        priv->y = y;
}

int point_get_x(Point* point)
{
        PointPrivate* priv = POINT_GET_PRIVATE(point);

        return priv->x;
}

int point_get_y(Point* point)
{
        PointPrivate* priv = POINT_GET_PRIVATE(point);

        return priv->y;
}

Finally, you should use those public methods after object of class Point is allocated in the main function. Here is the example:

point_set_value(point, 1, 1);
printf("x=%d, y=%d\n", point_get_x(point), point_get_y(point));

And that's it. The code for this example can be found in GitHub repository.

Virtual methods

The methods implemented using GObject in the previous section are non-virtual public methods. They are easy to implement, but the least flexible. So, to implement public virtual methods in GObject we need to:

1. In class structure (PointClass) add function pointer for each virtual method we need.
2. In header file declare dispatcher methods that will be called by the user of the class.
3. Define dispatcher methods in the source file.
4. Define implementation of virtual methods in the source file.
5. Initialize function pointers with the implementation of virtual methods.

Here are the steps in more details with the concrete example of Point class. Before continuing, note that virtual methods are possible only in derivable classes, not in final!

Step 1 is to add function pointer fields in the class structure. That means that you class structure definition has to have the following format:

typedef struct _PointClass
{
        GObjectClass parent_class;
        void (*set_value) (Point *self, int x, int y);
        int (*get_x) (Point *self);
        int (*get_y) (Point *self);
} PointClass;

Note the bold parts, those are function pointer that will point to implementation of virtual methods. Each method takes, as the first argument, pointer to the object it has to act upon.

Step 2 is to declare dispatcher methods. Those are very similar to non-virtual public methods in appearance, i.e. the name consists of a object name prefix and the function name. Again, this is recommended, not something that is mandated by GObject system. Anyway, in our case those are the lines you have to add to header file:

void point_set_value(Point* self, int x, int y);
int point_get_x(Point* self);
int point_get_y(Point* self);

Step 3 is to define dispatcher methods in the source file. I'll do it only for one method, the rest are the same. So, in case of point_set_value() it would look like this:

void point_set_value(Point* self, int x, int y)
{
PointClass *klass;
klass = _POINT_GET_CLASS(self);
klass->set_value(self, x, y);
}

As you can see, what the method does is it access class definition, and then invokes appropriate function through its pointer. I have to stress here two things:

1. I skipped assertions, i.e. if the object/classes are of right type!
2. Note the leading underscore in _POINT_GET_CLASS. It is the consequence of skipping module name when calling G_DECLARE_DERIVABLE_TYPE macro, i.e. I skipped third argument!

So, any user of a class will call those public methods which will dispatch the call to the real method.

In step 4 we have to define implementation of virtual methods in the source file. Again, I'll show it only for a single method. Also, you'll note that the implementation is the same as for non-virtual public methods. Only we have to call them differently so that compiler can differentiate between different function. I decided to use prefix objectname_private_, but it is my choice. I didn't manage to find something recommended by GObject. So, here it goes for the function to set value:

void point_private_set_value(Point* self, int x, int y)
{
PointPrivate *priv = POINT_GET_PRIVATE(self);
priv->x = x;
priv->y = y;
}

Finally, step 5 is to initialize pointers to dispatchers which in turn will call real methods. This has to be done by modifying definition of PointClass structure (or class structure in general). Here is how this structure should look like:

static void
point_class_init (PointClass* klass)
{
g_type_class_add_private (klass, sizeof (PointPrivate));
klass->set_value = point_private_set_value;
klass->get_y = point_private_get_y;
klass->get_x = point_private_get_x;
}

Note that we have the part that adds private data for the objects. The bold parts are the ones that initialize virtual functions.

Anyway, that's it for virtual function. You can compile the code on GitHub.

Inheritence and subclassing

Inheritance allows for specialization of some class without touching that class internals. It is one of very important OO mechanisms and as such it is also supported in GObject and used by, e.g. NetworkManager.

So, suppose we want to define 3D point by basing its implementation on Point class while in the same time adding only z coordinate. In C++ we would declare new class and say explicitly that we inherit base class Point:

#include "Point.h"

class Point3D: public Point
{
private:
        int z;
public:
        void setValue(int x, int y, int z);
        int getz(void);
}

As you can see we include declaration of 2D point and then we add third coordinate as well as new methods, one to retrieve the third coordinate and another one to be able to set all three coordinates. There is also definition (i.e. implementation) of a new class which is placed in Point3D.cpp file:

#include "Point3D.h"

void Point3D::setValue(int x, int y, int z)
{
        Point::setValue(x, y);
        this->z = z;
}

int Point3D::getz(void)
{
        return this->z;
}

Note how the base class method is called, by prefixing the method name with the base class name.

As usual, the question is how this is done in C using GObject library. First, GObject make distinction between final and derivable classes, as we already saw above. Base class must be derivable, while subclass might be either final or derivable.

So, we are going to define derivable class Point as follows:

#include <glib-object.h>

#define TYPE_POINT (point_get_type())
G_DECLARE_DERIVABLE_TYPE(Point, point, , POINT, GObject)

typedef struct _PointClass {
        GObjectClass parent_class;
} PointClass;

Point* point_new(void);

void point_set_value(Point *point, int x, int y);
int point_get_x(Point *point);
int point_get_y(Point *point);

This is almost the same as original Point declaration we had. The implementation part isn't changed. Now, Point3D is declared as follows:

#include <glib-object.h>
#include "point.h"

#define TYPE_POINT3D (point3d_get_type())
G_DECLARE_FINAL_TYPE(Point3D, point3d, , POINT3d, Point)

typedef struct _Point3D {
        Point parent_instance;
} Point3D;

Point3D* point3d_new(void);

void point3d_set_value(Point3D* point, int x, int y, int z);
int point3d_get_z(Point3D* point);

Note the last parameter in G_DECLARE_FINAL_TYPE is Point. Basically, that's the base class and up until now we had there GObject from which all object descend.

Defining and implementing interfaces

Interfaces in OO languages are used as a form of a contract between different components. Some languages, as Java for example, have direct support for defining interfaces, while others, like C++, don't have and instead use some indirect methods. Here I would like to show how interfaces are implemented in GObject and for that purpose I'll declare interface that will be implemented by Point class used so far for examples. We'll start with C++, as usual.

To define interface in C++ abstract classes are used. Abstract class can not be instantiated because it has at least one pure virtual method that needs to be implemented by subclass. So, here is how interface for Point class would look like in C++:

class PointInterface {
public:
virtual void setValue(int x, int y) = 0;
virtual int getx(void) = 0;
virtual int gety(void) = 0;
};

As you can see, the pure virtual method is declared by having "equal zero" addition. Now, just one more change is necessary and that is to declare that Point class implements this interface/abstract class. To do that only a single change is necessary, i.e.:

class Point : public PointInterface { ... }

In other words, Point inherits PointInterface.

So, how to do this in GObject? Here is the official documentation and we are going to do this for our existing Point class.

TBC...

Constructor

In the previous examples we already saw constructors, class constructor (*_class_init) and object constructor (*_init). Class constructor is called only once, when a first object of a given class is instantiated. Object constructor, on the other hand, is called every time an object of a given class is instantiated. You can easily see this modifying the previous example so that you place printf() statements into appropriate constructors (point_class_init() for a class constructor and point_init() for an object constructor). Then, by creating two objects of a Point class you'll clearly see that the class constructor is called only once, while the object constructor is called as many times as you instantiated objects.

Destructor

Destructor isn't necessary

Object/class initialization and removal

When creating a new object using g_object_new() function there is a certain sequence of steps executed. For an object that is first of its class, the sequence is:

1. _class_init()
2. constructor()
3. constructed()
4. _init()

Of those four, the first and fourth, are mandatory and have to be defined in the source. The second and third steps are optional and you define them in class constructor (_class_init()) by assigning appropriate pointers to the class interface.

Note that g_object_new() accepts an object type that should be created and a sequence of attribute-value pairs terminated with a single NULL. Those attribute-value pairs are used to set properties of an object (installed using g_object_class_install_property() function) but only after object constructor finishes. Also, the additional requirement is to define set_property class method that will be called to set each property.

For objects of next type only object constructor is called and then properties are initialized.

When object is removed then its destructor is called, if it exists. If this is the last object of its class, then finalize method is called, again, if it exists.

Name spaces

TBD

Literature

If you try to Google for GObject examples or tutorials you'll find some materials from Gnome Web site. But, the truth is that those tutorials leave some open questions and it is really hard to find something else. In the end I managed to find some references that I think are very valuable so here they are:

• Introduction to GObject
• Learning GObject basics
• Avoid G_TYPE_INSTANCE_GET_PRIVATE() in GObjects