Friday, November 30, 2012

Internet Freedom - Well done EU!

If you think that Internet brought revolution only to individuals (and maybe different businesses that market themselves over the Internet)  than you miss one important link, telecommunication companies. Before the Internet they were in charge of everything related to communication and they did whatever they wanted, in the supposed name of the customers. If they thought that something isn't good, then no matter what people wanted, they weren't getting it. And we shell not forget pricing, which generated huge revenues. But, after the tremendous success of the Internet, things drastically changed. For some of the underlying reasons you can read in my other post, but the key is that the control was given to users, not network (i.e. telecoms). Now, telecoms are what they should be: data carriers only.

All good, but the problem is that there are no huge profits in data transfer, at least not as it used to be and telecoms don't just sit and wait. And so, every now and then we hear of some brilliant idea coming from telecommunication industry by which they either try to bring back good old days, or they try to offer something that doesn't make sense. Just in case you didn't know, ATM was one such idea that, fortunately  was a big failure! Even more interesting is a comment on this blog post from a guy (or guys) that are trying to reimplement some protocols from mobile telephony. They criticize specifications produced by telecoms (and related industry) for introducing new things, not because they are necessary, but because they are patented and in that way allow manipulation!

But, these days there is one other "very interesting" idea. Probably not many people know that ITU is trying to introduce mechanisms in order to regulate the Internet. Fortunately, EU isn't approving that, along with US. I approve that wholeheartedly  and I can not describe how outraged I am when I think about telecoms and ITU!

But, it is probably enough to point who is proposing regulation and to be clear what real motives are. Also interesting are requirements by some countries that Google and other Internet providers would have to pay to them to be allowed to distribute content to their citizens. This is absurd, because who forces users to access Google?

And ITU is also something I really dislike, a lot! It is a bureaucratic institution that produces standards for telecommunications. It's a dinosaur of the past. If you, as a single person, want to propose something, or just take part in some activity, you first have to be member of some member state standardization body, which isn't free. Then, you have to be delegated as a representative to ITU, and only then you can take part in some activity. And now we come to the best part, specifications that were produced common purpose were quite pricey. Truth to be told, they are now distributing specifications free of charge, but if it weren't the Internet, we would still have to pay for them. Contrast that to IETF, where membership and participation is open to everyone who wants to participate. Also, all the specifications produced by IETF are available for free to anyone. Now, I'm not claiming that IETF is perfect, but I certainly do claim that IETF is much better than ITU.

And while I'm at ITU/IETF, it happened to me several years ago that I called our Ministry in order to ask for funding to visit IETF. Apparently, this particular Ministry was willing to do that, or so it was written on their Web pages. The only caveat was that it didn't include IETF for a simple reason it isn't so bureaucratic as ITU. To cut the story short, bureaucrat I talked with didn't understand what I was talking about, nor he was interested to find out. And it ended without a grant...

Thursday, November 29, 2012

Few notes about sslstrip tool...

I decided to test sslstrip tool. The idea was that I'll use it to demonstrate to users that they should take a note if there is https when they are accessing some site where they have to type password or some other sensitive data. To create test network I used Windows 7 running within VMware Workstation and using iptables I redirected traffic from virtual machine to local port 80 where I started sslstrip tool. But, no matter what I did, it didn't work. It seems that when VMWare is used iptables redirection  doesn't work as expected. In other words, it seems that netfilter hooks aren't placed within vmware network stack.

I managed to get around that issue by modifying hosts file within Windows. Namely, you should open file C:\Windows\System32\drivers\etc\hosts and add the following line there:
The exact IP address is the one assigned to vmnet8 interface on host operating system. Now start Firefox as usual and type in the URL bar:
Note that I'm explicitely telling Firefox to use http, not https. Anyway, after I did it this way everything worked as expected.

The next "problem" you migh have is that no matter what you do, the site you access automatically switches to https. The reason is HSTS. It is used by server to inform Web browser that it should be accessed only through SSL connections. For this reason sslstrip doesn't work with sites that use HSTS, like Google  and Twitter. But, it doesn't mean that those sites are completely protected. If the client is accessing those sites for the first time or the client never used https to access them, then HSTS can be prevented. The point is that HSTS information is transferred only via https connection. Anyway, to get around this clear history (i.e. go Tools then Clear Recent History... and select to clear everything).

And, for the end, I don't think that it is necessary to enable forwarding in the Linux kernel in order for sslstrip to work, i.e. the following command is unnecessary:
echo 1 > /proc/sys/net/ipv4/ip_forward
Namely, the kernel isn't doing forwarding of IP packets in order for this to work. sslstrip acts as a proxy and thus kernel isn't doing any relaying. But, in case you are diverting only a part of the traffic, e.g. only HTTP, and the kernel is handling the rest, i.e. DNS, then forwarding is necessary in the kernel.

Friday, November 23, 2012

Zimlets for managing posix & samba attributes...

Well, this isn't actually new news, but nevertheless I managed to avoid it for some time now. Namely, Zimbra, with upgrade to 7.2, removed plugins that are used to manage Samba and Posix accounts in its LDAP directory. Now, whenever someone asked what about this Zimlets, the answer was "This was never supported by Zimbra and thus someone from community has to step in." If you Google a bit, you'll easily find it, e.g. here or here. Now, this probably is a perfectly reasonable answer from the Zimbra's standpoint, but I believe that Zimbra should know that this plugin was more frequently used one (let me guess it: because it's useful?) and they have to listen their users.

But whatever is/was with those plugins, I had to have them because one of my setups is such that it contains all account databases in Zimbra. When I disabled Samba and Posix Zimlets, everything worked as usual, apart that I was unable to add new users via Web interface. After I managed to get with it for some time now, the time has come that I had to add another user account and I had to see what I'm going to do with non-working Web interface.

After some googling I discovered that someone managed to fix those two plugins, and also at the time this post was written, there are no news if those plugins work with version 8. So, in short, don't upgrade yet if you are using those plugins. To see what was changed in the plugins to make them work, take a look at this post. In any case, go to the Zimbra's gallery and download Posix and Samba Zimlets. The versions I used are 28.5.12 - v6.1 for both Zimlets. Now, before installing open the archives and in each one you'll find config_template.xml files. Open those files in text editor and fill in the correct values. The most important one is LDAP suffix which is by default set to dc=domain,dc=tld and which you should change to reflect your domain. For example, if your domain is then the suffix will be dc=example,dc=com. After you've made changes, save files and put them back in the archive. If you don't do that you'll receive error reports when logging in admin console, and also there will be no existing samba and possix groups. Not to mention that you'll be unable to create new accounts.

Ok, the last step is to undeploy the old versions - in the case you didn't already, and deploy new ones. After deploying, you should log out and then back in and you should see their options under the Configuration section (in the left pane). If you click on, e.g. Manage Samba Groups, you'll see your existing Samba groups. Similarly has to be with the option Posix Groups. If there are no groups (and you know there should be) than you probably messed with LDAP suffixes I was talking about.

And that's it. For the end, if someone from Zimbra is reading this post, then I have a message for you. Namely, don't answer to so many people that you don't support something because it was always unsupported. I don't think it's relevant. If people are using it, and they find it useful, then you should support it. Or at least devote one engineer day to fix the problem. It isn't so expensive and people will have much better opinion about Zimbra.

Friday, November 16, 2012

End2end design principle, middleboxes and a bit about TCP...

I was just watching guest lecture given by Jana Iyengar to the students of CS144 course on Stanford. In his lecture he talks about e2e design principle, and the rise of middleboxes. He then goes on to conclude that middleboxes (especially NATs) are a problem of today's Internet. And I couldn't agree more! It's known fact that Internet was built in such a way that the network is dumb, while end nodes are smart. When I say smart, it means that functionality is placed within smart part of the whole system, while when I say dumb, it means it only performs one simple function. In the case of the Internet, network only moves data from one end to the other, and almost nothing else. This design principle was a key feature that allowed Internet to evolve to today's form and become ubiquitous network. To better illustrate this point, contrast Internet with telephone network. Telephone network was built so that the network itself is smart, while the end nodes (telephones!) are dumb. There is a nice illustration of this difference I saw once I liked very much. Here is the reconstruction of the illustration I saw:

Now, when you wanted something new from your telephone, which included telephone network, you had to wait your telephone company to introduce the service in the network, and only after that you could use it. Contrast that to the Internet, you just had to install a server/service on you computer and whoever wanted to access it had to install client on its machine. And that's it, no changes necessary in the network. Actually, the network doesn't know, neither care, what you are doing and everything works. There is a great example of this: the Web. The Web wouldn't succeed if Tim Berners-Lee had to wait for telephone companies to do something. And since the Internet is popular thanks to the Web, the Internet itself wouldn't succeed. Note that the telephone network functions in such a way because of a historical reasons. But, the telephone provides don't have incentive to change that. When/if they control network, they have revenues. The moment they only transfer data, the revenues are with someone else. And that's the case today with content providers (Google, Yahoo, Microsoft, Youtube, ...) and ISPs.

There is also one additional reason to design network by pushing  functionality to the edge and that's for scalability reasons. I think that it's quite obvious that the simpler something is, the bigger it can be, and it's easier to increase size, so I won't argue this any more.

What is happening now, and for some time, is that NATs are proliferating throughout the network. And since NATs heavily inspect the packets that pass through them and they depend on knowing higher layer protocols, it means they have to have built-in knowledge of higher layer protocols. What that means in turn is that if you are introducing a new service, you have to have support built into NATs. And there are two problems there. First, abundance of installed NATs that can not be changed, and second, bugs within those devices. So, in essence, we are approaching the way telephone networks work. Of course, there are other problems with NATs, but this one is a huge one!

Jana Iyengar then talks about SCTP, and the fact that this protocol exists from 2000 and it still didn't manage to take some ground. And middleboxes, more specifically NATs, are to blame. They pass TCP, fiddling with it, but nothing else. So, one of the things he was doing is using TCP as a communication substrate. In other words, he relied on TCP being passed through middleboxes, and then he went to built protocol on top of it. This protocol then could be used to build another protocols that will work across middleboxes. The modification that they did to TCP allowed it to deliver out-of-order data which they termed as uTCP, unreliable TCP. And, it seems no-one thought of that before.

But, I have to say that in 2011. I worked with a student and we were trying to introduce certain QoS parameters into TCP. The motivation were streaming services. As a part of this we allowed TCP to be unreliable, i.e. it could drop data in order to meet other QoS parameters. The work is described in diploma thesis available online, unfortunately, only in Croatian. But, I intend now to rewrite it into a paper as it was an interesting experiment...

Tuesday, November 13, 2012

Biseri naših neukih novinara 7...

Dakle, sve je počelo s ovim člankom u Poslovnom. Tu me je već izbacilo iz takta stigmatiziranje akademske zajednice korištenjem izraza tipa "uhljebljene akademske ljenčine". Nakon toga nisam ni htio više čitati članak jer čim netko počne s takvim bombastičnim izjavama ne smatram ga/ju više relevantnim. Međutim, onda sam vidio na Facebook-u da je tu vijest prenio i Večernji. Ali Večernji je to podigao na potpuno novu razinu zbog koje je zaslužio da se taj članak, i njegov autor, spomenu u mojoj rubrici novinarskih bisera!

Naime, u naslovu je izjava da je netko "izumio navigacijsku aplikaciju"?! Kakva je to izjava!?!?! Da li je novinar svjestan značenja riječi izumjeti? Možda i je, ali u tom slučaju vjerojatno je živio u nekakvoj rupi i nije uopće svjestan događaja oko sebe, pa navigacijske aplikacije postoje od kada je GPS-a, a vjerojatno i prije toga. Neću više, samo bi volio da mi netko kaže što njegova aplikacija radi što ne radi već N drugih aplikacija (pri čemu je N vrlo velik broj)...

Ostale postove iz ove "serije" možete pronaći ovdje.

Monday, November 12, 2012

Do you want to connect to IPv6 Internet in a minute or so?

Well, I just learned a very quick way to connect to IPv6 Internet that really works! That is, if you have Fedora 17, but probably for other distributions it is equally easy. Here are two commands to execute that will enable IPv6 network connectivity to you personal computer:
yum -y install gogoc
systemctl start gogoc.service
First command installs package gogoc, while the second one starts it. Next time you'll need only start command. After the start command apparently nothing will happen but in a minute or so you'll have working IPv6 connection. Check it out:
# ping6
PING 56 data bytes
64 bytes from icmp_seq=1 ttl=54 time=54.0 ms
64 bytes from icmp_seq=2 ttl=54 time=55.0 ms
--- ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 54.023/54.551/55.080/0.577 ms
As you can see, Google is reachable on IPv6 addresses. You can also try traceroute6:
# traceroute6
traceroute to (2a00:1450:4016:801::1011), 30 hops max, 80 byte packets
1 2001:5c0:1400:a::722 (2001:5c0:1400:a::722) 40.097 ms 42.346 ms 45.937 ms
2 (2001:4de0:1000:a22::1) 47.548 ms 49.498 ms 51.760 ms
3 (2001:4de0:a::1) 55.613 ms 56.808 ms 60.062 ms
4 (2001:4de0:1000:34::1) 62.570 ms 65.224 ms 66.864 ms
5 (2001:4de0:1000:38::2) 72.339 ms 74.596 ms 77.970 ms
6 (2001:7f8:1::a501:5169:1) 80.598 ms 38.902 ms 39.548 ms
7 2001:4860::1:0:4b3 (2001:4860::1:0:4b3) 41.833 ms 2001:4860::1:0:8 (2001:4860::1:0:8) 46.500 ms 2001:4860::1:0:4b3 (2001:4860::1:0:4b3) 48.142 ms
8 2001:4860::8:0:2db0 (2001:4860::8:0:2db0) 51.250 ms 54.204 ms 57.569 ms
9 2001:4860::8:0:3016 (2001:4860::8:0:3016) 64.727 ms 67.339 ms 69.540 ms
10 2001:4860::1:0:336d (2001:4860::1:0:336d) 80.203 ms 82.302 ms 85.290 ms
11 2001:4860:0:1::537 (2001:4860:0:1::537) 87.769 ms 91.180 ms 92.931 ms
12 2a00:1450:8000:1f::c (2a00:1450:8000:1f::c) 61.213 ms 54.156 ms 55.931 ms
It simply can not be easier that that. Using ip command you can check address you were given:
# ip -6 addr sh
1: lo: mtu 16436
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: wlan0: mtu 1500 qlen 1000
    inet6 fe80::f27b:cbff:fe9f:a33b/64 scope link
       valid_lft forever preferred_lft forever
5: tun: mtu 1280 qlen 500
    inet6 2001:5c0:1400:a::723/128 scope global
       valid_lft forever preferred_lft forever
And also routes:
# ip -6 ro sh
2001:5c0:1400:a::722 via 2001:5c0:1400:a::722 dev tun  metric 0
2001:5c0:1400:a::723 dev tun  proto kernel  metric 256  mtu 1280
2a00:1450:4008:c01::bf via 2a00:1450:4008:c01::bf dev tun  metric 0
2a00:1450:400d:803::1005 via 2a00:1450:400d:803::1005 dev tun  metric 0
2a00:1450:4013:c00::78 via 2a00:1450:4013:c00::78 dev tun  metric 0
2a03:2880:2110:cf01:face:b00c:: via 2a03:2880:2110:cf01:face:b00c:: dev tun  metric 0
2000::/3 dev tun  metric 1
unreachable fe80::/64 dev lo  proto kernel  metric 256  error -101
fe80::/64 dev vmnet1  proto kernel  metric 256
fe80::/64 dev vmnet8  proto kernel  metric 256
fe80::/64 dev wlan0  proto kernel  metric 256
fe80::/64 dev tun  proto kernel  metric 256
default dev tun  metric 1
Probably I don't have to mention that if you open Google in a Web browser you'll be using IPv6. :) In case you don't believe me, try using tcpdump (or wireshark) on tun interface.
You can stop IPv6 network by issuing the following command:
systemctl stop gogoc.service
If you try ping6 and traceroute6 commands after that, you'll receive Network unreachable messages, meaning Google servers can not be reached via their IPv6 address.

Sunday, November 11, 2012

Using IEEEtran latex style for language other than English

I had to write a paper in Croatian and I decided to use IEEEtran latex style. The problem was that it uses, and outputs, English words as a default language. That means, for example, that it is outputting words Abstract, Keywords, etc. which don't fit my local language. I managed to solve that problem fairly easy, but I wasn't able to Google anything about this. So, I decided to write how I did it in case someone else needs this too. There are two things you have to translate. The first one is IEEEtran itself, while the other is bibliographic style.

To translate words that appear in your text after the article is generated you have to insert the following lines somewhere at the beginning of the document, but certainly after you've included IEEEtran style:
\def\IEEEkeywordsname{Ključne riječi}
Note that in this case I'm redefining what latex will output as abstract title and keywords title. Basically, whatever appears in your document in English you can translate by first searching for that words in IEEEtrans.cls and looking what associated keyword name is (e.g. abstractnameIEEEkeywordsname). Finally, you add \def commands similar to those I showed above. Note that you first have to find where IEEEtrans.cls is. On Fedora, if you used package tetex-IEEEtran, then it is in directory /usr/share/texmf/tex/latex/IEEEtran/.

Bibliography style, on the other hand, has to be localized separately and in a special way. First, you have to enter special reference in one of your bibliography databases, or, better yet, you can create a separate database just for that purpose. This bibliography entry has to have the following format:
  CTLname_url_prefix = "[Online]. Raspoloživ:"
In this case I'm changing default output for URL from "[Online]. Available: " to something more Croatian like (though not entirely, there is no equivalent word for Online in Croatian). Then, somewhere within your document you are writing, you have to cite this reference with a special cite command:
Now, if you make dvi/ps/pdf version of your document you'll see that the text really changed (in case you use entry that has url field defined). You can find some details in IEEEtran's bibliography manual. To find out what exactly I have to change, I was searching through IEEEtrans.bst file which is in directory /usr/share/texmf/bibtex/bst/IEEEtran/ (again, if you use Fedora's package).

Thursday, November 8, 2012

Installing certificate for Alfresco...

This post is continuation of the post about installing Alfresco using native Tomcat6 installation (on CentOS6). If you followed steps given in that post, you have running Alfresco installation but Tomcat uses self-signed certificate.

To install your own certificate first obtain it (you can use your own, self managed, CA or you can buy commercial one), then install it on your Tomcat instance. You'll find a lot of information about this in SSL Howto on Tomcat's Web pages, but that page assumes that everything you do, you are doing using keytool.

Here is a quick Howto with an assumption that you have files newcert.pem (containing certificate), newkey.pem (containing private key) and cacert.pem (your CA certificate). By default, tomcat's keystore is in its home (/usr/share/tomcat6) and it is named .keystore. Keystore file is password protected and default password for it is changeit. Note that the period isn't part of password! I suggest that you copy this file to root's home under the name keystore (note no leading dot!) or whatever else you wish so that you can restore old copy in case something goes wrong with the following steps.

The installation is two step process. First, you create keystore containing you certificate, private key and CA's certificate. In second step, you import that information to Tomcat's keystore.

First step is to pack certificate for Alfresco, its private key and CA's certificate into PKCS12 store using openssl tool as follows:
$ openssl pkcs12 -export \
        -in newcert.pem -inkey newkey.pem \
        -out mycert.p12 -name tomcat \
        -CAfile cacert.pem -caname root -chain
Enter Export Password:
Verifying - Enter Export Password:
This command assumes that all necessary files (newcert.pem, newkey.pem and cacert.pem) are in you current directory. Output of the command is also stored into current directory. Note that you are asked for password that will protect all the data. Enter something or later you'll see the following warning:
*****************  WARNING WARNING WARNING  *****************
* The integrity of the information stored in the srckeystore*
* has NOT been verified!  In order to verify its integrity, *
* you must provide the srckeystore password.                *
*****************  WARNING WARNING WARNING  *****************
And then you'll receive the following error:
keytool error: Get Key failed: / by zero
Second step is to import this pkcs12 file to tomcat's keystore using keytool as follows:
$ keytool -importkeystore -srckeystore mycert.p12 \
        -srcstoretype pkcs12 -destkeystore /usr/share/tomcat6/.keystore
Enter destination keystore password:
Enter source keystore password:
Existing entry alias tomcat exists, overwrite? [no]:  yes
Entry for alias tomcat successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or cancelled
Again, input file is in the current directory and you are importing directly into tomcat's keystore. Note that the existing certificate with the alias tomcat will be removed and you are asked to confirm that! The default alias Tomcat searches when it start is called tomcat.

Third step is to change private key's password that has to be the same as for the keystore. Do that using the following command:
keytool -keypasswd -alias tomcat -new <keypassword> -keystore /usr/share/tomcat6/.keystore
You'll be asked for the keystore's password and the password for the key will be set to keypassword.

And that's it. Restart tomcat and check if it is using new certificate.

Tuesday, November 6, 2012

Network troubleshooting...

Yesterday, I was giving a lecture to a third year students of computer science within a course Communication networks. Of course, that is not the only computing module on the Faculty, i.e. there are also modules computer engineering and software engineering, but other lecturers are giving lectures to them. Anyway, the topic of the lecture was Internet's networking layer and, among other things I was talking about autonomus systems, BGP routing protocol, and forwarding process. As a part of ICMP protocol, ping and traceroute commands were mentioned as an important addition to troubleshooting tool. I also mentioned several times to the students that ping and traceroute are the main troubleshooting tools of any network technician. I also told them that after they finish with this course they should be able to do basic troubleshooting and never ever again say something like "Internet isn't working"! Finally, I mentioned that there is a Routeviews project on the Internet that provides (read only) access to BGP routers that can be used to see routes exchanged on certain parts of the Internet.

So, the reason I'm writing this post is that I stumbled on a post Why Google Went Offline Today and a Bit about How the Internet Works which is highly recommended read for them, but also for anyone else learning about networking, with an emphasis on the Internet. They (students) should be able to follow and understand this post now after they learned basic terminology and mechanisms of the Internet layer. To fully understand it, they'll have to wait until we explain how DNS works.


Let me (mis)use this post to say more about Routeviews. Actually, there is also Looking Glass which is very similar, i.e. it allows to peek at certain points on the Internet, but it doesn't offer direct access to BGP routers so, its a bit less interesting, at least to me.

Before I continue with description of Routeviews let me state that Internet is highly irregular network, connecting autonomous systems, with the main irregularity coming from the peering relations between autonomous systems. This peering is largely kept confidential, and besides BGP routing is mainly driven by politics not by technology. What this means is that the real topology if the Internet is not known to anybody. And besides, it's a dynamic and constantly moving target. There are attempts of mapping Internet, but nevertheless they are only approximations.

So, how Internet looks depends on where are you looking from. The routeviews allow one to look on the Internet from different points and this is used for troubleshooting purposes, as well for research purposes. There are also historical data and that's very valuable information.

Anyway, if you go to Routeviews project you'll see a table with a list of DNS names. For each name there is additional information, how to access it (mainly telnet), what type of software/hardware it is running, and where it is. So, you can telnet to one of those routers, and using command show ip bgp determine BGP routing table of that router. Note that Cisco IOS (as well as Zebra/Quagga which are modelled after IOS) offer help command in the form of a question mark. At any point in the command you can type ? and the OS will show you what can you type at that point.

Here is a c/p from a session I made. It's obviously edited to be short and up to the point. First, I logged in to one of the routers:
$ telnet
Connected to
Escape character is '^]'.
                    Oregon Exchange BGP Route Viewer
 route views data is archived on
 This hardware is part of a grant from Cisco Systems.
 Please contact if you have questions or
 comments about this service, its use, or if you might be able to
 contribute your view.
 This router has views of the full routing tables from several ASes.
 The list of ASes is documented under "Current Participants" on
                          ************** is now using AAA for logins.  Login with
 username "rviews".  See

User Access Verification
Username: rviews
What I typed is in bold, and what I received is in ordinary text. Note that I used username rviews, as specified in a greeting message. Here is a bit of using help:
Exec commands:
  <1-99>           Session number to resume
  access-enable    Create a temporary Access-List entry
  access-profile   Apply user-profile to interface
  clear            Reset functions
  connect          Open a terminal connection
  crypto           Encryption related commands.
  disable          Turn off privileged commands
route-views>show ?
  aaa                   Show AAA values
  aal2                  Show commands for AAL2
  adjacency             Adjacent nodes
  alps                  Alps information
  appfw                 Application Firewall information
  aps                   APS information
  arp                   ARP table
  auto                  Show Automation Template
  backup                Backup status
  bfd                   BFD protocol info
  bgp                   BGP information
route-views>show i?
if-mgr  ima           inventory  ip
ipc     iphc-profile  ipv6    
Again, in bold is what I typed, and with three dots I marked output that was cutted not to clutter this post. Finally, here is part of the show bgp command output (note, show ip bgp is equivalent form):
route-views>show bgp
BGP table version is 3000708321, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network     Next Hop         Metric LocPrf Weight Path
r>                        0 19214 12989 2828 i
*          0             0 16150 15169 i
*             0             0 852 15169 i
*              6             0 2914 3356 15169 i
*              0             0 3356 15169 i
*                           0 3277 15169 i
*                            0 3267 15169 i
*                              0 3333 3356 15169 i
*                         0 101 101 15169 i
*                         0 6939 15169 i
*              0             0 4826 15169 i
*              0             0 6079 15169 i
*            1             0 3549 15169 i
Let me give you a short description of what you see here. First line is some general data (table version, and router's ID). Then there are status codes and origin code that are used later. Finally, the BGP table is dumped (and piped through more, use q to quit).

What you see in BGP table is:
  1. Network destination, i.e. some network on the Internet that is reachable from that particular BGP router who's tables you are examining. Note that the multiple lines belong to a single destination network and in that case, network isn't repeated. This is the case for network in the previous output.
  2. Each line in the output is one possible path to reach given network. Each path consists of a next hop (second column), and exact path (last column with numbers). The path that was found to be the best is marked with greater-then (>) in the first column (also, note the legend in the beginning of the output).
  3. Exact path is a sequence of autonomous systems through which destination network is reachable. Note that each path ends with a same number, i.e. same autonomous system number. That's because given network belongs to that autonomous system.
  4. At the end of each path there is letter that informs us from where this route was obtained. In this case i means it came from the peering BGP router in the same autonomous system as the router we are examining.
Finally, to exit use exit command. :) And that's it. I suppose you can play and research for yourself from this point on...

MaxTV nastavak...

Dakle, opet sam došao u doticaj s MaxTV opremom (točnije, prijemnikom) i ovaj puta sam pripremio sve potrebno da se ubacim između prijemnika i ADSL rutera. Zvuči kao da sam povezao kamion opreme sa sobom, a u stvari sam samo ponio laptop s PCMCIA mrežnom karticom tako da je dotični imao dvije mrežne kartice. Taman ono što treba. Pri tome sam imao više sreće nego pameti, naime nisam ponio cross-over mrežni kabl ali je ispalo da su mrežne na laptopu (ili na nekom od uređaja) to prepoznale i promijenile smjer kontakata.

Prikupljanje informacija

Uglavnom, ono što sam napravio je da sam isključio MaxTV prijemnik iz rutera te ga potom spojio na laptop, a laptop (koristeći drugu mrežnu utičnicu) spojio na ruter gdje je prije bio spojen prijemnik. Dodatno, kao root korisnik izvršio sam sljedeće naredbe:
# brctl addbr br0
# brctl addif br0 eth0
# brctl addif br0 eth1
# ip link set br0 up
# /etc/init.d/iptables stop
Te naredbe podrazumijevaju nekoliko stvari. Prvo, instaliran je paket bridge-utils u kojemu se nalazi naredba brctl. Konačno, mrežna sučelja na koja su spojeni prijenik i ADSL ruter zovu se eth0 i eth1. Zadnja naredba isključuje firewall, iako u načelu to ne bi trebalo, al' sigurno je sigurno. Ovo sam radio na CentOS 6 distribuciji i ako je neka druga u pitanju, treba napraviti odgovarajuće modifikacije.

Nakon toga sam isključio MaxTV prijemnik te sam pokrenuo snimanje prometa na laptopu:
# tcpdump -s0 -i br0 -w boot.pcap
i uključio MaxTV prijemnik. Promet je sniman u datoteku boot.pcap te sam ga puštao da snima promet dok god nije s ekrana televizora nestala oznaka Učitavam....

Dodatno, ispostavilo se da je komunikacija koja me zanima bila isprepletena s video zapisom zbog čega je datoteka bila vrlo velika. S obzirom da tako velika datoteka jako usporava rad prilikom njena anliziranja, odlučio sam izeliminirati video promet korištenjem tcpdump naredbe. U mom slučaju radilo se o video i audio podacima koji se šalju na adrese i (History i HTV1 kanali). Dakle, eliminacija je jednostavna:
# tcpdump -r boot.pcap -w boot1.pcap not host and not host
Navedena naredba čita datoteku boot.pcap primjenjuje filter (not host and not host i zapisuje rezultat u datoteku boot1.pcap u kojoj više nema video (i audio) podataka. Nego, jesam li spomenuo da obožavam naredbu tcpdump? :)

Koliko god tcpdump naredba bila dobra, u određenim situacijama wireshark je ipak zakon. I sada dolazimo u baš takvu situaciju, analiza snimljenog mrežnog prometa.

Analiza snimljenog mrežnog prometa

Snimljeni mrežni promet učitao sam u wireshark korištenjem sljedeće naredbe:
wireshark boot1.pcap
Ovdje sada neću pričati o tehnikalijama kako sam nešto točno napravio već ću se prvenstveno zadržati na onome što sam uspio pronaći.

Slijed događaja odmah nakon uključenja samog uređaja je sljedeći:
  1. Dohvat mrežnih podataka korištenjem protokola DHCP. Pri tome DHCP poslužitelj koristi ARP kako bi provjerio da adresa nije korištena, te kako bi nakon dodjele adrese provjerio da ju je uređaj doista i prihvatio. U DHCP podacima nalazi se informacija o DHCP poslužitelju (,, DNS poslužitelju i domeni, NTP poslužiteljima i Zanimljivo da je tu i opcija DOCSIS full security server IP s vrijednošću 31 kraj koje stoji oznaka "[TODO]". Čini se da to nije podatak s poslužitelja već da Wireshark trenutno ne dekodira tu informaciju.
  2. DHCP poslužitelj s IP adresom provjerava korištenjem ICMP echo request/response proba (efektivno, ping naredba) da li je klijent dostupan.
  3. Slijedi sinkronizacija vremena korištenjem NTP protokola.
  4. Ponovo DHCP upit, pri čemu se ovaj puta traži niz privatnih opcija, ali niti jedna nije pružena u odgovoru!
  5. Upit za ime Odgovor je višeodredišna adresa
  6. SRV upit DNS poslužitelju za Ovo je standardizirani upit kojim se traži informacija o usluzi _vqe_channel_cfg putem TCP protokola. Odgovor su poslužitelji (IP adresa i port 8554 te (IP adresa i port 8554.
  7. Pristup na poslužitelj korištenjem protokola RSTP. Rezultat je popis svih raspoloživih kanala u SDP formatu. O njoj ću malo kasnije.
  8. Spajanje na grupu (IP adresa dobijena u 5. koraku). S te adrese stalno se prihvaćaju podaci u SDP obliku. Primjer podataka koji se razmjenjuju.
  9. Upit za ime (IP adresa je i spajanje na port 12697 tog poslužitelja. Koliko sam vidio, dohvaćaju se certifikati. Tvrtka verimatrix izgleda da se bavi metodama naplate TV sadržaja.
  10. Dva puta se spaja na port 12698 poslužitelja i dohvaća nešto nepoznato.
  11. Spajanje na (IP adresa je port 12700 i dohvat nekakvih binarnih podataka.
  12. Dohvat podataka poslanih na adresu Podatke šalje poslužitelj (to je NTP poslužitelj ali očito ima još neku funkciju).
  13. Pokušaj spajanja na adresu i po port 8443, međutim došlo je do isteka vremenskog ograničenja (timeout). Kasnije je došao ICMP odgovor Destination unreachable (Communication administratively prohibited).
  14. Upit za imenom (IP adresa i potom slanje UDP paketa na port 2498. Odgovora na taj paket nije bilo.
  15. Upit za imenima (IP adresa i (IP adresa
  16. Spajanje na port 8085. U ovom slučaju radi se o autentifikacijskom zahtjevu koji je uspješno obavljen. Zahtjev se šalje korištenjem metode GET protokola HTTP. Rezultat se vraća u XML obliku.
  17. Potom ponovno spajanje na port 8085 rezultat je bio "Success".
  18. Potom ponovno spajanje na port 8085. Ovaj puta se traže podaci o pretplatniku koji su dobijeni u XML obliku unutar tijela odgovora.
  19. Potom ponovno spajanje na port 8085. Prijenos nepoznatih podataka.
  20. Spajanje i dohvat podataka s grupa,,, i
  21. Spajanje na port 8085.
  22. Spajanje na grupu
  23. Pet spajanja na port 8085. Prijenos nepoznatih podataka.
Nakon ovoga izgleda da dalje idu samo video i audio podaci između prijemnika i T-Com-a. U nastavku su produbljeni neki dijelovi gornjih koraka

Popis kanala

Popis kanala dohvaća se korištenje protokola RTSP. Datoteka se sastoji od dva dijela, upita i odgovora. Upit čine prve tri linije:

DESCRIBE rtsp:// RTSP/1.0
CSeq: 1
Accept: application/sdp
Nakon upita slijedi prazna linija te odgovor. Odgovor je dalje sačinjen od dva dijela, zaglavlja i tijela. Zaglavlje čini pet linija:
RTSP/1.0 200 OK
CSeq: 1
Date: 2 Nov 2012 22:16:49
Content-Type: application/sdp
Content-Length: 152588
Nakon toga dolazi prazna linija i tijelo. Tijelo se sastoji od opisa programa u SDP obliku. Po jedan blok za svaki program. Ovdje je primjer jednog bloka:
o=- 1350979665943 1350979665943 IN IP4 vcpt2
s=DiscWild HD
i=Channel configuration for DiscWild HD ( Unicast Retransmission Only )
t=0 0
a=group:FID 1 3
m=video 3058 RTP/AVPF 96
i=Original Source Stream
c=IN IP4
a=fmtp:96 rtcp-per-rcvr-bw=13
a=source-filter: incl IN IP4
a=rtpmap:96 MP2T/90000
a=rtcp:3059 IN IP4
a=rtcp-fb:96 nack

m=video 3058 RTP/AVPF 98
i=Re-sourced Stream
c=IN IP4
a=source-filter: incl IN IP4
a=rtpmap:98 MP2T/90000
a=rtcp:3059 IN IP4
m=video 50206 RTP/AVPF 99
i=Unicast Retransmission Stream
c=IN IP4
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=96
a=fmtp:99 rtx-time=3000
Ovdje se može razaznati četiri bloka: globani dio od početka do prve linije m=, i potom po jedan blok od kojih svaki započinje s linijom m=. Globalni blok daje informacije o kojem programu se radi. U ovom slučaju u pitanju je Discovery Wild HD. Od preostala tri bloka, zanimljiv nam je (za sada) samo prvi. Ovi ostali su retransmisija, nagađam u slučaju da se može naknadno gledati. U prethodnom ispisu masnim slovima sam označio što pripada prvom bloku. Iz toga se mogu iščitati sljedeće informacije:
  1. Radio se o RTP/AVPF video podacima koji se šalju na port 3058 (linija m=).
  2. Za primanje tih video podataka koristi se adresa (linija c=).
  3. Format u kojemu se prenosi video i audio je MP2T.
Web aplikacija

Web aplikacija nalazi se na IP adresi, port 8085. Poslužitelj je JBoss/Apache, a pristupa se sljedećim URL-ovima:
  • /broker/fcs/authenticate?
  • /broker/fcs/stbProperties?
  • /broker/fcs/subscriberInfo?
  • /broker/fcs/reminder?
  • /broker/fcs/rentedItems?
  • /broker/fcs/customCallerIDList?
  • /broker/fcs/favorite?
  • /broker/fcs/ContactManagement?
  • /broker/fcs/playList?
Tijekom autentikacije šalju se podaci o modelu prijemnika, programskoj podršci koja se izvršava i slično. Ne vidim nikakve tajne informacije u tom URL-u pa mi se čini da je autentikacija samo prijava pri čemu se MAC adresa koristi kao identifikator (jer se u svim ostalim URL-ovima javlja MAC adresa).

Analiza video i audio podataka

Ovo mi se čini kao put da se otkrije iz kojeg razloga nije moguće dekodirati strane kanale. Naime, iz snimljenog prometa izdvojio sam pojedini kanal i potom ga učitao u Wireshark. Izdvajanje kanala obavlja se po IP adresi. Primjerice, kako bi se izdvojio kanal može se koristiti sljedeća tcpdump naredba:
tcpdump -r boot.pcap -w stream1.pcap host
Na temelju popisa kanala, radi se o History kanalu. U toj naredbi pretpostavljam da je prilikom nastajanja datoteke boot.pcap bio odabran taj program na prijemniku. Nakon što se naredba izvrši, rezultat će biti pohranjen u datoteku stream1.pcap. Kada se ta datoteka učita u Wireshark on će analizirati promet samo do protokola UDP. Kako bi nastavio dalje treba mu reći da se radi o RTP prometu, što se postiže klikom na desnu tipku na nekom paketu, odabirom opcije Decode As... i potom se u listi protokola pronađe RTP. Nakon toga on će ispravno dekodirati i MP2T.

Međutim, za sada nisam uspio otkriti u tim podacima kako je video točno kodiran i u čemu je problem. Ako netko želi pogledati izdvojio sam taj promet u dvije pcap datoteke koje se mogu učitati direktno u Wireshark. Obje su veličine oko 1M. Prva sadrži HTV1, dakle ono što se može bez problema gledati. Druga sadrži History kanal s kojim je problem u dekodiranju. Obje datoteke možete direktno podmetnuti nekom programu za prikaz videa (kao što je primjerice mplayer) i primjetit ćete kako HTV1 prikazuje, History ne. Dodatno, mogu se datoteke učitati u Wireshark te potom ekstrahirati video pa pokušati, iako, rezultati će biti gotovo isti.

Podaci na kanalu

Podaci s ovog kanala se stalno prihvaćaju SDP opisi nekakvih binarnih podataka. Ovo su podaci (sadržaj linije s=):
  • 7710.HE.30.14.0000.00.reno.rel6-myrioi-71
  • 7710.HE.32.14.0000.00.reno.rel6-myrioi-110
  • 8000.HE.30.14.0000.00.reno.rel6-myrioi-71
  • 8000.HE.32.14.0000.00.reno.rel6-myrioi-110
  • DTVLINEUP_v.3.0
  • DTVLINEUP_v.3.1
  • sw_update.SAIPP330HD.SCIATL
  • sw_update.SAIPP330HD.SCIATL.reno.rel6-myrioi-110
  • sw_update.SAIPP330HD.SCIATL.reno.rel6-myrioi-71
  • ui_default
  • ui_reno.rel6-46_tcom
Interesantno da sam pretražujući Internet za ključnom riječju "x-carousel" naišao na ovaj forum. Međutim, radi se o postu na Francuskom jeziku koji baš ne razumijem. :)

Linije koje su istaknute masnim slovima označavaju datoteke koje je moguće dohvatiti korištenjem HTTP protokola. Za dohvat treba koristiti sljedeći URL:
Kao što primjećujete, datotekama treba dodati ektenziju .xml.gz.

Pristup Internetu

Putem MaxTV mreže moguće je pristupiti Internetu ali se za to mora koristiti proxy. Proxy je na adresi, port 3128. Primjerice, putem njega moguće je na Facebook ali uglavnom je pristup ograničen, odnosno, nije moguće posjetiti bilo koje Web stranice. Inače, zanimljivo je da port 3128 po "defaultu" koristi Squid pa se vjerojatno radi o toj poslužiteljskoj aplikaciji.


Dakle, očito je jasno kako još nisam uspio odrediti zašto se ne vide pojedini kanali i trebat će na tome još poraditi. S obzirom da mi audio/video nije baš jača strana trebat će vremena. Međutim, ako netko drugi želi istražiti što je problem u tekstu sam stavio linkove na snimljen promet dva programa, jednog koji se ispravno prikazuje (HTV1) i jednog koji se ne prikazuje ispravno (History Channel).

Za kraj, ovdje je link na raspravu na Bugovom forumu u kojemu se raspravlja o STB-u i njegovoj prenamjeni. Čini se interesantno, ali još nisam stigao detaljno pročitati post.

Thursday, November 1, 2012

What is Routing architecture...

I decided to write a post about Nimrod routing architecture but then I realized that I first have to define, or at least describe, what is meant by routing architecture. I searched a bit for a definition, and to my surprise, I wasn't able to find one. So, I decided to give my own view, and possibly definition. But when I started to write it, it turned out that the substantial part of the post about Nimrod will be about routing architecture. After a bit of thinking, I decided to split the text into two posts and the goal of this new post is to define routing architecture. I'll start by dissecting both words, routing and architecture, separately and then I'll combine them.

One note before proceeding. You have to keep one thing in mind when reading this post. My primarily viewpoint is from data networks, and I don't know much about voice, a.k.a. telephone, networks. So I don't claim, nor do I believe, this will be valid for them. In any case, I would gladly hear comments you might have about my view, or, you own views about this particular topic.


The term routing denotes a process who's purpose is to determine path, or route, through the network between two users, obeying some (possibly more that one) constraints and optimality requirements. This process, in essence, enables any two users to communicate. User might be on any node in the network, as is in the Internet, or only on a certain subset of nodes, as is in telephone network where only telephones can communicate and not telephone with some internal network switch. The example with a telephone isn't actually very good, since we are confined in the network layer (as per OSI RM) and telephone network doesn't have concept of a layer but instead uses interfaces! Anyway, users of routing are entities within a network layer, which in turn allow their users (transport layer) to communicate.

Closely related to routing, and frequently mixed with routing, is forwarding, but forwarding is actually concerned with moving data packets through the network, in contrast to routing that only determines where packets should go. In other words, forwarding is on data path, while routing is in control path. Note that users of routing are performing forwarding! There are two additional differences between forwarding and routing. First, routing, in order to work, needs a certain data about the network, and of course, decisions have to be taken somewhere. On the other hand, forwarding needs only a subset of that data to perform its function. In general this is so, but there are some cases it might not be a true subset. For example, RIP protocol has only data about destination, next hop and distance, and that data is passed to forwarding process. On the other hand OSPF routing process knows the complete topology of the network, and it passes only a subset of this data to forwarding process. The second difference is that routing is a global process, i.e. it needs a coordination throughout the whole network, while forwarding is a local process with no explicit coordination with other nodes.

Note that inseparable part of routing is addressing, i.e. how to identify nodes for the purposes of routing. There is also a concept of a name, but it is not directly related to routing. For an early discussion on distinction between names, addresses and routing take a look at Shoch's note from 1978.

So, strictly speaking we have two different things, routing and forwarding. But those two are inseparable, meaning that when you define a new routing it has an impact on forwarding too. After all, routing process has to give instructions to forwarding process what it has to do and also, when routing process tells forwarding process for nodes it has to do so via addresses, so, they have to agree about address structure and whatever information is embedded in them. In any case, by only finding routes, without being able to deliver data because you don't communicate with forwarding process, you didn't achieved much. So, I'll use the term routing  with two different meanings, routing in a strict sense, and routing in general. Routing in general will include also forwarding process and from now on I'll use this meaning of the word routing, unless specifically told otherwise.

For the end of this section let me just note that the routing process can have different characteristics:
  • centralized vs. distributed,
  • static vs. dynamic


Architecture is nowadays very heavily (mis)used word in computer science and related disciplines. Traditional architecture is a very old discipline and different fields of computer science borrow a lot of ideas from it. Probably the first use of word architecture in computer science was in relationship with computer organization, or more precisely, computer architecture and it appeared during the development of IBM's 7030 computer. Here is a definition, given by Frederic P. Brooks, Jr., of the purpose of computer architecture taken from the document Planning a Computer System - Project Stretch:
Computer architecture, like other architecture, is the art of  determining the needs of  the user of  a structure and then designing to meet those needs as effectively as possible  within economic and technological  constraints. Architecture must include engineering  considerations, so that the design will be economical and feasible; but the emphasis in architecture is upon the needs of  the user, whereas in engineering the emphasis is upon  the needs of  the fabricator.
Later, the term architecture spread in use for different levels of abstraction and different domains. For example, there is CPU architecture, as opposed to computer architecture, and just to mention few from software domain: software architecture, service oriented architecture (SOA), client server architecture, and many many others.

I don't think it's necessary to enumerate all the possible uses of this word and their definitions, since it seems to me that it is clear from the original definition what an architecture is and what its purpose is. So, the term architecture denotes a result of a design process, that takes the requirements from the users and outputs definition a system in terms of components, their behavior and interactions that fulfils the initial requirements. Architecture by itself is stripped from unnecessary technical details and from implementation details. It is also a work of art, but I can state that sentence in a more technical terms: it is a search process of a huge space of different possibilities which can not be automated, by current methods, and thus has to rely on experience (which, I believe, is a cause of intuition) of the architect.

Finally, two things have to be noted. First, there could be multiple views of the architecture (thus of a resulting system definition) each of which can represent architecture from different angle. Secondly, architecture is not absolute. As we move through the layers of abstractions, each layer has its architecture and associate implementation!

Routing architecture

So, finally we came to describing what routing architecture is. Routing architecture is
... a system, that consists of components with specified behavior and mutual interaction, built according to some requirements whose purpose is to find a path to a given destination and to allow communication between two users.
This definition is, of course, valid within the networking context. I'm emphasizing this because there is no single word about networks in the previous definition so if it is taken out of the context, it might sound meaningless. Note that by "to allowing communication" I'm also referring to a forwarding process, that is tightly bound to a routing process, as I already explained.

One of the key requirements of routing architecture, at least when Internet-like networks are in question, is scalability.

For additional discussion about routing architecture see this Nimrod presentation that, in the first part, discusses what is routing architecture. Note that this "presentation" is actually text document with titles and bullets, not exactly what should be called presentation.

Examples of routing architectures

There are a dozen of RFCs that deal with different aspect of routing architectures for the Internet. The oldest one I managed to find is The NSFNET Routing Architecture, defined in RFC1093 that was published in 1989. It got addition in 1991 in a form of RFC1222, Advancing the NSFNET Routing Architecture. Also, in 1991. IETF published RFC1287Towards the Future Internet Architecture. This RFC gives possible directions for the future Internet Architecture, and as such, is much broader than routing architecture we are discussing here. Still, it has a section about future routing architecture. Then, in 1993. ipngwg working group published RFC1550, Next Generation (IPng) White Paper Solicitation, which solicited input on requirements for next generation IP. Noel Chiappa was developing Nimrod at that time and he submitted white paper published as RFC1753IPng Technical Requirements Of the Nimrod Routing and Addressing Architecture. Basically, he published requirements on Nimrod that he believed will be put on whatever solution will be selected. Nimrod routing architecture was described in RFC1992Nimrod routing architecture. It was published in 1995 as a possible replacement for IPv4. I'm going to write about that proposal in a separate post.

Lately, IAB recognized that there are serious problems with current routing on the Internet (which are not solved with IPv6). This is documented in report published as RFC4948Report from the IAB Workshop on Routing and Addressing. To address problems identified in the workshop,  Routing Research Group was established with a goal of evaluating different proposed solutions. This group produced several RFCs related to routing architecture. First, there is RFC5772A Set of Possible Requirements for a Future Routing Architecture. Because, as we already saw, defining architecture requires some requirements, RRG group produced RFC6227Design Goals for Scalable Internet Routing, which sets requirements for routing architecture. The requirements defined in this RFC are also used to evaluate different proposals. There are more than dozen proposals. They are listed and shortly surveyed in RFC6115, Recommendation for a Routing Architecture.

For the end I'll mention RFC1958, Architectural Principles of the Internet, and its update RFC3439Some Internet Architectural Guidelines and Philosophy. While more broader that routing architectures, still apply to them and are very interesting by themselves.

About Me

scientist, consultant, security specialist, networking guy, system administrator, philosopher ;)

Blog Archive