End2end design principle, middleboxes and a bit about TCP...

I was just watching guest lecture given by Jana Iyengar to the students of CS144 course on Stanford. In his lecture he talks about e2e design principle, and the rise of middleboxes. He then goes on to conclude that middleboxes (especially NATs) are a problem of today's Internet. And I couldn't agree more! It's known fact that Internet was built in such a way that the network is dumb, while end nodes are smart. When I say smart, it means that functionality is placed within smart part of the whole system, while when I say dumb, it means it only performs one simple function. In the case of the Internet, network only moves data from one end to the other, and almost nothing else. This design principle was a key feature that allowed Internet to evolve to today's form and become ubiquitous network. To better illustrate this point, contrast Internet with telephone network. Telephone network was built so that the network itself is smart, while the end nodes (telephones!) are dumb. There is a nice illustration of this difference I saw once I liked very much. Here is the reconstruction of the illustration I saw:

Now, when you wanted something new from your telephone, which included telephone network, you had to wait your telephone company to introduce the service in the network, and only after that you could use it. Contrast that to the Internet, you just had to install a server/service on you computer and whoever wanted to access it had to install client on its machine. And that's it, no changes necessary in the network. Actually, the network doesn't know, neither care, what you are doing and everything works. There is a great example of this: the Web. The Web wouldn't succeed if Tim Berners-Lee had to wait for telephone companies to do something. And since the Internet is popular thanks to the Web, the Internet itself wouldn't succeed. Note that the telephone network functions in such a way because of a historical reasons. But, the telephone provides don't have incentive to change that. When/if they control network, they have revenues. The moment they only transfer data, the revenues are with someone else. And that's the case today with content providers (Google, Yahoo, Microsoft, Youtube, ...) and ISPs.

There is also one additional reason to design network by pushing  functionality to the edge and that's for scalability reasons. I think that it's quite obvious that the simpler something is, the bigger it can be, and it's easier to increase size, so I won't argue this any more.

What is happening now, and for some time, is that NATs are proliferating throughout the network. And since NATs heavily inspect the packets that pass through them and they depend on knowing higher layer protocols, it means they have to have built-in knowledge of higher layer protocols. What that means in turn is that if you are introducing a new service, you have to have support built into NATs. And there are two problems there. First, abundance of installed NATs that can not be changed, and second, bugs within those devices. So, in essence, we are approaching the way telephone networks work. Of course, there are other problems with NATs, but this one is a huge one!

Jana Iyengar then talks about SCTP, and the fact that this protocol exists from 2000 and it still didn't manage to take some ground. And middleboxes, more specifically NATs, are to blame. They pass TCP, fiddling with it, but nothing else. So, one of the things he was doing is using TCP as a communication substrate. In other words, he relied on TCP being passed through middleboxes, and then he went to built protocol on top of it. This protocol then could be used to build another protocols that will work across middleboxes. The modification that they did to TCP allowed it to deliver out-of-order data which they termed as uTCP, unreliable TCP. And, it seems no-one thought of that before.

But, I have to say that in 2011. I worked with a student and we were trying to introduce certain QoS parameters into TCP. The motivation were streaming services. As a part of this we allowed TCP to be unreliable, i.e. it could drop data in order to meet other QoS parameters. The work is described in diploma thesis available online, unfortunately, only in Croatian. But, I intend now to rewrite it into a paper as it was an interesting experiment...