This story is unbelievable example of doing security totally wrong and being totally irresponsible to customers and shareholders but also to one's own country!
What happened is that attackers (supposedly, but very probably, from China) obtained passwords of Nortel's seven top executives and used them to gain access into corporate network. Once in, they installed rootkits that allowed them to monitor everything what happened within the company! After some employees detected that there is a breach, top executives apparently didn't do anything to stop it, asses damages and introduce controls to prevent it. Not only that, but they (according to some comments) were the first ones to blame for a breach as a directly responsible because of their careless behavior.
What is basically even more serious is that Nortel, as well as any other company, has obligation towards its customers to keep them safe! Namely, by compromising Nortel it is highly likely, especially with a breach of such a size, that Nortel's products were compromised and that attackers had access to them. By gaining access to those products attackers certainly gained access to vulnerabilities which allowed them to endanger Nortel's customers too! This is unbelievable, and I have no words to express how I feel about it. It's like being in a Twilight Zone!
Also, shareholders were also victims because top management didn't properly protect company's assets and thus, they indirectly incurred damages to the company!
I believe that there have to be laws regulating such behavior as those are damaging to everyone, as I tried to explain. And without laws, nothing can be done to prosecute those responsible for such behavior!
Monday, February 27, 2012
Sunday, February 26, 2012
Computer units sold...
I stumbled on the following article that has a graphic about different computer units sold per year from 1975 until 2011. I reproduced it here for a completeness:
It's obvious that by "computer units" is meant both personal computers and mobile phones. Anyway, this graphic is very interesting even though many popular units are missing, e.g. Spectrum, Amstrad, Commodore 128, some Atari models. This graph was taken from Asymco.
What this make me wonder is if there is some page with this data in one place, so that curious person can play a bit with that data. Quick googling didn't reveal much. I tried expressions like c64 units sold, or home computer units sold per year. Still, I found some quite interesting pages. I'll list them here, and if it happens that you know where I can find all the data I was looking for, please do tell me! The most interesting, and relevant, page I found is an article on Wikipedia about Console wars in which some data can be found. This page also has a plenty of references, which I didn't pursue further. Also, while googling, this page frequently appeared. The interesting thing in that particular article is how they estimate number of C64 units sold per year using serial numbers! The method used is actually much older, it dates back to WWII when Allies used it to estimate number of tanks German army produced. Serial numbers of C64 computers were obtained from the Web site that collects C64 serial numbers. Using this method and data the article estimates that there were 12.5 million units of C64 produced.
All in all, very interesting stuff, but it seems that it would require much more time than I devoted to it, and that I'm ready to devote to it. :)
It's obvious that by "computer units" is meant both personal computers and mobile phones. Anyway, this graphic is very interesting even though many popular units are missing, e.g. Spectrum, Amstrad, Commodore 128, some Atari models. This graph was taken from Asymco.
What this make me wonder is if there is some page with this data in one place, so that curious person can play a bit with that data. Quick googling didn't reveal much. I tried expressions like c64 units sold, or home computer units sold per year. Still, I found some quite interesting pages. I'll list them here, and if it happens that you know where I can find all the data I was looking for, please do tell me! The most interesting, and relevant, page I found is an article on Wikipedia about Console wars in which some data can be found. This page also has a plenty of references, which I didn't pursue further. Also, while googling, this page frequently appeared. The interesting thing in that particular article is how they estimate number of C64 units sold per year using serial numbers! The method used is actually much older, it dates back to WWII when Allies used it to estimate number of tanks German army produced. Serial numbers of C64 computers were obtained from the Web site that collects C64 serial numbers. Using this method and data the article estimates that there were 12.5 million units of C64 produced.
All in all, very interesting stuff, but it seems that it would require much more time than I devoted to it, and that I'm ready to devote to it. :)
Thursday, February 23, 2012
Otvoreni kod u Hrvatskoj...
Što drugo reći nego: HVALA BOGU! Hvala Bogu da je konačno jedna Vlada u Hrvatskoj odlučila istaknuti otvoreni kod kao jednu od svojih bitnih odrednica. Istina, nije u pitanju velika revolucija, ali je ipak nekakav pomak. Iako je vijest stara već dva mjeseca, tek danas sam saznao za nju, tako da i ja pomalo kasnim. U stvari, saznao sam da postoji i Centar za otvoreni kod pri Nacionalnoj i sveučilišnoj knjižnici koji je zadužen za pripremu projekta uvođenja otvorenog koda u državnu upravu. Uglavnom, politika otvorenog koda u državnoj upravi može se pogledati ovdje.
Već jako dugi niz godina smatram kako se stotine milijuna kuna efektivno "bacaju" plaćajući licence za programsku podršku koja se razvija izvan HR. Prvenstveno je to SAD, zatim EU, a i Indija, kao jedan od glavnih outsourcing destinacija. Dio novaca, istina, ostaje i u HR, ali to je čisto dio koji se bavi prodajom. Slično kao i prodaja automobila, postoji biznis oko automobila u HR, ali taj biznis je samo prodajni, plus serviserski. Identična stvar je i s programskom podrškom. Isto, u oba slučaja, visokotehnološki razvoj je izvan HR! To između ostalog znači da nama ne trebaju visokoobrazovani ljudi, dovoljni su serviseri!
Kao što je činjenica da se novci odlijevaju van, a i da se razvoj obavlja vani, tako je i činjenica da je Hrvatska premala zemlja da se ovdje obavlja veliki razvoj, jednostavno, premalo nas je. Primjerice, izgraditi novu bazu podataka poput MySQL-a ili PostgreSQL-a, ili novi operacijski sustav poput Linux-a ili FreeBSD-a je ogroman posao! Međutim, mi se možemo, a po mom mišljenju i moramo, osloniti na otvoreni kod. Ideja je da se iskoristi rad milijuna ljudi po svijetu koji su napravili već prilično složene programske sustave te da se ti sustavi lokaliziraju i nadograđuju s novim funkcionalnostima. Na taj način bi se razvijala industrija u HR, Hrvatski građani bi imali pristup kvalitetnoj i besplatnoj programskoj podršci te ne bi morali trošiti novce na nju, a Hrvatske programerske tvrtke bi mogle konkurirati u svijetu sa stečenim znanjima i vještinama. Trenutno, HR tvrtke mogu konkurirati jedino sa serviserskim znanjima što bas i nije tako korisno.
Konačno, glavni pokretač cijelog sustava bi trebala biti država! Dakle, država bi morala na natječajima tražiti otvoreni kod, poticati lokalizaciju i razvoj, poticati uvođenje te programske podrške u škole, itd. A onda bi se na taj način s vremenom pokrenula industrija za koju vjerujem da ne bi bila samo prodavačka, već ipak i razvojna, a možda i istraživačka! Dodatno, sav razvoj programske podrške koji plaća država morao bi se besplatno dati građanima na korištenje iz jednostavnog razloga što su novci građana upotrebljeni za razvoj. To je slično kao često spominjana zamjerka da je HT (odnosno današnji T-Com) izgrađen s doprinosima građana, a potom je prodan strancima i sada građani moraju plaćati za usluge T-Coma.
No, što će od svega toga biti, i da li će uopće biti treba pričekati i vidjeti.
Edit: Kako bi bilo dobro da nisam slijep! Linkani PDF s odrednicama za korištenje otvorenog koda u državnoj upravi je iz 2006. godine!?
Već jako dugi niz godina smatram kako se stotine milijuna kuna efektivno "bacaju" plaćajući licence za programsku podršku koja se razvija izvan HR. Prvenstveno je to SAD, zatim EU, a i Indija, kao jedan od glavnih outsourcing destinacija. Dio novaca, istina, ostaje i u HR, ali to je čisto dio koji se bavi prodajom. Slično kao i prodaja automobila, postoji biznis oko automobila u HR, ali taj biznis je samo prodajni, plus serviserski. Identična stvar je i s programskom podrškom. Isto, u oba slučaja, visokotehnološki razvoj je izvan HR! To između ostalog znači da nama ne trebaju visokoobrazovani ljudi, dovoljni su serviseri!
Kao što je činjenica da se novci odlijevaju van, a i da se razvoj obavlja vani, tako je i činjenica da je Hrvatska premala zemlja da se ovdje obavlja veliki razvoj, jednostavno, premalo nas je. Primjerice, izgraditi novu bazu podataka poput MySQL-a ili PostgreSQL-a, ili novi operacijski sustav poput Linux-a ili FreeBSD-a je ogroman posao! Međutim, mi se možemo, a po mom mišljenju i moramo, osloniti na otvoreni kod. Ideja je da se iskoristi rad milijuna ljudi po svijetu koji su napravili već prilično složene programske sustave te da se ti sustavi lokaliziraju i nadograđuju s novim funkcionalnostima. Na taj način bi se razvijala industrija u HR, Hrvatski građani bi imali pristup kvalitetnoj i besplatnoj programskoj podršci te ne bi morali trošiti novce na nju, a Hrvatske programerske tvrtke bi mogle konkurirati u svijetu sa stečenim znanjima i vještinama. Trenutno, HR tvrtke mogu konkurirati jedino sa serviserskim znanjima što bas i nije tako korisno.
Konačno, glavni pokretač cijelog sustava bi trebala biti država! Dakle, država bi morala na natječajima tražiti otvoreni kod, poticati lokalizaciju i razvoj, poticati uvođenje te programske podrške u škole, itd. A onda bi se na taj način s vremenom pokrenula industrija za koju vjerujem da ne bi bila samo prodavačka, već ipak i razvojna, a možda i istraživačka! Dodatno, sav razvoj programske podrške koji plaća država morao bi se besplatno dati građanima na korištenje iz jednostavnog razloga što su novci građana upotrebljeni za razvoj. To je slično kao često spominjana zamjerka da je HT (odnosno današnji T-Com) izgrađen s doprinosima građana, a potom je prodan strancima i sada građani moraju plaćati za usluge T-Coma.
No, što će od svega toga biti, i da li će uopće biti treba pričekati i vidjeti.
Edit: Kako bi bilo dobro da nisam slijep! Linkani PDF s odrednicama za korištenje otvorenog koda u državnoj upravi je iz 2006. godine!?
Problem of encrypted hard disks in court...
Well, I already blogged about the problem (the post is in Croatian) of encrypted disks, that is, problem from the law enforcement (and probably by extension, a government's) point of view. The case that motivated me to write that previous post didn't finish yet. Namely, according to the latest news, the defendant in this case must turn in decrypted hard drive. Note that she doesn't have to turn password! The problem seems to be around the question how the Fifth Amendment that prevents self-incrimination applies to this case. This is a very tricky question. The problem is that EFF also took part in this case protecting the rights of users to encrypt their data. But, it is also true that criminals, terrorists and others can prevent law enforcement in their investigations. We'll see how this case develops.
In the end, I'm very confident that in Croatia (and similar countries) this kind of questions are like science fiction, even though they are important and relevant!
Newest developments:
In the end, I'm very confident that in Croatia (and similar countries) this kind of questions are like science fiction, even though they are important and relevant!
Newest developments:
Tuesday, February 21, 2012
Sigurnosni problemi u susjedstvu...
Vrlo zanimljivo kako naši novinari nisu popratili probleme u susjedstvu s kompromitiranjem kreditnih kartica, tim više što su u cijelu priču uključene i banke koje djeluju i na teritoriju Republike Hrvatske. Ova vijest je već i dosta stara, s obzirom da je od objave prošlo već dva mjeseca, a od samog incidenta vjerojatno i više.
Jako malo informacija je dostupno što se točno desilo te se sve svodi na nagađanje. Čini se kako je došlo do kompromitiranja nekog procesora (nije objavljeno kojega) te da su ugrožene samo Visa kartice (nekih 20-tak tisuća). Da li je problema bilo i u HR ne znam, ali moguće je jer se očito radi o međunarodnom incidentu.
Interesantno je kako bi svi procesori morali biti usklađeni s PCI DSS preporukama, te da bi to trebalo umanjiti mogućnost uspješnog napada. No, u konačnici ovo pokazuje kako ipak problema ima te da smo daleko od pravog rješenja, ako ga uopće ikada i ostvariom.
Jako malo informacija je dostupno što se točno desilo te se sve svodi na nagađanje. Čini se kako je došlo do kompromitiranja nekog procesora (nije objavljeno kojega) te da su ugrožene samo Visa kartice (nekih 20-tak tisuća). Da li je problema bilo i u HR ne znam, ali moguće je jer se očito radi o međunarodnom incidentu.
Interesantno je kako bi svi procesori morali biti usklađeni s PCI DSS preporukama, te da bi to trebalo umanjiti mogućnost uspješnog napada. No, u konačnici ovo pokazuje kako ipak problema ima te da smo daleko od pravog rješenja, ako ga uopće ikada i ostvariom.
Biseri naših neukih novinara 3...
Dakle, upravo sam naletio na ovaj naslov: Predsjednik Josipović zove Anonymouse na okrugli stol o ACTA-i, i potom se u članaku više niti jednom riječju ne spominje Anonymous!? Nevjerojatno, pravi primjer kako složiti naslov tako da bude što je moguće bombastični, bez obzira što to nema nikakve veze sa stvarnošću, i što se efektivno zavaravaju ljudi.
Ali ne samo to, već je i sam naslov glupost, treba samo pokušati odgovoriti na sljedeća pitanja kako bi se to uvidjelo:
Ostale postove iz ove "serije" možete pronaći ovdje.
Ali ne samo to, već je i sam naslov glupost, treba samo pokušati odgovoriti na sljedeća pitanja kako bi se to uvidjelo:
- Tko je Anonymous da ga se može zvati na bilo kakve sastanke?
- Ako i pretpostavimo da je Anonymous nekava organizacija, tko će ju predstavljati!?
- Od kada se kriminalci zovu na bilo kakve dogovore/pregovore?
Ostale postove iz ove "serije" možete pronaći ovdje.
Thursday, February 16, 2012
Spoofed mail messages...
I was reading about email hack attacks advisory by FINRA and it shows what I'm constantly stress in my education talks about email service: It is very insecure service in general and it shouldn't be used for anything serious. What is not clear is why it is necessary to hack someone's mail account in order to spoof a message. I suppose that the reason is some secret code (i.e. password) that is necessary to provide within mail message in order to prove message authenticity? In any case, email is insecure and that's it. Yeah, I know, there are S/MIME and PGP, but they are still not used much!
But, there is always but, and it is the case when the correspondence is internal to an organization. That case can be treated as a more trustworthy for a simple reason that the messages are passing through mail server, and in general through network infrastructure, under the local control (e.g. some company). Of course, this assumes that the network of that company is sufficiently secure, or that a threat of unauthorized access to a network is sufficiently small.
Now I here you saying that there is a problem: someone from the outside can falsify a message so that it looks like it originates from some internal users. Basically, this is not a problem for users that know where to look (i.e. headers), but for the majority, and those that don't look offten headers, this is indeed a problem. Still, I think, there are two possible solutions. The first one is for mail server (actually spam filter) to reject messages claiming to originate from some internal user and coming from the outside. The second one is via some plugin for mail client (which doesn't exist yet as far as I know) that would analyze From and Received attributes in a message header and in case they match (mail originating IP address is internal and From field is internal mail address) it would show a message in a green color, otherwise in a red color, signalizing to a user that something is strange with this message.
For the end of the post, here is a link to educational games I found during search for a original advisory from FINRA. Namely, the article I first encountered about email attacks didn't provide its source, so I googled for it and in due course I found those games which I think are interesting!
But, there is always but, and it is the case when the correspondence is internal to an organization. That case can be treated as a more trustworthy for a simple reason that the messages are passing through mail server, and in general through network infrastructure, under the local control (e.g. some company). Of course, this assumes that the network of that company is sufficiently secure, or that a threat of unauthorized access to a network is sufficiently small.
Now I here you saying that there is a problem: someone from the outside can falsify a message so that it looks like it originates from some internal users. Basically, this is not a problem for users that know where to look (i.e. headers), but for the majority, and those that don't look offten headers, this is indeed a problem. Still, I think, there are two possible solutions. The first one is for mail server (actually spam filter) to reject messages claiming to originate from some internal user and coming from the outside. The second one is via some plugin for mail client (which doesn't exist yet as far as I know) that would analyze From and Received attributes in a message header and in case they match (mail originating IP address is internal and From field is internal mail address) it would show a message in a green color, otherwise in a red color, signalizing to a user that something is strange with this message.
For the end of the post, here is a link to educational games I found during search for a original advisory from FINRA. Namely, the article I first encountered about email attacks didn't provide its source, so I googled for it and in due course I found those games which I think are interesting!
Moderni Robin Hood, a.k.a. Anonymous...
Čitam malo komentare ljudi na vijest kako je Anonymous napao stranice Ministarstva vanjskih poslova i primjećujem kako ljudi podržavaju i navijaju za Anonymous i slične grupe. Svaka čast, ima iznimki, ali izgleda da su to samo iznimke. Mislim da to nije dobro, i odlučio sam ovdje napisati komentar povodom toga, a ne na forume iz jednostavnog razloga da komentar bude na jednom mjestu.
Očito je u psihi ljudi, pogotovo na brdovitom balkanu, da traže nekakve buntovnike koji se suprotstavljaju i sramote omraženu vlast, koji krše pravila i bore se protiv tih istih pravila. Usudio bi se čak reći da je to i povijesno uvjetovano jer, kada se gleda povijest Hrvatske i okolnih zemalja, uvijek smo bili pod nečijom vlašću.
No, mislim da ljudi nisu svjesni opasnosti koja se krije iza Anonymousa. U stvari, ovdje bi mogao tvrditi da Anonymous podržavaju baš oni koji o računalnom kriminalu nemaju pojma! Kao nekakvu analogiju mogao bi upotrijebiti mafiju i organizirani kriminal. Pretpostavljam da danas nikome normalnome ne pada na pamet podržavati te grupe jer je svima jasno da se radi o vrlo opasnom obliku organiziranja ljudi. Također, svima je jasno da u takvom društvu u kojemu dominira mafija, odnosno, kriminal, nisu ljudima zajamčena temeljna prava i da se može desiti da nekoga mrak proguta, i nitko ne traži što vam se desilo. U stvari, to se dešava i u dikaturama isto, i opet, mislim da nitko normalan neće podržati diktaturu.
Sad će odmah netko reći kako se radi o potpuno različitim stvarima! Anonymous i mafija, odnosno diktatura, nemaju međusobne nikakve veze. Međutim, imaju! I jedno i drugo funkcioniraju prema vlastitim pravilima koja su vođena isključivo interesom za dobiti. Isto tako, ljudi znaju što Anonymous radi na temelju onoga što se objavi u novinama. Ali, da li je bilo tko siguran da oni ne rade više od toga? A ako ima netko siguran, onda me zanima koliko su sigurni da će Anonymous uvijek takav biti? Volio bi vidjeti što bi svatko od tih rekao da odjednom Anonymous njih hakira? Bi li ih i dalje podržavali? Bi li ih podržavali da ukradu brojeve kreditnih kartica skinu novce i daju ih siromašnima?
Ja sam uvjeren da se to nekome desi da bi onda ti koji podržavaju Anonymous promijenili priču! A možda Anonymous to već radi te ovo javno djelovanje koristi kako bi pridobio ljude na svoju stranu i na taj način se osigurao? U stvari, gotovo sigurno to već radi, jer ispisivanjem parola dobijaju se pristaše, a ne koštaju, niti obavezuju, ništa...
Uglavnom, što se mene tiče, Anonymous je isto što i kriminalna organizacija i to je to.
Očito je u psihi ljudi, pogotovo na brdovitom balkanu, da traže nekakve buntovnike koji se suprotstavljaju i sramote omraženu vlast, koji krše pravila i bore se protiv tih istih pravila. Usudio bi se čak reći da je to i povijesno uvjetovano jer, kada se gleda povijest Hrvatske i okolnih zemalja, uvijek smo bili pod nečijom vlašću.
No, mislim da ljudi nisu svjesni opasnosti koja se krije iza Anonymousa. U stvari, ovdje bi mogao tvrditi da Anonymous podržavaju baš oni koji o računalnom kriminalu nemaju pojma! Kao nekakvu analogiju mogao bi upotrijebiti mafiju i organizirani kriminal. Pretpostavljam da danas nikome normalnome ne pada na pamet podržavati te grupe jer je svima jasno da se radi o vrlo opasnom obliku organiziranja ljudi. Također, svima je jasno da u takvom društvu u kojemu dominira mafija, odnosno, kriminal, nisu ljudima zajamčena temeljna prava i da se može desiti da nekoga mrak proguta, i nitko ne traži što vam se desilo. U stvari, to se dešava i u dikaturama isto, i opet, mislim da nitko normalan neće podržati diktaturu.
Sad će odmah netko reći kako se radi o potpuno različitim stvarima! Anonymous i mafija, odnosno diktatura, nemaju međusobne nikakve veze. Međutim, imaju! I jedno i drugo funkcioniraju prema vlastitim pravilima koja su vođena isključivo interesom za dobiti. Isto tako, ljudi znaju što Anonymous radi na temelju onoga što se objavi u novinama. Ali, da li je bilo tko siguran da oni ne rade više od toga? A ako ima netko siguran, onda me zanima koliko su sigurni da će Anonymous uvijek takav biti? Volio bi vidjeti što bi svatko od tih rekao da odjednom Anonymous njih hakira? Bi li ih i dalje podržavali? Bi li ih podržavali da ukradu brojeve kreditnih kartica skinu novce i daju ih siromašnima?
Ja sam uvjeren da se to nekome desi da bi onda ti koji podržavaju Anonymous promijenili priču! A možda Anonymous to već radi te ovo javno djelovanje koristi kako bi pridobio ljude na svoju stranu i na taj način se osigurao? U stvari, gotovo sigurno to već radi, jer ispisivanjem parola dobijaju se pristaše, a ne koštaju, niti obavezuju, ništa...
Uglavnom, što se mene tiče, Anonymous je isto što i kriminalna organizacija i to je to.
Napadi Anonymousa po Hrvatskoj...
Eto, zadnja vijest je da je Anonymous uspješno napao stranice Ministarstva vanjskih poslova, treći javni napad nakon stranica Predsjednika Republike i ZAMP-a.
Dalo bi se filozofirati što je taj "Anonymous" budući da svaki hakerčić može na stranicu staviti izjavu "Ja sam Anonymous." To s jedne strane nije bitno, a s druge je. :) Nebitno je zato što je napad učinjen i očito je bilo propusta te je nebitno tko konkretno je to napravio, postoji propust i to je to! S druge strane, kada ono protiv čega se borite nema organizaciju, onda nemate ni gdje udariti kako bi ste tu organizaciju onesposobili! To je veliki problem u stvari, i zbog toga je bitno kako Anonymous funkcionira!
No, bitno je isto tako kako je razina sigurnosti i svijesti o sigurnosti u Hrvatskoj na izrazito niskim granama. Moram priznati kako je najnapredniji dio sustava bankarski, odnosno, financijski sektor koji kontrolira HNB. HNB sve bolje upravlja tim sustavom sa sigurnosnog stanovišta. Istina, kakva je situacija u obavještajnim službama i policiji ne znam, a moguće je i da velike internacionalne tvrtke u HR isto imaju odgovarajuće mehanizme zaštite. No, većina u Hrvatskoj, što javnih što privatnih poduzeća je u katastrofalnom stanju!
Konačno, to me dovodi i do jedne pozitivne stvari u cijeloj ovoj priči s Anonymousom. Ovo do sada su relativno bezazleni napadi (bar se tako čini za sada!) čija jedina šteta je utjecaj na reputaciju. Iako, moram reći, da reputacija ionako nije nešto prevelika. No, da se vratim na bitnu stvar, naime, nadam se da odgovorni u Hrvatskoj postaju sve svjesniji sigurnosnih problema te da će to potaknuti ulaganje više sredstava u zaštitu, ne samo institucija već i pojedinaca.
Za kraj, zanima me da li MVPEI, odnosno policija, traže počinitelja...
Dalo bi se filozofirati što je taj "Anonymous" budući da svaki hakerčić može na stranicu staviti izjavu "Ja sam Anonymous." To s jedne strane nije bitno, a s druge je. :) Nebitno je zato što je napad učinjen i očito je bilo propusta te je nebitno tko konkretno je to napravio, postoji propust i to je to! S druge strane, kada ono protiv čega se borite nema organizaciju, onda nemate ni gdje udariti kako bi ste tu organizaciju onesposobili! To je veliki problem u stvari, i zbog toga je bitno kako Anonymous funkcionira!
No, bitno je isto tako kako je razina sigurnosti i svijesti o sigurnosti u Hrvatskoj na izrazito niskim granama. Moram priznati kako je najnapredniji dio sustava bankarski, odnosno, financijski sektor koji kontrolira HNB. HNB sve bolje upravlja tim sustavom sa sigurnosnog stanovišta. Istina, kakva je situacija u obavještajnim službama i policiji ne znam, a moguće je i da velike internacionalne tvrtke u HR isto imaju odgovarajuće mehanizme zaštite. No, većina u Hrvatskoj, što javnih što privatnih poduzeća je u katastrofalnom stanju!
Konačno, to me dovodi i do jedne pozitivne stvari u cijeloj ovoj priči s Anonymousom. Ovo do sada su relativno bezazleni napadi (bar se tako čini za sada!) čija jedina šteta je utjecaj na reputaciju. Iako, moram reći, da reputacija ionako nije nešto prevelika. No, da se vratim na bitnu stvar, naime, nadam se da odgovorni u Hrvatskoj postaju sve svjesniji sigurnosnih problema te da će to potaknuti ulaganje više sredstava u zaštitu, ne samo institucija već i pojedinaca.
Za kraj, zanima me da li MVPEI, odnosno policija, traže počinitelja...
Sunday, February 12, 2012
Muljanje s titulama...
Upravo sam primio/pročitao jedan mail i u njemu primjetio kako si osoba koja ga je poslala stavlja titulu dipl. ing. telekomunikacija?! Ni uz najbolji trud ne mogu se sjetiti koji to fakultet, odnosno sveučilište, u Zagrebu, odnosno u Hrvatskoj, daje titulu dipl. ing. telekomunikacija? Znam da postoji smjer telekomunikacije na Fakultetu prometnih znanosti, ali u tom slučaju titula je dipl. ing. prometa. Na FER-u također postoji smjer telekomunikacije, ali u tom slučaju titula je dipl. ing. elektrotehnike. Prema tome, tko daje tu titulu? U ovom slučaju čak mislim da je ta osoba završila FER, pa pretpostavljam da ljepše zvuči kada se potpiše s dipl. ing. telekomunikacija nego dipl. ing. elektrotehnike.
Uglavnom, jedan lijepi primjer kako ljudi pate na titule te kako si vole smisliti titule koje nemaju veze s onim što im piše na diplomi, ali koje njima više odgovaraju!
Uglavnom, jedan lijepi primjer kako ljudi pate na titule te kako si vole smisliti titule koje nemaju veze s onim što im piše na diplomi, ali koje njima više odgovaraju!
Who's listening on an interface...
While I was trying to deduce whether arpwatch honors multiple -i options and listens on multiple interfaces I had a problem of detecting on which interface exactly does it listen? To determine that, I started arpwatch in the following way:
Then, I remembered that there is a ss command that is specific to Linux, and that is used to provide information about sockets! So, I looked into man page and there it says that the option -0 (or -f link) is used to show PACKET sockets, so I tried:
One final thing bothered me. From where does ss take this information? But it's easy to find out that, use strace! :) So, using strace I found out that ss is using /proc/net/packet file:
but that didn't report to me which interface did the command bound to. Using -d option (debug) didn't help either. So, the first attempt was looking into files open by the command. It can be done using lsof command, or by directly looking into aprwatch's proc directory. So, I found out PID (in this particular case that was 23833) of the command (use ps for that) and then I went into directory /proc/PID/fd. In there, I saw the following content:arpwatch -i wlan0 -i em1
# cd /proc/23833/fdThis wasn't so useful! Actually, it tells me that arpwatch closed it's stdin and stdout descriptors and opened only appropriate sockets to get arp frames. lsof command also didn't show anything direct:
# ls -l
total 0
lrwx------. 1 root root 64 Feb 12 12:32 0 -> socket:[27121497]
lrwx------. 1 root root 64 Feb 12 12:32 1 -> socket:[27121498]
# lsof -p 23833 -nBut it did show that there is a raw socket in use, but not anything more than that. So, the next step was to find out how to list all raw sockets? netstat can list all open and listening sockets, so, looking into man page it turns out that the option --raw or -w is used to show raw sockets, i.e.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
arpwatch 23833 root cwd DIR 253,2 4096 821770 /var/lib/arpwatch
arpwatch 23833 root rtd DIR 253,2 4096 2 /
arpwatch 23833 root txt REG 253,2 34144 2672471 /usr/sbin/arpwatch
arpwatch 23833 root mem REG 253,2 168512 2228280 /lib64/ld-2.14.90.so
arpwatch 23833 root mem REG 253,2 2068608 2228301 /lib64/libc-2.14.90.so
arpwatch 23833 root mem REG 253,2 235944 2675058 /usr/lib64/libpcap.so.1.1.1
arpwatch 23833 root mem REG 0,6 27121497 socket:[27121497] (stat: No such file or directory)
arpwatch 23833 root 0u pack 27121497 0t0 ALL type=SOCK_RAW
arpwatch 23833 root 1u unix 0xffff8802e3024ac0 0t0 27121498 socket
# netstat -wWell, neither this was very useful, but by default netstat doesn't show listening sockets so I repeated command adding -l option:
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
# netstat -w -lSo, this is definitely not what I'm looking for. This RAW socket listens for ICMP messages, and arpwatch definitely isn't capturing those. In man page it also says that netstat looks for information about raw sockets from /proc/net/raw file, so I looked into its content:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
raw 0 0 *:icmp *:* 7
# cat /proc/net/rawAlso not useful! There was inode listed (47105) but how to find out information about particular inode? I looked throughout /proc file system but didn't find anything. I also checked lsof manual but wasn't able to find something useful (though I didn't read manual from start to finish, I just search word inode!).
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
1: 00000000:0001 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 47105 2 ffff880308e18340 0
Then, I remembered that there is a ss command that is specific to Linux, and that is used to provide information about sockets! So, I looked into man page and there it says that the option -0 (or -f link) is used to show PACKET sockets, so I tried:
Again nothing, but it quickly occured to me that it doesn't show listening sockets by default, so I tried with -l (and -n to avoid any resolving):# ss -f link
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
Woohoo, I was on something, finally! I see raw socket bound to em1 interface (note that I started arpwach with the intention that it listens on wlan0 and em1 interfaces!) Only, I still don't know who is exactly using it. I only see that the other socket is datagram type, meaning network layer, and probably not used by arpwatch. man page for ss again helped, it says to use -p option to find out which process owns a socket, so I tried:# ss -f link -ln
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
p_raw UNCONN 0 0 *:em1 *
p_dgr UNCONN 0 0 [34958]:wlan0 *
# ss -f link -lnpWow!! That was it! I found out that arwatch is listening only to a single interface, and later I confirmed it by looking into the source! I also saw that the other socket is used by wpa_supplicant, i.e. for a wireless network management purposes.
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
p_raw UNCONN 0 0 *:em1 * users:(("arpwatch",23833,0))
p_dgr UNCONN 0 0 [34958]:wlan0 * users:(("wpa_supplicant",1425,9))
One final thing bothered me. From where does ss take this information? But it's easy to find out that, use strace! :) So, using strace I found out that ss is using /proc/net/packet file:
# cat /proc/net/packetMaybe I would get to that earlier if I had looked more closely into available files in /proc/net when /proc/net/raw turned out to be wrong file! But it doesn't matter, this search was fun and educative. :)
sk RefCnt Type Proto Iface R Rmem User Inode
ffff880308cec000 3 3 0003 2 1 0 0 27121497
ffff880004358800 3 2 888e 7 1 0 0 25292685
Malo o velikim sustavima i percepciji ljudi...
Ja sam laik što se tiče bankarskog sustava i ovo ovdje je moje viđenje stvari. Dakle, moram reći kako se prilično uzrujam kada čitam pojedine komentare u kojima se napada HNB. Neću tvrditi da HNB nije bez grijeha, nitko nije bez grijeha. Ali kada su argumenti u najmanju ruku nebulozni, što je čest slučaj onih koji komentiraju, onda me to stvarno uzruja i posljedica toga je ovaj post. Iako ovdje spominjem konkretan primjer bankarskog sustava, u biti radi se o općenitoj stvari o čemu ću na kraju posta više.
Naime, dosta često se u raznim raspravama na forumu vadi argument kako HNB negativno djeluje prema građanima i štiti interese banaka, a kao jedan od razloga za to se navodi što je Rohatinski član bankarskog kruga i u interesu mu je da radi za banke. Međutim, mislim da takve izjave pokazuju osnovno nepoznavanje velikih sustava, jer situacija nije baš tako jednostavna. Dakle, bankarski sustav je velik i složen i prva činjenica je da ga ne može voditi bilo tko, dapače, mora ga voditi netko tko jako dobro poznaje bankarski sustav i tko ima iskustva u vođenju velikih sustava, da ne spominjem i smisao za vođenje i rad s ljudima. Kada se uzme sve to u obzir, jasno je kako se radi o vrlo malom broju ljudi koji zadovoljavaju te uvjete. Naravno, uvijek se može staviti na bilo koji položaj bilo koga, ali da mi je vidjeti tko će tvrditi da će to dobro završiti - osim ako se ne poziva na čistu koincidenciju, što je moguće, ali izuzetno malo vjerojatno. Druga činjenica je da je stabilnost financijskog sustava, pa prema tome i banaka, nužan preduvjet stabilnosti države. Primarna zadaća HNB-a je održavanje stabilnosti financijskog sustava RH, a jedan od mehanizama s kojim se to postiže je i kontrola banaka. Sve super, samo da nema jednog neizbježnog problema. Postoji izreka koja kaže tko je jači taj kači, i to je i ovdje slučaj. Bez obzira što je država, u osnovi, najjači igrač naspram svih banaka, rat između države i banaka bi bila pirova pobjeda za državu. Dakle, kompromisi se jednostavno moraju raditi i to je neizbježno, a kompromis znači da se mora popuštati na obje strane.
Sada, konačno, dolazim i do percepcije ljudi. Ljudi vide stvari koje ih direktno tište, ali ne vide cjelinu. Pa prema tome, kada HNB napravi nešto što ide kao minus građanima, onda se pljuje po HNB-u, kada napravi nešto što ne ide na ruku bankama, onda najčešće to ti isti građani ne vide i u konačnici smatraju da HNB radi samo protiv njih, a u korist banaka. S tim ne želim reći da je HNB uvijek u pravu, čak obratno, vjerojatno je načinjeno dosta pogrešaka. Međutim, u svakom složenom sustavu, i uz najbolju namjeru, pogreške su neumitne!
Osobno mislim kako bilo koji veliki sustav ima iste "probleme" kao i HNB, međutim, razlika je da to ljudi ne osjete direktno na svojoj koži i prema tome ne reagiraju na to. Zaključak je da u Hrvatskoj ima četiri milijuna izbornika, guvernera HNB-a, predsjednika vlade, itd.
Naime, dosta često se u raznim raspravama na forumu vadi argument kako HNB negativno djeluje prema građanima i štiti interese banaka, a kao jedan od razloga za to se navodi što je Rohatinski član bankarskog kruga i u interesu mu je da radi za banke. Međutim, mislim da takve izjave pokazuju osnovno nepoznavanje velikih sustava, jer situacija nije baš tako jednostavna. Dakle, bankarski sustav je velik i složen i prva činjenica je da ga ne može voditi bilo tko, dapače, mora ga voditi netko tko jako dobro poznaje bankarski sustav i tko ima iskustva u vođenju velikih sustava, da ne spominjem i smisao za vođenje i rad s ljudima. Kada se uzme sve to u obzir, jasno je kako se radi o vrlo malom broju ljudi koji zadovoljavaju te uvjete. Naravno, uvijek se može staviti na bilo koji položaj bilo koga, ali da mi je vidjeti tko će tvrditi da će to dobro završiti - osim ako se ne poziva na čistu koincidenciju, što je moguće, ali izuzetno malo vjerojatno. Druga činjenica je da je stabilnost financijskog sustava, pa prema tome i banaka, nužan preduvjet stabilnosti države. Primarna zadaća HNB-a je održavanje stabilnosti financijskog sustava RH, a jedan od mehanizama s kojim se to postiže je i kontrola banaka. Sve super, samo da nema jednog neizbježnog problema. Postoji izreka koja kaže tko je jači taj kači, i to je i ovdje slučaj. Bez obzira što je država, u osnovi, najjači igrač naspram svih banaka, rat između države i banaka bi bila pirova pobjeda za državu. Dakle, kompromisi se jednostavno moraju raditi i to je neizbježno, a kompromis znači da se mora popuštati na obje strane.
Sada, konačno, dolazim i do percepcije ljudi. Ljudi vide stvari koje ih direktno tište, ali ne vide cjelinu. Pa prema tome, kada HNB napravi nešto što ide kao minus građanima, onda se pljuje po HNB-u, kada napravi nešto što ne ide na ruku bankama, onda najčešće to ti isti građani ne vide i u konačnici smatraju da HNB radi samo protiv njih, a u korist banaka. S tim ne želim reći da je HNB uvijek u pravu, čak obratno, vjerojatno je načinjeno dosta pogrešaka. Međutim, u svakom složenom sustavu, i uz najbolju namjeru, pogreške su neumitne!
Osobno mislim kako bilo koji veliki sustav ima iste "probleme" kao i HNB, međutim, razlika je da to ljudi ne osjete direktno na svojoj koži i prema tome ne reagiraju na to. Zaključak je da u Hrvatskoj ima četiri milijuna izbornika, guvernera HNB-a, predsjednika vlade, itd.
Tuesday, February 7, 2012
A bit more of history...
Well, I wrote about a post in which Rob Landley explains how directory hierarchy within Unix is actually artefact of the shortcoming of the available technology in a specific point in time. And, what's more interesting, not technology in general but in this case creators of Unix didn't have larger disks available to them!
Today, I stumbled on another post, (also here and see here about password "problem" in general) which describes, probably, the first password hack. But reading that post I learned several more things. First, how mail was actually invented before Internet, but also I have read a more detailed history of CTSS system. CTSS was very influential operating system and precursor to Multics, another very influential operating system. I think that knowing at least Multics, could be regarded as a basic knowledge of anyone calling himself a computer scientist or anything similar.
It was also interesting to read how people that made mail were afraid of US postal service. Basically, they thought that this could be regarded as a competition to regular postal service and that they could be fined. I believe that US Postal Service, as well as AT&T, were unimaginable monopolies from today's standpoint. Nevertheless, the same situation was similar in Europe, too. This became clear to me while reading a book about Internet history. First, the fact that AT&T didn't think ARPANet will ever work and thus were not interfering, actually helped ARPANet a lot. On the other hand, Franch PTT actually killed CYCLADES network that had all prerequisites to become the first true Internet.
And while I'm at mail, it is widely regarded that the first spam was sent in 1978, but according to the post about mail, first spam message was an Vietnam anti-war message sent by a MIT engineer who abused his privileges in order to be able to send a message to everyone.
All this, very influential operating systems, electronic mail service, Internet, spam, all that happened in the MIT during 60ties and 70ties. Somehow I wish I had a chance to take part in that, but then again, we still have a chance to do something else. :)
For the end, have you ever wondered when the term device driver was invented? Here is an explanation. Also, here are descriptions of early C compilers and a history of C programming language.
Today, I stumbled on another post, (also here and see here about password "problem" in general) which describes, probably, the first password hack. But reading that post I learned several more things. First, how mail was actually invented before Internet, but also I have read a more detailed history of CTSS system. CTSS was very influential operating system and precursor to Multics, another very influential operating system. I think that knowing at least Multics, could be regarded as a basic knowledge of anyone calling himself a computer scientist or anything similar.
It was also interesting to read how people that made mail were afraid of US postal service. Basically, they thought that this could be regarded as a competition to regular postal service and that they could be fined. I believe that US Postal Service, as well as AT&T, were unimaginable monopolies from today's standpoint. Nevertheless, the same situation was similar in Europe, too. This became clear to me while reading a book about Internet history. First, the fact that AT&T didn't think ARPANet will ever work and thus were not interfering, actually helped ARPANet a lot. On the other hand, Franch PTT actually killed CYCLADES network that had all prerequisites to become the first true Internet.
And while I'm at mail, it is widely regarded that the first spam was sent in 1978, but according to the post about mail, first spam message was an Vietnam anti-war message sent by a MIT engineer who abused his privileges in order to be able to send a message to everyone.
All this, very influential operating systems, electronic mail service, Internet, spam, all that happened in the MIT during 60ties and 70ties. Somehow I wish I had a chance to take part in that, but then again, we still have a chance to do something else. :)
For the end, have you ever wondered when the term device driver was invented? Here is an explanation. Also, here are descriptions of early C compilers and a history of C programming language.
Sunday, February 5, 2012
Getting CentOS on RAID after using text mode installation...
Well, this was a hack. I got a firewall to install with only 512MB RAM which isn't enough for graphical install. And, using text install means that there are no customizations available, in particular, no customized disk partitioning. But since I don't install anything important without RAID I had to somehow do it. The options are:
So, the general idea is as follow:
Install CentOS on a helper partition
This is a first step and it's easy. Boot from DVD and start installation process. Except from one small detail. Namely, how to persuade installer to use helper partition when it does almost everything automatically? Well, that's actually easy to solve. After Anaconda starts, but before doing anything, switch to second virtual terminal (Alt+F2) and use fdisk to create partitions for RAID. In my case those were:
Create RAID arrays and move CentOS there
After the installer finishes, don't boot into a new system. Boot again from DVD and select rescue mode. Also, allow installer to search for existing installations and select to mount them in read-write mode! Finally, select shell from a menu that appears.
Now, create RAID arrays. If you used partitions as I did, then the following sequence of commands will do the work.
Finally, remove temporary partition. Do that using fdisk, just delete that partition. Note that this will take effect on next reboot, so nothing will happen and the computer won't crash. :)
And, that's it for moving CentOS to RAID partition.
Fixing boot loader
This is the last step. Boot from DVD and select rescue mode. Again allow installer to scan disk for Linux partitions. But this time installer should find RAID partitions and CentOS on them. Mount them in read-write mode and enter shell again. Execute the following command:
First, we have to recreate /etc/mdadm.conf. Note that if you don't do that the system won't boot. This troubled me until I figured out that that file contained stale data filled by Anaconda and I didn't refresh its content. So, to refresh it, do the following:
Now, recreate initramfs image. Do that with the following command:
One final step and that's it, reinstall the boot loader! Do this with the following commands:
- Temporarily add more RAM only during installation process.
- Use kickstart file.
- Juggle with partitions.
So, the general idea is as follow:
- Install CentOS on a temporary partition.
- Create RAID array and move CentOS there.
- Fix boot.
Install CentOS on a helper partition
This is a first step and it's easy. Boot from DVD and start installation process. Except from one small detail. Namely, how to persuade installer to use helper partition when it does almost everything automatically? Well, that's actually easy to solve. After Anaconda starts, but before doing anything, switch to second virtual terminal (Alt+F2) and use fdisk to create partitions for RAID. In my case those were:
- /dev/sda1 and /dev/sdb1 of size 256MB for /boot partition
- /dev/sda2 and /dev/sdb2 of size 2G for swap
- /dev/sda3 and /dev/sdb3 that take the rest of space to fill 80GB and that will be root (/) partition.
Create RAID arrays and move CentOS there
After the installer finishes, don't boot into a new system. Boot again from DVD and select rescue mode. Also, allow installer to search for existing installations and select to mount them in read-write mode! Finally, select shell from a menu that appears.
Now, create RAID arrays. If you used partitions as I did, then the following sequence of commands will do the work.
This creates first RAID1 array for boot partition. Note that I'm using metadata version 0.90. This is because grub doesn't understand later formats! That "little" fact costed me a lot of time!mdadm -C /dev/md0 --metadata=0.90 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm -C /dev/md1 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2Those two commands create RAID1 arrays for swap and root. Maybe at this point you'll need to activate arrays (i.e. it isn't done automatically by above mdadm commands):
mdadm -C /dev/md2 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
mdadm --activate --scanNow, create file systems on those arrays:
mkfs.ext4 /dev/md0RAID arrays are now created. The next step is to move installed CentOS from temporary partition. First, you have to mount destination filesystems:
mkfs.ext4 /dev/md2
mkswap /dev/md1
After they are mounted, copy all the files:mkdir /mnt/smount /dev/md2 /mnt/smkdir /mnt/s/bootmount /dev/md0 /mnt/s/boot
Note that I skipped some in-memory file systems like proc, dev, and similar ones. You should only create those directories without content as their content is recreated during each boot and held in memory:cd /mnt/sysimagersync -av bin boot etc home lib lib64 opt root sbin tmp usr var /mnt/s/
cd /mnt/sThat's it for filesystems. We need now to adjust /etc/fstab and /etc/grub.conf files as they reference temporary partition/filesystem used by installer. So, change /etc/fstab to contain the following lines/filesystems:
mkdir dev media mnt proc selinux srv sys
/dev/md2 / ext4 defaults 1 1There will be also lines starting with tmpfs, devpts, sysfs and proc. Leave those as is and remove all the other lines. As for the /etc/grub.conf file, you need to modify any occurrences of (hdN,M). Those will be in two places. One in splashimage line, that one isn't so important. And the other in line starting with root keyword. That one IS important! Also, remove from the line that starts with keyword kernel any word that contains substring LV (logical volume!). After a change, this line should look something like this:
/dev/md0 /boot ext4 defaults 1 2
/dev/md1 swap swap defaults 0 0
kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/md2 LANG=en_US.UTF-8 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=usNote that this is a single line! The important part is root=, others might differ if you selected some other language or keyboard layout during installation process. Also, I removed keywords rhgb and quiet. Those prevent me to see kernel messages during the boot process. And those messages could be very important, especially in this case when we are doing something that could impact early boot process itself!
Finally, remove temporary partition. Do that using fdisk, just delete that partition. Note that this will take effect on next reboot, so nothing will happen and the computer won't crash. :)
And, that's it for moving CentOS to RAID partition.
Fixing boot loader
This is the last step. Boot from DVD and select rescue mode. Again allow installer to scan disk for Linux partitions. But this time installer should find RAID partitions and CentOS on them. Mount them in read-write mode and enter shell again. Execute the following command:
chroot /mnt/sysimageThis will activate targeted CentOS on RAID.
First, we have to recreate /etc/mdadm.conf. Note that if you don't do that the system won't boot. This troubled me until I figured out that that file contained stale data filled by Anaconda and I didn't refresh its content. So, to refresh it, do the following:
Open /etc/mdadm.conf in the editor and add the following line at the beginning:mdadm --examine --scan > /etc/mdadm.conf
Also be careful, I typed etc instead of dev and it costed me two additional reboots. :)DEVICES /dev/sd*[0-9] /dev/hd*[0-9]
Now, recreate initramfs image. Do that with the following command:
mkinitrd -f -v /boot/initramfs-2.6.32-220.el6.x86_64.img 2.6.32-220.el6.x86_64Be careful, -f forces mkinitrd to overwrite existing initramfs file, so maybe it would be good to make a copy of that file, just in case.
One final step and that's it, reinstall the boot loader! Do this with the following commands:
If the two commands return error (some problem with stage1 or stage2 files) then do it "manually" like this:grub-install /dev/sdagrub-install /dev/sdb
# grubNote that you type text in bold, while everything else is response from the system. And that's it! Reboot the system, this time from the hard disk, and you should have minimal CentOS installation on RAID partitions!
grub> root (hd0,0)
grub> setup (hd0)
grub> root (hd1,0)
grub> setup (hd1)
grub> exit
Error: cannot open tty-output
I wrote a script whose purpose is to offer a user an option to select test or production environment when connecting to a server and then, based on the selection, script configures environment variables appropriately. To make a script a bit more user friendly I used dialog utility. All good, until user logs in and then switches to another, non-root(!), user using su command at which point the following error is reported:
There was solution to use temporary file for that purpose, but I was reluctant to do so. The first idea was to use pipe and read statement, something like this:
In the end, I modified dialog invocation in the following way:
cannot open tty-outputWell, a simple and quick use of strace revealed what is the problem. Namely, initial user is owner of pseudo-terminal (e.g. /dev/pts/3) and when he/she switches to another user the ownership isn't changed so opening a terminal device is unsuccessful. What caused dialog tool to try to open pseudo-terminal in the first place was --stdout switch, and if this switch isn't used then there is no error about tty-output. But then, there is another problem and that's why this option was used in the first place. Namely, I used dialog in the following way:
ANS=`dialog ... --stdout`to catch output of the command into variable and to be able to test it within if statement. The problem was that there was no output without --stdout option, as it was going to stderr and thus invoking shell couldn't catch output and place it into variable ANS. That is, dialog tool uses stdout to draw on terminal and stderr to output user's response!
There was solution to use temporary file for that purpose, but I was reluctant to do so. The first idea was to use pipe and read statement, something like this:
dialog ... | read ANSand in that way to avoid use of --stdout. But the problem is that pipe components (command on left and right of pipe character) are executed within subshells and thus, the result of read command isn't visible to a parent shell. In other words, this is dead end. And besides, I still have a problem of read getting the data via dialog's stdout!
In the end, I modified dialog invocation in the following way:
ANS=`dialog ... 2>&1 > /dev/tty`What this does is the following: parent shell invokes subshell that will execute everything between backticks, but first, it redirects stdout so that it can later place output to variable ANS. Subshell then duplicates stderr so that it points to stdout (which isn't actually stdout but is a pipe to parent shell). In this way the output of the dialog command will be taken by parent bash, i.e. the one that starts subshell to execute backtick command. Then, stdout is redirected to /dev/tty that will be always possible to open/read/write and that allows dialog to control display on terminal.
Calculating TCP RTO...
I was reading RFC6298 on RTO calculation and decided to try to see within Linux kernel how and where it is calculated. Basically, RTO, or Retransmittion Timeout, determines how long TCP waits for acknowledgment (ACK) of transmitted segment. If the acknowledgment isn't received within this time it is deemed lost. Actually, ACK could be lost too, but there is no way for sender to differentiate between those two cases, as illustrated by the following figure:
Thus I'll treat them equally and always refer to segment loss.
The important part of calculating RTO is to determine how long it takes for a segment to go to the receiver and for ACK to come back from receiver to sender. This is a Round Trip Time, or RTT. In some ideal world (and very static for that matter) this value would be constant and would never change. And RTO would be easy to determine, it is equal to RTT, maybe slightly slightly larger, but nevertheless the two would be almost equal.
But we are not living in an ideal word and this process is complicated by the fact that network conditions constantly change. Not only that, but receiver has also certain freedom to chose when it will return ACK, though this time has upper bound of 500ms. So, RTT is never used directly as RTO, some additional calculations are used. The key is to estimate as good as possible the true value of RTT that will be experienced by the next segment to be transmitted and in the same time avoid abrupt changes resulting in transient conditions, and not to react too slow on network condition changes.This is illustrated with the following figure:
In order to achieve that, two new variables are introduced, smoothed RTT, or short SRTT, and RTT variance, or RTTVAR. Those two variables are updated, whenever we have a new RTT measurement, like this (taken from the RFC6298):
Finally, RTO is calculated like this:
Initial values
Still, there is a question of initial values, i.e. what to do when first SYN segment is sent? In that case RFC specifies you have to set RTO to 1 second, which is actually lower than specified in the previous RFC that mandated minimum value of 3 seconds. When first acknowledgment returns its RTT value is stored into SRTT and variance is set to RTT/2. Then, RTO is calculated as usual.
Some complications
There are some additional rules that are required by RFC. First, if calculated RTO is less than 1s, then it has to be rounded to 1second. It is a minimul RTO value allowed by RFC.
Next, in case some segment is retransmitted, then when acknowledgement arrives it is not taken into calculation of SRTT and RTTVAR. This is called Karn's algorithm, even though it is not algorithm but more a rule. The reason for it is that it is impossible to know if this is acknowledgement for a first transmission, or for retransmission and thus we could skew SRTT. This ambiguity is illustrated with the following figure:
But, there is possibility for TCP to negotiate timestamp option on a certain connection and in that case, the previous ambiguity is resolved so each ACK can be used to calculate SRTT and RTTVAR.
Implementation within Linux kernel
Now, let us see how and where is this implemented within the Linux kernel. I'm using the latest stable kernel at the time this post was written and that is 3.2.4. If you are looking at some later or earlier kernel, then the things might be more or less different.
The call graph of the relevant functions is shown in the following figure:
The main function to calculate RTO is tcp_valid_rtt_meas() which updates RTT estimation and sets new RTO for future segments that will be sent. It is called by two functions, tcp_ack_saw_tstamp() which processes ACK that has embedded timestamp option, or tcp_ack_no_tstamp() which processes ACK without timestamp option. In both cases, tcp_valid_rtt_meas() is called with socket structure that determines to which connection this measurement belongs to and also measured RTT value.
But before describing functions that do calculations, first we are going to describe used data structures. Actually, we'll describe only those elements that are used to calculate RTO.
Data structures
The main data structure passed between all the functions we describe is struct sock. This is a fairly large structure used to describe network layer data about every possible type of socket. Every socket from any higher layer has this structure placed at the beginning. The following figure illustrates this:
In our case, the higher layer structure we are interested in is TCP. The structure used to describe TCP sockets is struct tcp_sock. So, when our functions get struct sock as argument, they use use tcp_sk inline function to convert (cast!) it into struct tcp_sock data structure. Note that, if you think a little about it, this tcp_sk inline function actually is a no-op after compilation! It's only purpose is casting which is high-level thing, not something important for assembly/machine code.
Anyway, in struct tcp_sock there is a set of variables used for RTO calculation:
In there we note two expected variables, srtt and rttvar, but also several other ones. Also what is important to realize is that srtt var contains number of seconds shifted to left by three bits, i.e. multiplied by 8. So, in order to store 1 second in srtt you'll have to write there number 8. In other words, every value counts as 125ms, so if you want to store 0.5s you'll write there number 4, i.e. 4*125ms = 500ms. Similarly, mdev is counted in units of 250ms, i.e. its value is four time smaller and to store there 1s you need to write number 4 (4*250ms = 1s).
We'll see later that this way of storing data, along with the requirements for calculating RTO specified in RFC, allow for very efficient (if somewhat obscured) code to determine srtt and rttvar.
As indicated in comments embedded in code, there are additional variables that allow RTTVAR to be updated every RTT time units. The mechanism to achieve that is the following. At each ACK, mdev is calculated and if this mdev is higher than the current highest one (mdev_max) then it is stored into mdev_max field. When RTT time units passes, mdev_max is used to update RTTVAR. To know when RTT time units passed, the field rtt_seq is used. Namely, the sequence number within ACK received is checked against rtt_seq, and if it is larger than rtt_seq than RTTVAR update is triggered and in the same time rtt_seq is set to sequence number that will be used in the next outgoing segment (snd_nxt, also part of the tcp_sock structure).
Functions
Now that we have described relevant data structures, let us turn our attention to the functions themselves. We'll start from the end, i.e. from the calculation of RTO.
tcp_set_rto()
This is the function that calculates current RTO, even though it actually does this via a call to inline function __tcp_set_rto() . RTO is calculated as follows:
The function tcp_set_rto() also makes sure that RTO isn't larger than TCP_RTO_MAX, which is set to 120 seconds (i.e. 120*HZ).
This function, in addition to struct sock parameter that holds data for TCP connection whose SRTT and RTTVAR variables should be updated, also receives measured RTT value for the received acknowledgment, i.e.
So, the first thing that's done within this function is the following (slightly modified for explanation):
It also has to update mdev, which is done with the following line (note that mrtt isn't changed while the previous calculations were performed, i.e. it is still R' - SRTT):
When mdev is calculated, it is checked against mdev_max and if it is larger, then mdev_max is updated to a new value:
One more thing is done too, if mdev_max is larger then rttvar then rttvar is also immediately updated with a new value. Note a trick here. RFC requires RTTVAR to be multiplied by a constant K which is set to be 4! This is accomplished with assigning mdev_max (which is already multiplied by 4) directly to rttvar!
What's left to do is to check if RTT time units has passed from last (regular) update to RTTVAR, and if it is then it's time to update it again. This is done within the following code fragment:
Note that if rtt_var is larger than mdev_max nothing happens, i.e. this code only decreases the value of rttvar! But, if it is smaller, then rttvar is adjusted by the following quantity (as per RFC):
One more thing left to explain is the new value of mdev_max and the function tcp_rto_min that is called to provide it. This function hides some complexity, but in the simplest possible case it will return constant TCP_RTO_MIN which has value 200ms (HZ/5). In more general case, the ip command from iproute package allows RTT and RTTVAR to be specified per destination so this function checks if it is specified and if it is then returns given value.
The special case for this function is the initial state, i.e. when the connection is opened. In that case argument mrtt will be zero, and also srtt will be zero. If mrtt is zero, assumed value is 1, and note that it is the initial RTT defined by RFC. srtt being zero triggers it's initialization:
And that's it. I intend also to describe TCP congestion control within Linux kernel in some future post.
Thus I'll treat them equally and always refer to segment loss.
The important part of calculating RTO is to determine how long it takes for a segment to go to the receiver and for ACK to come back from receiver to sender. This is a Round Trip Time, or RTT. In some ideal world (and very static for that matter) this value would be constant and would never change. And RTO would be easy to determine, it is equal to RTT, maybe slightly slightly larger, but nevertheless the two would be almost equal.
But we are not living in an ideal word and this process is complicated by the fact that network conditions constantly change. Not only that, but receiver has also certain freedom to chose when it will return ACK, though this time has upper bound of 500ms. So, RTT is never used directly as RTO, some additional calculations are used. The key is to estimate as good as possible the true value of RTT that will be experienced by the next segment to be transmitted and in the same time avoid abrupt changes resulting in transient conditions, and not to react too slow on network condition changes.This is illustrated with the following figure:
In order to achieve that, two new variables are introduced, smoothed RTT, or short SRTT, and RTT variance, or RTTVAR. Those two variables are updated, whenever we have a new RTT measurement, like this (taken from the RFC6298):
alpha and beta are parameters that determine how fast we forget the past. If this parameter is too small new measurements will have little influence on our current understanding of expected RTT and we will slowly react to changes. If, on the other hand, alpha approaches 1 then the past will not influence our current estimation of RTT and it might happen that a single RTT was huge for whatever reason and that suddenly we have wrong estimation. Not only that, but we could have erratic behavior of SRTT. So, alpha and beta parameters have to be carefully selected. The values recommended by RFC are alpha=1/8 and beta=1/4.RTTVAR <- (1 - beta) * RTTVAR + beta * |SRTT - R'| SRTT <- (1 - alpha) * SRTT + alpha * R'
Finally, RTO is calculated like this:
Constant K is set to 4, and G is a clock granularity in seconds, i.e. if you get timer interrupt each second, then G is 1 second. The max function is used so that, e.g. you don't get 400ms for K*RTTVAR and try to wait for that long while your clock has resolution of 1s. In that case, 1s will prevail and will be selected as a variance.RTO <- SRTT + max (G, K*RTTVAR)
Initial values
Still, there is a question of initial values, i.e. what to do when first SYN segment is sent? In that case RFC specifies you have to set RTO to 1 second, which is actually lower than specified in the previous RFC that mandated minimum value of 3 seconds. When first acknowledgment returns its RTT value is stored into SRTT and variance is set to RTT/2. Then, RTO is calculated as usual.
Some complications
There are some additional rules that are required by RFC. First, if calculated RTO is less than 1s, then it has to be rounded to 1second. It is a minimul RTO value allowed by RFC.
Next, in case some segment is retransmitted, then when acknowledgement arrives it is not taken into calculation of SRTT and RTTVAR. This is called Karn's algorithm, even though it is not algorithm but more a rule. The reason for it is that it is impossible to know if this is acknowledgement for a first transmission, or for retransmission and thus we could skew SRTT. This ambiguity is illustrated with the following figure:
But, there is possibility for TCP to negotiate timestamp option on a certain connection and in that case, the previous ambiguity is resolved so each ACK can be used to calculate SRTT and RTTVAR.
Implementation within Linux kernel
Now, let us see how and where is this implemented within the Linux kernel. I'm using the latest stable kernel at the time this post was written and that is 3.2.4. If you are looking at some later or earlier kernel, then the things might be more or less different.
The call graph of the relevant functions is shown in the following figure:
The main function to calculate RTO is tcp_valid_rtt_meas() which updates RTT estimation and sets new RTO for future segments that will be sent. It is called by two functions, tcp_ack_saw_tstamp() which processes ACK that has embedded timestamp option, or tcp_ack_no_tstamp() which processes ACK without timestamp option. In both cases, tcp_valid_rtt_meas() is called with socket structure that determines to which connection this measurement belongs to and also measured RTT value.
But before describing functions that do calculations, first we are going to describe used data structures. Actually, we'll describe only those elements that are used to calculate RTO.
Data structures
The main data structure passed between all the functions we describe is struct sock. This is a fairly large structure used to describe network layer data about every possible type of socket. Every socket from any higher layer has this structure placed at the beginning. The following figure illustrates this:
In our case, the higher layer structure we are interested in is TCP. The structure used to describe TCP sockets is struct tcp_sock. So, when our functions get struct sock as argument, they use use tcp_sk inline function to convert (cast!) it into struct tcp_sock data structure. Note that, if you think a little about it, this tcp_sk inline function actually is a no-op after compilation! It's only purpose is casting which is high-level thing, not something important for assembly/machine code.
Anyway, in struct tcp_sock there is a set of variables used for RTO calculation:
/* RTT measurement */ u32 srtt; /* smoothed round trip time << 3 */ u32 mdev; /* medium deviation */ u32 mdev_max; /* maximal mdev for the last rtt period */ u32 rttvar; /* smoothed mdev_max */ u32 rtt_seq; /* sequence number to update rttvar */
In there we note two expected variables, srtt and rttvar, but also several other ones. Also what is important to realize is that srtt var contains number of seconds shifted to left by three bits, i.e. multiplied by 8. So, in order to store 1 second in srtt you'll have to write there number 8. In other words, every value counts as 125ms, so if you want to store 0.5s you'll write there number 4, i.e. 4*125ms = 500ms. Similarly, mdev is counted in units of 250ms, i.e. its value is four time smaller and to store there 1s you need to write number 4 (4*250ms = 1s).
We'll see later that this way of storing data, along with the requirements for calculating RTO specified in RFC, allow for very efficient (if somewhat obscured) code to determine srtt and rttvar.
As indicated in comments embedded in code, there are additional variables that allow RTTVAR to be updated every RTT time units. The mechanism to achieve that is the following. At each ACK, mdev is calculated and if this mdev is higher than the current highest one (mdev_max) then it is stored into mdev_max field. When RTT time units passes, mdev_max is used to update RTTVAR. To know when RTT time units passed, the field rtt_seq is used. Namely, the sequence number within ACK received is checked against rtt_seq, and if it is larger than rtt_seq than RTTVAR update is triggered and in the same time rtt_seq is set to sequence number that will be used in the next outgoing segment (snd_nxt, also part of the tcp_sock structure).
Functions
Now that we have described relevant data structures, let us turn our attention to the functions themselves. We'll start from the end, i.e. from the calculation of RTO.
tcp_set_rto()
This is the function that calculates current RTO, even though it actually does this via a call to inline function __tcp_set_rto() . RTO is calculated as follows:
(tp->srtt >> 3) + tp->rttvarWhich is actually the same as in RFC apart from right shift. The right shift is necessary because srtt is expressed in 8 times smaller units and has to be normalized before being added to rttvar. rttvar, on the other hand, is coded "normally", i.e. number one means 1 second.
The function tcp_set_rto() also makes sure that RTO isn't larger than TCP_RTO_MAX, which is set to 120 seconds (i.e. 120*HZ).
tcp_rtt_estimator()
This function, in addition to struct sock parameter that holds data for TCP connection whose SRTT and RTTVAR variables should be updated, also receives measured RTT value for the received acknowledgment, i.e.
static void tcp_rtt_estimator(struct sock *sk, const __u32 mrtt);Update process is done in a highly optimized way, which makes things a bit obscure at the first sight. This is the consequence of the fact that srtt and mdev are counted in units of 125ms and 250ms, respectively (as explained in the subsection about data structures).
So, the first thing that's done within this function is the following (slightly modified for explanation):
mrtt -= (tp->srtt >> 3);Now, mrtt holds value R' - SRTT (using notation from RFC). To calculate new SRTT the following line, immediatelly after the previous one, is used:
srtt += mrtt;And that's it for SRTT! But, it is easier to understand this calculation if we write it as follows:
mrtt' = mrtt - srtt/8which is actually the equation given in RFC. Again, srtt is 8 times larger so I can normalize it to show that the value will be correct:
srtt = srtt + mrtt' = srtt + mrtt - srtt/8 = 7/8 * srtt + mrtt
8*rsrtt = 7/8 * 8 * rsrtt + mrtt (divide by 8 yields):I'm using rsrtt to denote real srtt.
rsrtt = 7/8 rsrtt + mrtt/8
It also has to update mdev, which is done with the following line (note that mrtt isn't changed while the previous calculations were performed, i.e. it is still R' - SRTT):
mrtt -= (tp->mdev >> 2);again, I slightly simplified the code. The simplification is related to the fact that I'm assuming mrtt is positive after substraction that changes it into R' - SRTT. Again, the trick is used that mdev is stored in 4 time smaller units.
tp->mdev += mrtt;
When mdev is calculated, it is checked against mdev_max and if it is larger, then mdev_max is updated to a new value:
if (tp->mdev > tp->mdev_max) {
tp->mdev_max = tp->mdev;
if (tp->mdev_max > tp->rttvar)
tp->rttvar = tp->mdev_max;
}
One more thing is done too, if mdev_max is larger then rttvar then rttvar is also immediately updated with a new value. Note a trick here. RFC requires RTTVAR to be multiplied by a constant K which is set to be 4! This is accomplished with assigning mdev_max (which is already multiplied by 4) directly to rttvar!
What's left to do is to check if RTT time units has passed from last (regular) update to RTTVAR, and if it is then it's time to update it again. This is done within the following code fragment:
if (after(tp->snd_una, tp->rtt_seq)) {
if (tp->mdev_max < tp->rttvar)
tp->rttvar -= (tp->rttvar - tp->mdev_max) >> 2;
tp->rtt_seq = tp->snd_nxt;
tp->mdev_max = tcp_rto_min(sk);
}
As we said when we were talking about data structures, indicator that RTT units has passed is signaled with sequence number within ACK being after saved value in snd_una field. You might wonder why simple less-then operator isn't used? The reason is that sequence numbers might wrap around so it is necessary to take that into account!Note that if rtt_var is larger than mdev_max nothing happens, i.e. this code only decreases the value of rttvar! But, if it is smaller, then rttvar is adjusted by the following quantity (as per RFC):
rttvar - (rttvar - mdev) / 4 = 3/4 * rttvar + mdev/4Again, we have some trickery with different scales of rttvar and mdev. You can understand it as follows: New rttvar consists of 3/4 old rttvar. rttvar itself is multiplied by 4 (i.e. by constant K from RFC). RFC also specifies that mdev must participate with 1/4 (i.e. factor beta). Additionaly, mdev is already 4 times larger, and thus, it is already pre-multiplied by constant K! Thus, it can be added without further multiplications.
One more thing left to explain is the new value of mdev_max and the function tcp_rto_min that is called to provide it. This function hides some complexity, but in the simplest possible case it will return constant TCP_RTO_MIN which has value 200ms (HZ/5). In more general case, the ip command from iproute package allows RTT and RTTVAR to be specified per destination so this function checks if it is specified and if it is then returns given value.
The special case for this function is the initial state, i.e. when the connection is opened. In that case argument mrtt will be zero, and also srtt will be zero. If mrtt is zero, assumed value is 1, and note that it is the initial RTT defined by RFC. srtt being zero triggers it's initialization:
tp->srtt = mdev << 3;Basically, transcodes mdev into appropriate units and stores the value into srtt (i.e. mdev is 1, so into srtt has to be stored 8). At first, it might seem that mdev is calculated wrongly, but according to RFC, it's initial value has to be half of the initial RTT. This translates into:
tp->mdev = mdev << 1;
mdev<<2 / 2 = mdev*4/2 = mdev*2 = mdev <<1So, mdev's initial value is correct!
And that's it. I intend also to describe TCP congestion control within Linux kernel in some future post.
Friday, February 3, 2012
More news about security incidents...
During this week there were several hacks and security related events. I'll summarize them here.
First, there was a news that VeriSign was hacked, which is actually quite a big news. Here are some reactions to it. Not much is known what happened or what is a damage. It turns out that the hack happened in 2010. but neither management nor public were notified by technical staff. VeriSign reported incident in SEC filling where Reuters spotted it. What is important is that VeriSign is actually in a security business and it runs two very important services, issuing of SSL certificates and DNS system.
Second incident is related to Anonymous posting a recording of a conference call between FBI, Scotland Yard and some other law enforcement agencies. The conference call was about investigation of Anonymous and other similar groups. The mail message was sent to 44 different addresses across 8 different organizations. Quite a huge number. Apparently, anonymous got hold on the mail from one or more of the recipients of a mail message. The mail message contained also access code and conference call bridge telephone number (BridgeTN). This probably allowed crackers to dial to conference call bridge, enter access code and get into a conference call. Actually easy. The point is that Anonymous didn't intercept the call as many Web pages are screaming!
First, there was a news that VeriSign was hacked, which is actually quite a big news. Here are some reactions to it. Not much is known what happened or what is a damage. It turns out that the hack happened in 2010. but neither management nor public were notified by technical staff. VeriSign reported incident in SEC filling where Reuters spotted it. What is important is that VeriSign is actually in a security business and it runs two very important services, issuing of SSL certificates and DNS system.
Second incident is related to Anonymous posting a recording of a conference call between FBI, Scotland Yard and some other law enforcement agencies. The conference call was about investigation of Anonymous and other similar groups. The mail message was sent to 44 different addresses across 8 different organizations. Quite a huge number. Apparently, anonymous got hold on the mail from one or more of the recipients of a mail message. The mail message contained also access code and conference call bridge telephone number (BridgeTN). This probably allowed crackers to dial to conference call bridge, enter access code and get into a conference call. Actually easy. The point is that Anonymous didn't intercept the call as many Web pages are screaming!
Promjena uvjeta korištenja Google usluga...
Google je nedavno promijenio uvjete korištenja svojih usluga i o tome je slao obavijesti na različite načine kako bi svatko mogao vidjeti koji su novi uvjeti. Međutim, tko uopće čita uvjete korištenja? ;) I onda se desi nekakav problem i ispadne da je nešto pisali sitnim slovima, ali tko čita uopće sitna slova?! Toliko puta ispričana priča u svakodnevnom životu, ne samo vezano za Internet i računala. Primjerice, oni famozni krediti gdje su se ljudi žalili na neke odredbe, a u stvari nisu čitali što piše. Uglavnom, zaključak je da je bitno znati što piše, bar okvirno.
Što se tiče Google-a malo je zakamuflirao promjene, međutim, nakon što im je par političara poslalo konkretno pitanje što je izmjenjeno, Google je jasno odgovorio. Ovdje imate sažetak na Engleskom jeziku, a ja ću ponoviti stvari na Hrvatskom. :)
Do sada Google je imao takve uvjete korištenja podataka da nije smio miješati podatke o korisnicima koje je prikupio, primjerice, na YouTubu ili tijekom pretraživanja s Google+ uslugom. To je značilo da ako bi korisnik pretraživao recepte putem tražilice, Google nije mogao korisniku ponuditi video isječke na YouTubu koji su vezani uz kuhanje, i to bez obzira što je korisnik bio prijavljen na oba servisa. Po novome, on će to moći. Ako netko ne želi da mu se miješaju podaci, tj. želi i dalje držati podatke razdvojenima, rješenje je otvaranje više korisničkih računa, za svaku uslugu po jedan.
Što se tiče Google-a malo je zakamuflirao promjene, međutim, nakon što im je par političara poslalo konkretno pitanje što je izmjenjeno, Google je jasno odgovorio. Ovdje imate sažetak na Engleskom jeziku, a ja ću ponoviti stvari na Hrvatskom. :)
Do sada Google je imao takve uvjete korištenja podataka da nije smio miješati podatke o korisnicima koje je prikupio, primjerice, na YouTubu ili tijekom pretraživanja s Google+ uslugom. To je značilo da ako bi korisnik pretraživao recepte putem tražilice, Google nije mogao korisniku ponuditi video isječke na YouTubu koji su vezani uz kuhanje, i to bez obzira što je korisnik bio prijavljen na oba servisa. Po novome, on će to moći. Ako netko ne želi da mu se miješaju podaci, tj. želi i dalje držati podatke razdvojenima, rješenje je otvaranje više korisničkih računa, za svaku uslugu po jedan.
Subscribe to:
Posts (Atom)