Showing posts with label disk. Show all posts
Showing posts with label disk. Show all posts

Saturday, August 23, 2014

Memory access latencies

Once, I saw a table in which all the memory latencies are scaled in such a way that CPU cycle is defined to be 1 second, and then L1 cache latency is several seconds, L2 cache even more, and so on up to SCSI commands timeout and system reboot. This was very interesting because I have much better developed sense for seconds and higher time units that for nanoseconds, microseconds, etc. Few days ago I remembered that table and I wanted to see it again, but couldn't find it.  This was from some book I couldn't remember the name. So, I started to google for it, and finally, after an hour or so of googling, I managed to find this picture. It turns out that this was from the book Systems performance written by Brendan Gregg. So, I decided to replicate it here for a future reference:


Table 2.2: Example Time Scale of System Latencies
Event Latency Scaled
1 CPU Cycle 0.3 ns 1 s
Level 1 cache access 0.9 ns 3 s
Level 2 cache access 2.8 ns 9 s
Level 3 cache access 12.9 ns 43 s
Main memory access (DRAM, from CPU) 120 ns 6 min
Solid-state disk I/O (flash memory) 50 - 150 us 2-6 days
Rotational disk I/O 1-10 ms 1-12 months
Internet: San Francisco to New York 40 ms 4 years
Internet: San Francisco to United Kingdom 81 ms 8 years
Internet: San Francisco to Australia 183 ms 19 years
TCP packet retransmit 1-3 s 105-317 years
OS virtualization system reboot 4 s 423 years
SCSI command timeout 30 s 3 millennia
Hardware (HW) virtualization system reboot 40 s 4 millennia
Physical system reboot 5 min 32 millennia

It's actually impressive how fast CPU is with respect to other components. It is also very good argument for multitasking, i.e. assigning CPU to some other task while waiting for, e.g. disk, or something from the network.

One additional impressive thing is written below the table in the book. Namely, if you multiply CPU cycle with speed of light (c) you can see that the light can travel only 0.5m while CPU does one instruction. That's really impressive. :)

That's it for this post. For the end, while I was searching for this table, I stumbled on some additional interesting links:



Thursday, January 17, 2013

USB cable and strange behavior with disk in enclosure...

I think one of disks in USB disk enclosure I had just got broken because of faulty, or something, USB cable. Now, I don't know how it is possible, nor what exactly happened, but I have a strong feeling that I'm right. Namely, what happened is that when I plugged the cable into the enclosure I heard strange sounds, like the heads are trying to move but are being retracted back to initial position; a series of clicks, about a second apart. That happened almost every time I used that cable. At first, I thought that the problem is that the USB ports are USB3.0 while enclosure is USB2.0 and something is wrong with currents or who know what. But googling didn't turned anything about that. Then, I tried another one and disk worked normally. WTF?!

Well, I found out that when power source isn't strong enough the symptoms are clicking that's heard in the disk. In that case you should unplug the disk as soon as possible. Also, you probably received additional cable with a caddy that will allow you to solve this problem. What happened in my case, probably, is that the cable is somehow faulty and probably decreased current so that disk didn't have enough power.

Friday, January 4, 2013

Partition vs. whole disk and creating encrypted filesystem...

With a new laptop I got a 1T disk which I intend to use as a data disk. So, it will have a single encrypted partition. This is a new disk with a 4K sector size and because of that fdisk tool offers me to start partition on 2048th sector. This is some alignment stuff from the old days of MSDOS, and obviously I don't want to waste disk space for those reasons. You can read more about that on Linux ATA Wiki. Linux is the only OS I'll use with this disk. It is possible to start partition from 63rd sector but if you are using fdisk you'll have to first create a partition and then switch to expert menu (option x) where it is possible to move beginning of the partition from 2048th to 63rd sector (option b).

Now, it is also possible to use the whole disk for a filesystem, without partition table. I found some discussions of pros and cons of this approach. Additional question is if the Fedora will recognize such disks during a boot process. LWM HOWTO also talks about this issue. It seems that everything boils down to the problem if some other tools or operating systems, that expect disk to be partitioned, treat disk as unpartitioned and thus destroy data on it. Also, someone noted possible performance degradation, but this was not confirmed by simple testing (look at the first link I gave), and besides, why would that happen when you use the whole disk? It can not be better aligned, can it? Also, someone used the whole disk for his Gentoo OS and then he had to install GRUB. Since GRUB, during installation, asks you whether you want it to be installed on, e.g. /dev/sda or /dev/sda1, it seems that it isn't important if you don't have partition table. But, I didn't go more deeper in this.

In the end, I decided to use the whole disk, no partitions. This disk will hold a single partition, will have only data on it, it will never be used on anything other than Linux, actually, on anything other than my laptop. So, this is the way I decided to go.

So, from that point on everything was very simple:
  1. Encrypt the whole disk
  2. # cryptsetup luksFormat /dev/sdc

    WARNING!
    ========
    This will overwrite data on /dev/sdc irrevocably.

    Are you sure? (Type uppercase yes): YES
    Enter LUKS passphrase:
    Verify passphrase:
  3. Open crypted disk:
    # cryptsetup luksOpen /dev/sdc cryptodev1
    Enter passphrase for /dev/sdc:
  4. Create file system:
  5. # mkfs -t ext4 /dev/mapper/cryptodev1
    mke2fs 1.42.5 (29-Jul-2012)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    Stride=0 blocks, Stripe width=0 blocks
    61054976 inodes, 244190134 blocks
    12209506 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=4294967296
    7453 block groups
    32768 blocks per group, 32768 fragments per group
    8192 inodes per group
    Superblock backups stored on blocks:
            32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
            4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
            102400000, 214990848

    Allocating group tables: done
    Writing inode tables: done
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done
  6. Remove reserved blocks (5% by default):
  7. # tune2fs -m 0 /dev/mapper/cryptodev1
    tune2fs 1.42.5 (29-Jul-2012)
    Setting reserved blocks percentage to 0% (0 blocks)
  8. Finally, mount a disk:
  9. # mount /dev/mapper/cryptodev1 /mnt
And that's basically it. When you want to use disk, and it is not mounted, then you first have to open crypted device (step 2) and then you mount newly created file system.

Tuesday, January 24, 2012

Kritpiranje diskova - razlozi za i neki pravni problemi...

Prilično je jasno kako je trend zadnjih nekoliko godina da prijenosna računala zamjenjuju stolna. Više je prednosti prijenosnih računala, ali imaju i jednu veliku manu. Puno lakše ih je izgubiti ili ukrasti nego što je to slučaj sa stolnim računalima. To samo po sebi ne bi predstavljalo veliki problem da se na diskovima ne nalaze tajni podaci koje, iz raznih razloga, ne bi željeli da vide drugi. To mogu biti osobni podaci, podaci tvrtke za koju radite i slično.

Sada će netko reći da je stavio/la lozinku te da je nemoguće pristupiti računalu bez da se prijavi na sustav. Dodatno, moguće je i da ste administratorski račun dodatno osigurali, primjerice, prilično dugačkom i jakom lozinkom. Problem s tom zaštitom je što ništa ne sprečava napadača da otvori računalo, izvadi disk, premjesti ga u drugo računalo kao dodatni disk te na tom drugom računalu pogleda sadržaj diska! To je jednostavno za napraviti i mora se uzeti u obzir kao vrlo vjerojatni slijed događaja.

Međutim, problem je, sam po sebi, relativno jednostavno riješiti. Treba uključiti enkripciju diskova. To drugim riječima znači da se cjelokupni sadržaj diska šifrira te se prilikom uključivanja računala traži da upišete tu šifru. Ako ne upišete šifru, ne možete pristupiti disku. Ne moram ni govoriti koliko je bitno da tu šifru ne zaboravite! Uglavnom, napadač sada - u slučaju da izvadi disk i stavi ga u drugo računalo - ne može do njegovog sadržaja, nema šifru! Ono što može je pokušati ju pogađati. Zbog toga je vrlo bitno da šifra bude dugačka i dobra. Najbolje da je to cijela rečenica zajedno s interpunkcijskim i numeričkim znakovima. U tom slučaju pogađanje i s najbržim računalnim klasterima može trajati godinama.

I da, ako koristite kriptirane diskove ne smijete hibernirati računalo jer se u tom slučaju hibernira otključani disk! Ako ste postavili lozinku na korisničko ime, tj. potrebno je pisati lozinku prije nego što se mora nastaviti s radom, sigurni ste koliko i ta lozinka. Postoje i dodatne metode uz pomoć kojih bi se moglo na hiberniranom računalu otkriti koja je lozinka koja otključava disk, ali to nije toliko trivijalno.

Što se tiče programske podrške za kritpiranje diskova, imaju ih skoro svi operacijski sustavi, a postoje i slobodno dostupni alati za kritpiranje diskova, primjerice, jedan od poznatijih je TrueCrypt. Prema tome, radi se o prilično pristupačnoj tehnologiji.

Sada dolazimo i do jednog velikog problema. Naime, što je s pitanjem kada kriminalac, ili jednostavno neka osoba koja je potencijalno prekršila zakon, koristi takvu zaštitu te policija ne može do podataka na računalu? Na žalost, mislim da je to kod nas siva zona, odnosno, da policija ne može ništa u tom slučaju. Volio bi kada bi me netko ispravio, ali mislim da je tako. Prije nekih godinu dana bio sam na jednom predavanju na kojemu je pričao jedan naš forenzičar, sudski vještak. Uglavnom, postavio sam pitanje što se dešava u takvim situacijama kada osumnjičeni ne želi otkriptirati disk te se pravi da je, primjerice, zaboravio lozinku. Na žalost, mislim da me uopće nije shvatio što pitam(!) tako da odgovor nisam dobio.

Danas sam naletio na vijest vezanu uz takvu situaciju u SAD-u. U tom slučaju, sudac je "naredio" optuženoj da omogući pristup svome prijenosnom računalu te je rekao da se (famozni) peti amandman ne odnosi na taj slučaj.

About Me

scientist, consultant, security specialist, networking guy, system administrator, philosopher ;)

Blog Archive