Downloading a password protected file from a Web page using wget...

Today I was downloading some stuff from Web, and the download speeds were unbelievably slow (i.e. criminally slow)! The most I got was somewhere around few kilobytes per second. So, it's no wonder then that they lasted for a very long time, and some of them were reported as being finished, while actually only a part of the file was downloaded. To make things even more strange, ping reported RTT times that were within usual range of values when download speeds were much higher! Anyway, I didn't have a slightest clue what was happening. Usually, in those, cases, I use wget (or curl) to fetch a file, as they are more easily controlled and have much more features that the download mechanism embedded within Firefox. The problem was that, to access those files, I had to login and the login process isn't implemented using regular HTTP mechanism, but it's part of the Web application. Needless to say, wget (nor curl) don't know how to handle this situation.

The first thing that occurred to me is that wget supports loading of cookies from Firefox browser. So, I looked into wget manual page, and there it is an option --load-cookies. The only problem is that this option expect cookies to be stored in the old textual format, while Firefox starting from version 3 uses SQLite database. So, quick googling reveled several pages, of which I looked at this one. In short, that guy wrote a shell script that exports SQLite database into text file with a format expected by wget command. But reading comments, I found even better solution! It is a plugin called cliget. When you install this plugin it attaches itself to download dialog where it shows a wget command line you can use to download selected file from a terminal window. I tried it and it works great! So that solved my problems.

How to resume download in Firefox...

I started a large download in Firefox and it turned out to be very slow. Since I was downloading from Oracle, I was probably given "wrong" mirror to download from. So, I was thinking about stopping this particular download and resuming later but I didn't want to lose what I already had. The problem was that it seemd that the Firefox doesn't support that particular scenario, at least not without some fiddling with files.

Anyway, I found a tip on the Internet how to resume broken downloads which helped me manage this particular case. The algorithm is simple, first, go to the directory where partially downloaded files are and move them to some temporary location. Note that there are actually two files per download. One with the "right" name and the other with ".part" extension. Next, start download as usual, then, pause it immediately after download starts. It's important to pause it, not stop. Now, move saved files back to the download directory, overwriting existing files, and resume download. And that's it!

There is just one thing to note. In case you want to stop downloading and resume it later, don't press 'stop' button because it will remove everything that's partially downloaded. Use pause button instead.