Most of Linux users are using wget from time to time, sometimes even when they don’t know about it – many GUI download managers for Linux are based on wget. Anyways wget is command line tool for downloading files over HTTP, HTTPs and FTP protocols within single session. It works like a charm with default settings for downloading simple static files, at the same time if content is protected by cookies and/or referrer then wget may seem useless but it’s actually not.
For example, imagine you need to download some time from restricted area of website: it requires to enter username/password to access it and then stores cookies with session id in browsers cache. When that cookies file is in place you’ll be permitted to download files but once cookies file is missing you’ll get username/password prompt to establish new session. In regards of wget such attempt to download file without cookies will result in html page with login page layout inside file instead of its actual contents.
There is simple solution for this. First, you need to establish session with wget, save resulting cookies file somewhere and then load that cookies to download files from restricted area:
1.
wget --post-data="username=admin&pwd=yourpassword" --cookies=on --keep-session-cookies --save-cookies=cookie.txt "http://www.someserver.com/login.php" -O
2.
wget --referer="http://www.someserver.com/login.php" --cookies=on --keep-session-cookies --load-cookies=cookie.txt http://www.someserver.com/somefile.iso
Key --referer is optional but most probably you’ll need it besides cookies to get access to content protected by username/password.
I had been using this wget’s functionality inside the script that establishes session to Cacti and creates graphs for dozens of hosts. Without wget cookies support I would have need to create those graphs by hands (that sucks).
