wget usage tutorial, whole site download, website mirror, etc.

Try the naming method of UC shock department

Let's go back to the normal title.

This command, if you are smart enough, should have guessed that it is wget. This command is very familiar to us and we usually use it for testing. In fact, it is overkill and wget is extremely powerful. Now let’s get to know wget.

Baidu Encyclopedia's explanation of wet: wget is a free tool that automatically downloads files from the Internet. It supports downloading through the three most common TCP/IP protocols: HTTP, HTTPS, and FTP, and can use HTTP proxy. The origin of the name wget is the combination of "World Wide Web" and "get".

The explanation is very superficial. The advantage of wget is that it is stable. If the download fails due to network reasons, wget will continue to try until the entire file is downloaded; if the server interrupts the download process, it will connect to the server again and stop. to continue downloading. In general, download without hesitation.

The syntax of wget is as follows:

wget(options)(parameters)

Here are some examples

Download a single file

wget http://cachefly.cachefly.net/100mb.test

If you want to save it under a different name, you can do it like this, and the file will be saved as 138vps.test

wget -O 138vps.test http://cachefly.cachefly.net/100mb.test

Because wget will by default be named after the last character that matches /, so if our download address is something like this [http://xx.com/?id=138vps] If you don’t rename it, the name of the file you downloaded It’s [id=138vps], which is obviously unscientific.

wget speed limited download

We don’t want wget to eat up our bandwidth when downloading, especially when we download very large files, so speed limiting downloads is a must.

wget --limit-rate=300k http://cachefly.cachefly.net/100mb.test

Use wget to resume the download

This is very practical. Due to uncontrollable factors, SSH was disconnected when we were using wget, so we definitely hope to be able to resume the download instead of starting from 0, especially for large files.

wget -c http://cachefly.cachefly.net/100mb.test

Use wget background download

When we download large files, you definitely don’t want to keep staring at the progress bar and do something else. At this time, it is very important to use background downloading.

wget -b http://cachefly.cachefly.net/100mb.test

If you want to know whether the download is complete, just use this command to check. If you want to exit, just [ctrl+c]

tail -f wget-log

Fake browser download

Some websites will not let you download if they find that you are not using a browser, so you know

wget --user-agent="Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16" http://cachefly.cachefly. net/100mb.test

Download multiple files simultaneously

What if I want to download multiple files at the same time and don’t want to manually wget them one by one? It's very simple. Put these URLs into a text file. Assume the file name is [138vps.txt], one address per line and one command to get it done.

wget -i 138vps.txt

Download ftp

In fact, wget supports direct downloading of ftp. If it is anonymous, then

wgetftpurl

If a username and password are required, then

wget --ftp-user=USERNAME --ftp-password=PASSWORD ftpurl

Mirror website

What kind of whole-site downloader is needed with wget? Just download the entire website with wget.

wget --mirror -p --convert-links -P ./ http://soft.138vps.com

--miror starts mirror download.

--convert-links After downloading, convert it to a local link.

-P Save all files and directories to the specified local directory.

-r Recursive download, download all files in a directory (including subdirectories) of the specified web page

-np does not search the upper directory when downloading recursively

-L does not enter other hosts during recursion. If the host you download has links to other websites, this will be very troublesome.

If you just want to download images:

wget -r -A.jpg,.png,.gif http://soft.138vps.com

If you just don't want to download pictures

wget --reject=jpg http://soft.138vps.com

Conclusion

When wget downloads some resources, you must use the option --no-check-certificate, otherwise it will prompt that downloading is not allowed without certification.

Such as https.

wget --no-check-certificate url

Recommended site search: Korean server recommendation, Hong Kong vps host, dynamic ip server, foreign server rental, cpanel host, free server website, registration number, ip online agent free virtual host, foreign trade host,

p>