diff --git a/content/cheat-sheets/wget.md b/content/cheat-sheets/wget.md index 5585223..36ebd63 100644 --- a/content/cheat-sheets/wget.md +++ b/content/cheat-sheets/wget.md @@ -53,7 +53,7 @@ wget --mirror --no-host-directories --convert-links --adjust-extension --page-re The name of options are quite straightforward, and the shorten version of it is: `wget -mkEp -np ` -### ignoring robots.txt +### Ignoring robots.txt Sometimes, [robots.txt](https://en.wikipedia.org/wiki/Robots_exclusion_standard) forbids you the access to some resources. You can easily bypass this with the option `-e robots=off`.