Merge pull request #3510 from wallabag/update-robotstxt

Updated robots.txt to prevent crawling
2017-12-16 09:02:40 +01:00
parent a68a80f654 e4ae6b976f
commit 9b606725fe
1 changed files with 1 additions and 3 deletions
--- a/web/robots.txt
+++ b/web/robots.txt
@ -1,4 +1,2 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
 User-agent: *
+Disallow: /