forked from wallabag/wallabag
Updated robots.txt to prevent crawling
This commit is contained in:
@ -1,4 +1,2 @@
|
|||||||
# www.robotstxt.org/
|
|
||||||
# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
|
|
||||||
|
|
||||||
User-agent: *
|
User-agent: *
|
||||||
|
Disallow: /
|
||||||
|
|||||||
Reference in New Issue
Block a user