|
|
8e940ff288
|
add sitemaps.xml && update robots.txt & README
|
2015-05-02 10:34:18 +03:00 |
|
|
|
771c667c2a
|
r: redirects elsewhere
|
2015-05-01 10:23:02 +03:00 |
|
|
|
74951a8e88
|
robots.txt: add Tumblr sitemap
[SKIP CI]
|
2015-04-29 08:26:53 +03:00 |
|
|
|
5df02aed87
|
robots.txt: remove empty lines
Apparently some robots (or at least robots.txt validators) don't like
them.
|
2015-04-26 20:30:57 +03:00 |
|
|
|
083bfcc98a
|
robots.txt: be more strict
|
2015-04-26 20:24:28 +03:00 |
|
|
|
354683729f
|
fix robots.txt
I somehow misremembered that you Allow and Deny while in reality you
Allow and Disallow.
|
2015-04-26 20:10:47 +03:00 |
|
|
|
aec43c7c17
|
Does the sitemap really need to include 404 error?
|
2015-04-26 18:50:25 +03:00 |
|
|
|
0a2a8fb126
|
block bot & channel in sitemap.xml & robots.txt
|
2015-04-26 18:45:04 +03:00 |
|
|
|
c028d2d0de
|
fix 82f87bb41f
|
2015-02-06 08:46:58 +02:00 |
|
|
|
82f87bb41f
|
add robots.txt
|
2015-02-06 08:39:24 +02:00 |
|
