Mikaela/mikaela.github.io
1
0
Fork 0
mirror of https://github.com/mikaela/mikaela.github.io/ synced 2024-11-25 13:59:25 +01:00
Code Issues Projects Releases Wiki Activity

block bot & channel in sitemap.xml & robots.txt

Browse Source
This commit is contained in:
Aminda Suomalainen 2015-04-26 18:45:04 +03:00
parent a4048c4d7e
commit 0a2a8fb126
2 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
_config.yml
View File

@ -26,4 +26,9 @@ gems:
- jekyll-sitemap
sitemap:
file: "/sitemap.xml"
exclude:
# Even if you can see the existence of these pages here, I don't
# wish to get search engine attention to them.
- "/bot.html"
- "/channel.html"
include: [PGP, PHP, keybase.txt, robots.txt]

5
robots.txt
View File

@ -1,4 +1,9 @@
User-agent: *
Allow: /
# I prefer search engines to not advertise these pages even if you can find
# them here or in `_config.yml`.
Deny: /bot.html
Deny: /channel.html
Sitemap: https://mikaela.info/sitemap.xml