Mikaela
/
mikaela.github.io
mirror of https://github.com/mikaela/mikaela.github.io/
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

robots.txt: Disallow /txt/ and /php/ as they don't contain HTML noindex

This commit is contained in:
Aminda Suomalainen 2020-11-23 13:39:27 +02:00
parent c008068460
commit d340409002
Signed by: Mikaela
GPG Key ID: 99392F62BAE30723
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
robots.txt
View File

@ -2,5 +2,9 @@ User-agent: *
Allow: /
Allow: /$
Allow: /index.html
Disallow: /txt/
Disallow: /php/
Noindex: /txt/
Noindex: /php/
Sitemap: https://mikaela.info/sitemap.xml
Sitemap: https://mikaela.info/sitemaps.xml