Mikaela/mikaela.github.io
1
0
Fork 0
mirror of https://github.com/mikaela/mikaela.github.io/ synced 2024-11-22 20:19:32 +01:00
Code Issues Projects Releases Wiki Activity

robots.txt: disallow/noindex blog posts pre 2018, r/, or/ & n/

Browse Source 
Yes, I know Noindex is deprecated
This commit is contained in:
Aminda Suomalainen 2022-11-10 12:20:08 +02:00
parent 9700a2c3a4
commit 4e97e1679c
Signed by: Mikaela
SSH Key Fingerprint: SHA256:CXLULpqNBdUKB6E6fLA1b/4SzG0HvKD19PbIePU175Q
1 changed files with 16 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
robots.txt
View File

@ -2,9 +2,25 @@ User-agent: *
Allow: / Allow: /
Allow: /$ Allow: /$
Allow: /index.html Allow: /index.html
Disallow: /blog/english/2015/
Disallow: /blog/english/2016/
Disallow: /blog/finnish/2015/
Disallow: /blog/finnish/2016/
Disallow: /blog/finnish/2017/
Disallow: /txt/ Disallow: /txt/
Disallow: /php/ Disallow: /php/
Disallow: /r/
Disallow: /or/
Disallow: /n/
Noindex: /blog/english/2015/
Noindex: /blog/english/2016/
Noindex: /blog/finnish/2015/
Noindex: /blog/finnish/2016/
Noindex: /blog/finnish/2017/
Noindex: /txt/ Noindex: /txt/
Noindex: /php/ Noindex: /php/
Noindex: /r/
Noindex: /or/
Noindex: /n/
Sitemap: https://www.aminda.eu/sitemap.xml Sitemap: https://www.aminda.eu/sitemap.xml
Sitemap: https://www.aminda.eu/sitemaps.xml Sitemap: https://www.aminda.eu/sitemaps.xml