mikaela.github.io/robots.txt

# robots.txt for https://mkaysi.github.io/
User-agent: *
Disallow: /pages/photos.html
Disallow: /pages/photos.html.md
Disallow: /pages/ext/
Disallow: /IRC/
Disallow: /template/
Some changes: * Restore whatever those google files were. * Add GA to index and identifying. * add photos page * add robots.txt * denies photos * denies pages/ext/ * what else? 2014-06-08 21:50:33 +02:00			`# robots.txt for https://mkaysi.github.io/`
			`User-agent: *`
			`Disallow: /pages/photos.html`
			`Disallow: /pages/photos.html.md`
robots.txt: `Disallow: /IRC/` it's symlink to pages/ . 2014-06-11 14:56:42 +02:00			`Disallow: /pages/ext/`
			`Disallow: /IRC/`
_config.yml & robots.txt: there is no reason to crawl my templates. 2014-06-15 11:45:05 +02:00			`Disallow: /template/`