controlled allowing of letting the bots in

This commit is contained in:
Aminda Suomalainen 2024-05-31 06:29:21 +03:00
parent 8ab675d87d
commit 9bda780c40
Signed by: Mikaela
SSH Key Fingerprint: SHA256:CXLULpqNBdUKB6E6fLA1b/4SzG0HvKD19PbIePU175Q
33 changed files with 76 additions and 47 deletions

View File

@ -12,7 +12,8 @@ baseurl: "" # the subpath of your site, e.g. /blog/
# At least it's just a static page.
#url: "https://www.aminda.eu" # the base hostname & protocol for your site
url: "https://aminda.eu" # the base hostname & protocol for your site
# Default meta-robots tags, can be set per-page
# Default meta-robots tags, can be set per-page. SET PER PAGE, DON'T ALLOW
# THEM HERE! Remember /n and less good blog posts etc.
robots: noai, nofollow, noindex
codeberg_username: Mikaela

View File

@ -5,6 +5,7 @@ title: "Little on my life currently"
category: [english]
tags: [irc, english, life]
redirect_from: /english/2015/03/25/leaving-bots-life.html
robots: noai
---
## A little on my life currently

View File

@ -8,6 +8,7 @@ redirect_from:
- /filter.html
- /weechat-filter.html
- /english/2015/05/31/weechat-filter.html
robots: noai
---
_Everyone knows the `/ignore` command, but there is also `/filter` which

View File

@ -7,6 +7,7 @@ tags: [english]
redirect_from:
- /ufw/
- /english/2015/06/12/ufw.html
robots: noai
---
_This post describes my UFW config and is here so I find it from somewhere

View File

@ -9,6 +9,7 @@ redirect_from:
- /IPv6/
- /english/2015/06/22/ipv6.html
lang: en
robots: noai
---
_There appears to be a lot of confusion on IPv6 and in this post I try to

View File

@ -6,6 +6,7 @@ category: [finnish]
tags: [finnish, translaki]
redirect_from: /finnish/2015/11/21/sahkopostia-lakivaliokunnalle-translaki.html
lang: fi
robots: noai
---
_Toivottavasti tämä viesti on edes jotenkuten järkevä, se olisi ollut

View File

@ -10,6 +10,7 @@ redirect_from:
- /openwireless_nomap.html
- /openwireless5_nomap.html
- /english/2015/11/29/SSID_optout_nomap.html
robots: noai
---
_Anxiety is now focusing to WLAN mapping & key sharing and how easy it is

View File

@ -6,6 +6,7 @@ category: [english]
tags: [IPv6, english]
redirect_from: /english/2015/12/25/asus-tunnelbroker-net.html
lang: en
robots: noai
---
_I am typing this, because there is a lot of misinformation on this,

View File

@ -5,6 +5,7 @@ title: "Post trans process, orchiectomy"
category: [english]
tags: [english, life, buddhism]
redirect_from: /english/2017/04/30/post-trans.html
robots: noai
---
_A little confusing flood of what my fingers brought up on the last past

View File

@ -8,6 +8,7 @@ redirect_from:
- /dns.html
- /dot.html
lang: en
robots: noai
---
_Since I first heard of Android 9 Private DNS I wondered how it will work

View File

@ -7,6 +7,7 @@ redirect_from:
- /matrix.html
- /element.html
lang: en
robots: noai
---
_Having used Matrix since 2016 and hearing about its greatness without any

View File

@ -4,6 +4,7 @@ title: "Leaving Privacy Guides team"
category: [english]
tags: [privacyguides]
lang: en
robots: noai
---
_PrivacyGuides.org was supposed to be the continuation of PrivacyTools.io

View File

@ -4,6 +4,7 @@ title: "Matrix abuse protection model for community maintainers: security by obs
category: [english]
tags: [matrix]
lang: en
robots: noai
---
_I am administrator or moderator in multiple communities in Matrix, the most sizable

View File

@ -3,6 +3,7 @@ layout: post
title: "bash/zsh functions for easier SSH signing and verification"
category: [english]
tags: [ssh]
robots: noai
---
_I have been using SSH signed git commits from 8 months and started signing things with my SSH key instead of PGP keys and thought to share how to do that more easily_

View File

@ -8,6 +8,7 @@ redirect_from:
- /httpseverywhere.html
- /https.html
lang: en
robots: noai
---
_I used to be sad since the EFF discontinued HTTPS Everywhere extension since the setting often didn't sync and it only applied to me as opposed to everyone using a shared computer. However since I have dived into browser policies, this is no longer an issue for me._

View File

@ -8,6 +8,7 @@ redirect_from:
- /ublockorigin.html
- /https2.html
lang: en
robots: noai
---
_I previously wrote about enforcing HTTPS for all users/profiles through browser policy receiving some positive feedback and I felt like continuing on the subject by instructing with extension installation. This barely scratches the surface of what browser policy can do for you either though._

View File

@ -5,6 +5,7 @@ navigation: true
permalink: /blog/
excerpt: "Blog index, posts in English and posts in Finnish — Blogin etusivu, postaukset englanniksi ja postaukset suomeksi."
lang: en
robots: noai
---
<!-- editorconfig-checker-disable -->

View File

@ -2,6 +2,7 @@
layout: index
title: Index
excerpt: "I am a Highly Sensitive Autistic Pirate with Linux experience since 2008. I am familiar with git and looking for employment."
robots: noai
---
<p id="avatar">

View File

@ -1,8 +1,9 @@
---
layout: mini
permalink: /n/coauthor.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# Git forge private emails

View File

@ -6,8 +6,9 @@ redirect_from:
- /n/rethink.html
- /n/esni.html
- /n/ech.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# DNS notes

View File

@ -1,8 +1,9 @@
---
layout: mini
permalink: /n/essentialsoftware.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# Essential software for new systems

View File

@ -2,8 +2,9 @@
layout: mini
permalink: /n/f-droid.html
redirect_from: /n/fdroid.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
F-Droid is kind of apt for Android with multiple repositories available. This

View File

@ -3,8 +3,9 @@ layout: mini
permalink: /n/fairbuds.html
redirect_from:
- /n/fairbudsxl.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# Fairbuds XL configuration

View File

@ -3,8 +3,9 @@ layout: mini
permalink: /n/firewalld.html
redirect_from:
- /n/firewall-cmd.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# Quick note on firewalld usage

View File

@ -13,7 +13,8 @@ redirect_from:
- /n/wkd.html
- n/keybase.html
lang: en
sitemap: false
sitemap: true
robots: noai
---
# Quick GPG notes where I may find them

View File

@ -2,8 +2,9 @@
layout: mini
permalink: /n/keyoxide.html
redirect_from: /n/keyoxide-flutter.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
> Keyoxide is a decentralized tool to create and verify decentralized online identities.

View File

@ -6,8 +6,9 @@ redirect_from:
- /locales.html
- /english/2016/10/16/adjusting-locales.html
- /blog/english/2016/10/16/adjusting-locales.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
# Quick notes on `localectl` settings

View File

@ -4,8 +4,9 @@ permalink: /n/matrixspoilers.html
redirect_from:
- n/nheko.html
- n/matrixspoiler.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
The Matrix specification supports spoilers since version 1.1:

View File

@ -1,8 +1,9 @@
---
layout: mini
permalink: /n/obtainium.html
sitemap: false
sitemap: true
lang: en
robots: noai
---
[Obtainium](https://github.com/ImranR98/Obtainium/) downloads apps directly from forge/whatever without need for app

View File

@ -6,6 +6,7 @@ redirect_from: /discuss/index.html
excerpt: "Introduction to my public instant messaging chats"
navigation: true
lang: en
robots: noai
---
I have multiple instant messaging chatrooms.

View File

@ -4,6 +4,7 @@ title: IRC
navigation: true
permalink: /irc/
excerpt: "Where do you find me at IRC and verify that it's me. Also includes my IRC related posts that are hopefully helpful."
robots: noai
---
IRC has been a big part of my life, I discovered it during junior high school

View File

@ -7,6 +7,7 @@ excerpt: "Matrix, a decentralised conversation store"
redirect_from:
- /matrix.html
lang: en
robots: noai
---
Just like [IRC](/irc/), _Matrix_ has became a part of my social life online. My

View File

@ -1,48 +1,15 @@
User-agent: *
Allow: /
Allow: /$
Allow: /index.html
Disallow: /blog/english/2015/
Disallow: /blog/english/2016/
Disallow: /blog/finnish/2015/
Disallow: /blog/finnish/2016/
Disallow: /blog/finnish/2017/
Disallow: /txt/
Disallow: /php/
Disallow: /r/
Disallow: /or/
Disallow: /n/
Noindex: /blog/english/2015/
Noindex: /blog/english/2016/
Noindex: /blog/finnish/2015/
Noindex: /blog/finnish/2016/
Noindex: /blog/finnish/2017/
Noindex: /txt/
Noindex: /php/
Noindex: /r/
Noindex: /or/
Noindex: /n/
Disallow: /ir/
Sitemap: https://aminda.eu/sitemap.xml
Sitemap: https://aminda.eu/sitemaps.xml
# These were previously part of darkvisitors but are not anymore for some reason?
User-agent: ChatGPT-User
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: omgilibot
Disallow: /
# https://darkvisitors.com/robots-txt-builder
# Dark Visitors robots.txt
# AI Data Scraper
# https://darkvisitors.com/agents/anthropic-ai
User-agent: anthropic-ai
Disallow: /
# Dark Visitors Robots.txt
# AI Data Scraper
# https://darkvisitors.com/agents/bytespider
@ -56,6 +23,18 @@ Disallow: /
User-agent: CCBot
Disallow: /
# AI Data Scraper
# https://darkvisitors.com/agents/claudebot
User-agent: ClaudeBot
Disallow: /
# AI Data Scraper
# https://darkvisitors.com/agents/diffbot
User-agent: Diffbot
Disallow: /
# AI Data Scraper
# https://darkvisitors.com/agents/facebookbot
@ -79,3 +58,21 @@ Disallow: /
User-agent: omgili
Disallow: /
# Undocumented AI Agent
# https://darkvisitors.com/agents/anthropic-ai
User-agent: anthropic-ai
Disallow: /
# Undocumented AI Agent
# https://darkvisitors.com/agents/claude-web
User-agent: Claude-Web
Disallow: /
# Undocumented AI Agent
# https://darkvisitors.com/agents/cohere-ai
User-agent: cohere-ai
Disallow: /