Commit Graph

4098 Commits

Author SHA1 Message Date
Valentin Lorentz 9ec4eb9563 irclib: Fix assertion failure on InspIRCd when Sigyn plugin is loaded 2022-04-08 12:44:42 +02:00
Valentin Lorentz 088d0f54dd irclib: Catch exception from outFilter, as with inFilter 2022-03-17 22:29:10 +01:00
Valentin Lorentz 862fca1602 conf: Use imports instead of sys.modules to detect module availability
`conf.supybot.databases()` may be called without any plugin supporting
sqlite3 being loaded yet, which causes `sqlite3` to be missing from
`sys.modules`; so it wouldn't be used by plugins loaded afterward.
2022-03-06 13:39:47 +01:00
Valentin Lorentz d58d8d4a71 conf: Prioritize sqlite3 over sqlalchemy in supybot.databases
It's usually a better choice for plugins that implement both
2022-03-06 13:39:41 +01:00
Valentin Lorentz 9e995d8ac2 conf: Remove 'sqlite' module from supybot.databases
It is wholy replaced by 'sqlite3', now
2022-03-06 13:39:19 +01:00
PeGaSuS 80b26c013d
Fix typo in conf.py (#1501)
Fixed typo that lead to a wrong variable name being pointed out
2022-03-06 12:30:02 +01:00
Valentin Lorentz 66d986e820 Web: Add overrides to support Youtube and Reddit; remove Twitter from tests. 2022-03-03 22:16:53 +01:00
Valentin Lorentz 78bdc469e4 htmlToText: Don't replace inline element tags with spaces
They are not meant to be displayed like this, so they look weird sometimes.
For example, Mastodon splits long links between spans, so the Fediverse plugin
always displayed them broken.
2022-02-19 21:00:51 +01:00
Valentin Lorentz bc76f7ead4 Fix utils.time.iana_timezone on python < 3.9 2022-02-17 12:18:13 +01:00
Valentin Lorentz 1643cb1b09 registry.Regexp: Better safeguard against misuses of _setValue() 2022-01-11 20:38:44 +01:00
Valentin Lorentz dcca9f9637 RSS: Make feed announces actually network-specific
.getSpecific() always returned the  channel-specific but non-network-specific var,
unless the channel-specific and network-specific one was manually set.
2022-01-11 20:13:39 +01:00
Valentin Lorentz 40155ede22 Fix syntax DeprecationWarning. 2022-01-09 14:10:15 +01:00
Valentin Lorentz 74f86654b1 Remove fallbacks for old Python versions. 2022-01-01 18:11:36 +01:00
Valentin Lorentz 3f77c53ed5 utils/structures: Fix typos, remove unused code 2021-12-19 20:53:11 +01:00
Valentin Lorentz e223b507dd utils.time: Add support for timezones without a slash.
eg. UTC, CET, ...
2021-12-07 23:30:16 +01:00
Valentin Lorentz ef0cd48bd1 registry: Better explain String.__str__ 2021-12-03 18:03:27 +01:00
Valentin Lorentz fba5ed5fa4 Fix crash caused by registry.Json.__str__ returning non-str objects 2021-12-03 18:02:14 +01:00
Valentin Lorentz baa8cda814 Fix bypassed cache invalidation checks, causing '@config reload' to be partially ignored
`registry.Value.__call__()` is a wrapper around access to
`registry.Value.value`, that checks if the value was set before the latest
call to `registry.open_registry`; and updates the `value` if needed.

When accessing `registry.Value.value` directly, this cache can't be
invalidated, causing the old value to still be used, until the next call
to `registry.Value.__call__()`.
2021-12-03 16:56:06 +01:00
Valentin Lorentz 2236c7552a irclib: Request draft/account-registration
Servers might reject the REGISTER command if it's not ACKed.
2021-11-17 23:26:06 +01:00
Valentin Lorentz feaa02ab6d Add docstring to supybot.utils.time.iana_timezone 2021-11-08 23:20:28 +01:00
Valentin Lorentz 12c526bc1c Update default user agent to mention Limnoria 2021-11-08 23:20:13 +01:00
Valentin Lorentz abe59022b4 irclib: Remove broken + useless code
This function does not exist anymore
2021-11-08 21:59:05 +01:00
Valentin Lorentz 8b26b675ba Use stdlib instead of pytz on Python >= 3.9
Python 3.9 introduced the zoneinfo module, which provides the only
feature we used pytz for (getting a datetime.tzinfo object from
an IANA timezone id); so let's use it instead of a third-party
dependency.
2021-11-08 21:46:48 +01:00
Valentin Lorentz 1f618b9773 log: Fix support of 'supybot-test --clean'
Broken in dff394df64.
2021-10-18 10:08:41 +02:00
Valentin Lorentz 63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz dff394df64 log: Add support for rotating messages.log. 2021-10-14 23:52:16 +02:00
Valentin Lorentz 666acd9f1f log: Simplify encoding in BetterFileHandler.
I don't think any these cases are useful anymore.

Let's keep the last one anyway, just in case I'm wrong.
2021-10-14 23:47:32 +02:00
Valentin Lorentz ebaa346619 callbacks: Don't raise AssertionError when addressd() is called with a NOTICE
Some plugins (like LinkRelay) share code between NOTICE and PRIVMSG
and they might call it unconditionally.
2021-09-22 21:48:59 +02:00
Valentin Lorentz de277edb42 irclib: Make requestCapabilities a public method
So it can be used by plugins like Sigyn.
2021-09-22 18:54:47 +02:00
Valentin Lorentz dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz 76bd1c89bf Socket: Gracefully handle invalid lines with UTF8ONLY
Just skip the line, instead of crashing the whole loop (which
contains the next lines from the same server too)
2021-09-13 18:45:49 +02:00
Valentin Lorentz ee9f0dc1bf STS: When persisting STS keys, use the actual port instead of the one from the policy
'Servers MAY send this key to securely connected clients, but it will be ignored.'
-- https://ircv3.net/specs/extensions/sts\#the-port-key
2021-09-03 20:15:18 +02:00
Valentin Lorentz 74073b2736 irclib: Log the server's error message on SASL failure. 2021-09-01 18:25:54 +02:00
Valentin Lorentz d593f84a66 Socket: disable charset heuristics when the server advertizes UTF8ONLY. 2021-08-27 19:03:18 +02:00
Valentin Lorentz db05081ff6 commands: fix _checkUrl 2021-08-25 23:28:25 +02:00
Valentin Lorentz 0131cd0691 commands: Add converters for IRIs
Since the 'url' and 'httpUrl' converters no longer support them...
2021-08-25 21:55:31 +02:00
Valentin Lorentz be88530fa4 commands: Disallow IRIs from 'url' and 'httpUrl' converters.
urllib doesn't support IRIs, and gives out a cryptic
'UnicodeEncodeError: 'ascii' codec can't encode character ...'
if we don't validate it.
2021-08-25 21:54:55 +02:00
Valentin Lorentz 62db3a92fc callbacks: Print unexpected commands 2021-08-25 21:29:19 +02:00
James Lu 09392478e6
getUrlFd: chain the original exception so that plugins can handle them further (#1487) 2021-08-01 13:35:45 -07:00
Valentin Lorentz db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz 5e2c2122aa Add as scram-sha-256 prefered SASL mechanism. 2021-07-31 11:40:28 +02:00
Valentin Lorentz 406369d53f Expose standardSubstitute's var dictionary
So it can be reused by plugins if they want to use a different templating
engine.
2021-07-21 09:45:28 +02:00
Valentin Lorentz 8fe517f48a irclib: Fix previous commits so RPL_NAMREPLY actually works
Turns out irclib parses ISUPPORT PREFIX properly, so it's not just
a raw string; but I didn't test the change live...
2021-07-16 22:53:38 +02:00
Valentin Lorentz 45c7615f4a irclib: Properly populate nickToHostmasks on RPL_NAMREPLY
It used to set the nick instead of the hostmask as values...
2021-07-16 19:53:25 +02:00
Valentin Lorentz d308329461 irclib: Fix support of power prefix chars in RPL_NAMREPLY
nickFromHostmask now (legitimately) complains when it's getting @ or !
at the beginning of a hostmask; so we need to strip them before passing
it to nickFromHostmask.

Then re-add them before calling c.addUser, because it uses them to
sort users in the right sets (ops/halfops/voices).

Additionally, this commit replaces the hardcoded set of prefix chars
(`@%+&~!`) with the one advertised in ISUPPORT when possible.
2021-07-16 19:50:13 +02:00
Valentin Lorentz 0f1011081e Socket: Fix cascading crash when Socket.run() crashes.
When a driver's run() method crashes, supybot.drivers.run() marks it
as dead and sets its 'irc' attribute to None.

This would be fine for "normal" independent drivers (like Socket used
to be), because this driver would never be called again.

But now that we use select(), some other thread may hold a reference
to this driver in a select() call frame, and call the dead driver's
'_read()' method when there is data to be read from the socket.

There is already a safeguard in '_read()' in the case the socket could
be read from, but this safeguard was missing from _handleSocketError.
This caused the "live" driver's select() to crash, which propagagated
to its run(), which caused the driver to be marked as dead, etc.

Eventually, all drivers could die, and we end up with the dreadful
"Schedule is the only remaining driver, why do we continue to live?"
in an infinite loop.
2021-07-14 23:55:31 +02:00
Valentin Lorentz e19282a2d3 Actually parse weird hostmasks like RFC1459 recommends
Thanks to @Noisytoot for pointing out the RFC actually specific them
with this:

<user>       ::= <nonwhite> { <nonwhite> }
2021-07-14 23:43:11 +02:00
Valentin Lorentz 5baf87ddba ircutils: Improve robustness when faced with invalid hostmasks
eg. @ in nicks, which happened on pissnet earlier today.
2021-07-14 23:25:11 +02:00
Aminda Suomalainen 1952dae6f7 src/conf.py: fix typo in supybot.reply.error.withNotice
Co-authored-by: MetaNova <MetaNova@users.noreply.github.com>
Co-authored-by: Val Lorentz <progval+github@progval.net>
2021-06-26 01:23:36 +02:00
Tim Gates 649048443e
Fix a few simple typos (#1476)
Closes #1475
2021-06-21 01:07:50 +02:00
Valentin Lorentz 65ab65cbb1 irclib: Fix crashes when ERROR is part of a batch. 2021-06-20 23:59:51 +02:00
Clark Boylan 67a39a3adb
Fix joins to many channels (#1473)
* Fix joins to many channels

If you have enough channels that the 512 byte message limit on the JOIN
message is hit then limnoria was losing the channel that put it over the
limit and not including it in the next JOIN message. This resulted in
losing one channel for every JOIN message that pushed us over 512 bytes.

We fix this by generating the JOIN message immediately after resetting
the channels list to ensure we include the channel that pushed us over
the limit. Then the next time through our JOIN msg construction we'll
add subsequent channels without forgetting the one that pushed us over.

* Add test for channel join lists

This adds a test for the issue that is fixed in the previous commit. We
ensure that when JOINs are split over multiple messages we JOIN to all
channels that were part of the input list and don't forget any of them.
2021-06-14 23:15:43 +02:00
Valentin Lorentz c8053dad54 Socket: Silence TLS warning for Tor hidden services.
They are already end-to-end encrypted and don't need TLS.
2021-06-12 21:03:15 +02:00
Valentin Lorentz 69c948bd5f irclib: fix _getTarget when to= is given and self.private=True 2021-06-08 21:56:42 +02:00
Valentin Lorentz 8a52902727 irclib: Fix overhead computation by using the real target computation algo 2021-06-08 21:56:08 +02:00
Valentin Lorentz 225c249ec2 irclib: Fix crash on SASL authentication failure 2021-06-05 15:50:05 +02:00
Valentin Lorentz aaca81c5b9 drivers: Fix log message when the STS policy uses the same port as configured. 2021-06-02 16:51:09 +02:00
Valentin Lorentz 43aada5b33 Store ignored hostmasks in Expiring HostmaskSet to prevent their pattern cache from expiring too soon
See e0fdcb67c0 for the rationale
(tl;dr: prevents triggering a degenerate case of the LRU cache when
there are over 1000 ignore masks)
2021-05-30 19:35:05 +02:00
Valentin Lorentz 698dfc2fca httpserver: Hide well-known callback from the index
Closes GH-1460.
2021-05-29 11:44:22 +02:00
Valentin Lorentz e0fdcb67c0 Store IrcUser hostmasks in HostmaskSet to prevent their pattern cache from expiring
When the number of hostmasks exceeds 1000 (the hardcoded size of
_patternCache and _hostmaskPatternEqualCache), this triggers
a pathological case in the LRU caches, that causes all calls to be
a cache miss.
This means that on every IRC message received, ircdb.checkIgnored triggers
a recompilation of *all* user hostmasks, which is very expensive
computationally.

This commit stores them in their own cache to prevent them from
expiring.
2021-05-27 22:15:52 +02:00
Valentin Lorentz a5cd870bd2 irclib: Refactor hostmaskPatternEqual to expose compileHostmaskPattern.
This clarifies the code a bit.

And I want to store compiled patterns in ircdb.User in a future
commit, so they can outlive the LRU cache.
2021-05-27 21:36:43 +02:00
Valentin Lorentz 711db72ad0 Socket: Fix crash non-TLS connections when using a SOCKS proxy 2021-05-27 01:08:33 +02:00
Valentin Lorentz 3bedc86479 Socket: Disable resolution when using a SOCKS proxy
1. Avoids leaking DNS queries
2. necessary for onion services
2021-05-27 00:08:02 +02:00
Valentin Lorentz cca1156b90 drivers: Log SOCKS proxy on connecting 2021-05-26 23:43:23 +02:00
Valentin Lorentz c3f39fc93b callbacks, Misc, Relay: Fix when the message prefix is just a nick
<user> and <host> are optional in https://datatracker.ietf.org/doc/html/rfc1459#section-2.3.1

Closes GH-1451.
2021-05-26 17:50:29 +02:00
Valentin Lorentz e59e0f6908 irclib: Fix target computation
It mistakenly used the bot's nick as target when the message is in private,
so 'more' after a private message always answered the user did not send
a command before (because said command was attributed to the bot)
2021-05-26 00:30:27 +02:00
Valentin Lorentz 710d16f301 Admin: Fix python 3.4 support. 2021-05-25 19:24:21 +02:00
Aminda Suomalainen fa3707ed66 src/conf.py: add :port to explanation of networks.X.socksproxy 2021-05-25 18:58:35 +02:00
Valentin Lorentz 1455a83391 Merge branch 'nopassword' into testing 2021-05-25 00:12:36 +02:00
Valentin Lorentz d91303271d ircdb: Fix 'IrcUser.nicks' on non-lowercase network names
This dict was filled with IrcString keys, which is hashed
as lowercase, so when queried with a non-lowercase string,
the key would not be found, and lead to very confusing errors.
2021-05-24 17:21:57 +02:00
Valentin Lorentz 742f4f825d Add the option to disable password login.
Actually resolves GH-1140, finally.
2021-05-23 15:33:38 +02:00
Valentin Lorentz 8559ea9848 httpserver: Fix pointless tracebacks
because doGet does not accept the 'form' argument
2021-05-17 18:24:21 +02:00
Valentin Lorentz db99332079 callback: Fix AssertionError when action is given with implicit noLengthCheck 2021-05-07 20:10:57 +02:00
Valentin Lorentz 177c20267c irclib: Fix crash caused by 4d2efc65ef.
dynamicScope inspects the stack looking for this variable, so we need to keep it.
2021-04-24 21:21:57 +02:00
David Macek 151fcf8caf Don't throttle when delay is exactly equal
If throttle time is set to 0.0 (e.g. in tests) and the reported time
since last dequeue is also 0.0 (maybe because the timer resolution is
too low), takeMsg would throttle.  This would make tests fail because
they expect messages immediately.

This issue was observed on Windows 10 20H2 with both a mingw-w64 Python
v3.8.9 from MSYS2 and several version of official Python v3.x.
Confirmed at least as far back as
aa98d987a7.

Since it's a non-integer comparison, the change shouldn't have any
observable effect in normal operation.
2021-04-24 21:04:42 +02:00
David Macek dce969e4b9 tests: Pass setUp arguments
ChannelPluginTestCase#setUp accepts the same parameters as its
super-method PluginTestCase#setUp, but it didn't pass the arguments down
the line.  Now it does.
2021-04-24 21:03:48 +02:00
David Macek 4d2efc65ef Remove some unused names 2021-04-24 21:01:40 +02:00
David Macek e740953068 Fix some typos 2021-04-24 21:01:07 +02:00
Valentin Lorentz c7d85e73d7 httpserver, Fediverse: show an error message on 404 2021-04-18 16:25:37 +02:00
Valentin Lorentz 3c1c4a69e9 callbacks: Make snarfers' output paginatable with @more
by moving the 'smart' reply() method from NestedCommandsIrcProxy
to ReplyIrcProxy.

There is no reason only commands should have a paginated output
and not snarfers defined in PluginRegexp.
2021-04-16 23:38:44 +02:00
Valentin Lorentz 24ca278b93 Reorganize the kwargs of _sendReply
I think it's more readable and modular this way
2021-04-16 23:11:02 +02:00
Valentin Lorentz ab3dbcfecd Fix computation of message overhead in multiline batches. 2021-04-16 23:09:01 +02:00
Valentin Lorentz 6b1460e160 Log loading times for each plugin
It makes it easier to debug long startup times.
2021-04-11 16:35:42 +02:00
Valentin Lorentz fd873fa0a5 httpserver: Don't serve_forever when documenting
1. it binds the port, which is useless while documenting
2. in some circumstances, it prevents supybot-plugin-doc from exiting
   (because the server threads are still running)
2021-04-11 13:24:10 +02:00
Valentin Lorentz 9869136289 src/setup: Add support for ReST readmes by default 2021-04-11 10:38:05 +02:00
Valentin Lorentz 4fc6580a09 callbacks: Document replySuccess() and error() 2021-04-05 22:47:44 +02:00
Valentin Lorentz f11684a1cf callbacks: Add/improve method docstrings 2021-04-05 20:34:49 +02:00
Valentin Lorentz 9323302704 Socket: Recover from socks.GeneralProxyError exceptions. 2021-04-05 13:41:29 +02:00
Valentin Lorentz c28d517d6f irclib: Copy messages before altering them at all
Fixes a bug triggered by Relay when it relays to a network that supports
labeled-response and an other that doesn't
2021-04-02 20:54:21 +02:00
Valentin Lorentz b030a7bb76 callbacks: Fix py 3.4 support 2021-04-01 20:10:17 +02:00
Valentin Lorentz d452a7051c callbacks: Don't write nick prefixes in the middle of multiline messages. 2021-04-01 20:10:17 +02:00
Valentin Lorentz 41c5589bea utils.str: Rely only on the documented methods of textwrap.TextWrapper.
'_split_chunks()' should be fine, but '_wrap_chunks()' is the only one explicitly
documented as overridable, so it's probably safer to use.
2021-04-01 20:10:17 +02:00
Valentin Lorentz bbc2e9de0d irclib: 'lift' message tags to the batch when sending a multiline batch 2021-04-01 20:10:17 +02:00
Valentin Lorentz 92399bb6a7 drivers: Don't strip all whitespaces before parsing messages.
They matter in multiline messages, and there isn't much point to
stripping them anyway.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 7cb3ae12da Misc: Make @more reply in batches when possible. 2021-04-01 20:10:17 +02:00
Valentin Lorentz bf90a7c94d callbacks: Make reply() send multiline batches when supybot.reply.mores.instant > 1 2021-04-01 20:10:17 +02:00
Valentin Lorentz d5d22985f5 irclib: Remove special-casing of the last instant message in _sendReply
There is no reason for it to be special; and this special-casing would be
annoying when we add support for outgoing multiline batches.
2021-04-01 20:10:17 +02:00
Valentin Lorentz de29218bdb irclib: Make NestedCommandsIrcProxy._replyOverhead count in bytes instead of chars
It was, once again, a bug to count characters, because they
might contain multi-byte characters, and truncation happens
after the 512th byte.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 8a3efe4379 irclib: Split _sendReply from NestedCommandsIrcProxy.reply
This function was getting uncomfortably big.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 567618392f Add tests + fix bugs 2021-04-01 20:10:17 +02:00