Commit Graph

4078 Commits

Author SHA1 Message Date
Valentin Lorentz 12c526bc1c Update default user agent to mention Limnoria 2021-11-08 23:20:13 +01:00
Valentin Lorentz abe59022b4 irclib: Remove broken + useless code
This function does not exist anymore
2021-11-08 21:59:05 +01:00
Valentin Lorentz 8b26b675ba Use stdlib instead of pytz on Python >= 3.9
Python 3.9 introduced the zoneinfo module, which provides the only
feature we used pytz for (getting a datetime.tzinfo object from
an IANA timezone id); so let's use it instead of a third-party
dependency.
2021-11-08 21:46:48 +01:00
Valentin Lorentz 1f618b9773 log: Fix support of 'supybot-test --clean'
Broken in dff394df64.
2021-10-18 10:08:41 +02:00
Valentin Lorentz 63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz dff394df64 log: Add support for rotating messages.log. 2021-10-14 23:52:16 +02:00
Valentin Lorentz 666acd9f1f log: Simplify encoding in BetterFileHandler.
I don't think any these cases are useful anymore.

Let's keep the last one anyway, just in case I'm wrong.
2021-10-14 23:47:32 +02:00
Valentin Lorentz ebaa346619 callbacks: Don't raise AssertionError when addressd() is called with a NOTICE
Some plugins (like LinkRelay) share code between NOTICE and PRIVMSG
and they might call it unconditionally.
2021-09-22 21:48:59 +02:00
Valentin Lorentz de277edb42 irclib: Make requestCapabilities a public method
So it can be used by plugins like Sigyn.
2021-09-22 18:54:47 +02:00
Valentin Lorentz dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz 76bd1c89bf Socket: Gracefully handle invalid lines with UTF8ONLY
Just skip the line, instead of crashing the whole loop (which
contains the next lines from the same server too)
2021-09-13 18:45:49 +02:00
Valentin Lorentz ee9f0dc1bf STS: When persisting STS keys, use the actual port instead of the one from the policy
'Servers MAY send this key to securely connected clients, but it will be ignored.'
-- https://ircv3.net/specs/extensions/sts\#the-port-key
2021-09-03 20:15:18 +02:00
Valentin Lorentz 74073b2736 irclib: Log the server's error message on SASL failure. 2021-09-01 18:25:54 +02:00
Valentin Lorentz d593f84a66 Socket: disable charset heuristics when the server advertizes UTF8ONLY. 2021-08-27 19:03:18 +02:00
Valentin Lorentz db05081ff6 commands: fix _checkUrl 2021-08-25 23:28:25 +02:00
Valentin Lorentz 0131cd0691 commands: Add converters for IRIs
Since the 'url' and 'httpUrl' converters no longer support them...
2021-08-25 21:55:31 +02:00
Valentin Lorentz be88530fa4 commands: Disallow IRIs from 'url' and 'httpUrl' converters.
urllib doesn't support IRIs, and gives out a cryptic
'UnicodeEncodeError: 'ascii' codec can't encode character ...'
if we don't validate it.
2021-08-25 21:54:55 +02:00
Valentin Lorentz 62db3a92fc callbacks: Print unexpected commands 2021-08-25 21:29:19 +02:00
James Lu 09392478e6
getUrlFd: chain the original exception so that plugins can handle them further (#1487) 2021-08-01 13:35:45 -07:00
Valentin Lorentz db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz 5e2c2122aa Add as scram-sha-256 prefered SASL mechanism. 2021-07-31 11:40:28 +02:00
Valentin Lorentz 406369d53f Expose standardSubstitute's var dictionary
So it can be reused by plugins if they want to use a different templating
engine.
2021-07-21 09:45:28 +02:00
Valentin Lorentz 8fe517f48a irclib: Fix previous commits so RPL_NAMREPLY actually works
Turns out irclib parses ISUPPORT PREFIX properly, so it's not just
a raw string; but I didn't test the change live...
2021-07-16 22:53:38 +02:00
Valentin Lorentz 45c7615f4a irclib: Properly populate nickToHostmasks on RPL_NAMREPLY
It used to set the nick instead of the hostmask as values...
2021-07-16 19:53:25 +02:00
Valentin Lorentz d308329461 irclib: Fix support of power prefix chars in RPL_NAMREPLY
nickFromHostmask now (legitimately) complains when it's getting @ or !
at the beginning of a hostmask; so we need to strip them before passing
it to nickFromHostmask.

Then re-add them before calling c.addUser, because it uses them to
sort users in the right sets (ops/halfops/voices).

Additionally, this commit replaces the hardcoded set of prefix chars
(`@%+&~!`) with the one advertised in ISUPPORT when possible.
2021-07-16 19:50:13 +02:00
Valentin Lorentz 0f1011081e Socket: Fix cascading crash when Socket.run() crashes.
When a driver's run() method crashes, supybot.drivers.run() marks it
as dead and sets its 'irc' attribute to None.

This would be fine for "normal" independent drivers (like Socket used
to be), because this driver would never be called again.

But now that we use select(), some other thread may hold a reference
to this driver in a select() call frame, and call the dead driver's
'_read()' method when there is data to be read from the socket.

There is already a safeguard in '_read()' in the case the socket could
be read from, but this safeguard was missing from _handleSocketError.
This caused the "live" driver's select() to crash, which propagagated
to its run(), which caused the driver to be marked as dead, etc.

Eventually, all drivers could die, and we end up with the dreadful
"Schedule is the only remaining driver, why do we continue to live?"
in an infinite loop.
2021-07-14 23:55:31 +02:00
Valentin Lorentz e19282a2d3 Actually parse weird hostmasks like RFC1459 recommends
Thanks to @Noisytoot for pointing out the RFC actually specific them
with this:

<user>       ::= <nonwhite> { <nonwhite> }
2021-07-14 23:43:11 +02:00
Valentin Lorentz 5baf87ddba ircutils: Improve robustness when faced with invalid hostmasks
eg. @ in nicks, which happened on pissnet earlier today.
2021-07-14 23:25:11 +02:00
Aminda Suomalainen 1952dae6f7 src/conf.py: fix typo in supybot.reply.error.withNotice
Co-authored-by: MetaNova <MetaNova@users.noreply.github.com>
Co-authored-by: Val Lorentz <progval+github@progval.net>
2021-06-26 01:23:36 +02:00
Tim Gates 649048443e
Fix a few simple typos (#1476)
Closes #1475
2021-06-21 01:07:50 +02:00
Valentin Lorentz 65ab65cbb1 irclib: Fix crashes when ERROR is part of a batch. 2021-06-20 23:59:51 +02:00
Clark Boylan 67a39a3adb
Fix joins to many channels (#1473)
* Fix joins to many channels

If you have enough channels that the 512 byte message limit on the JOIN
message is hit then limnoria was losing the channel that put it over the
limit and not including it in the next JOIN message. This resulted in
losing one channel for every JOIN message that pushed us over 512 bytes.

We fix this by generating the JOIN message immediately after resetting
the channels list to ensure we include the channel that pushed us over
the limit. Then the next time through our JOIN msg construction we'll
add subsequent channels without forgetting the one that pushed us over.

* Add test for channel join lists

This adds a test for the issue that is fixed in the previous commit. We
ensure that when JOINs are split over multiple messages we JOIN to all
channels that were part of the input list and don't forget any of them.
2021-06-14 23:15:43 +02:00
Valentin Lorentz c8053dad54 Socket: Silence TLS warning for Tor hidden services.
They are already end-to-end encrypted and don't need TLS.
2021-06-12 21:03:15 +02:00
Valentin Lorentz 69c948bd5f irclib: fix _getTarget when to= is given and self.private=True 2021-06-08 21:56:42 +02:00
Valentin Lorentz 8a52902727 irclib: Fix overhead computation by using the real target computation algo 2021-06-08 21:56:08 +02:00
Valentin Lorentz 225c249ec2 irclib: Fix crash on SASL authentication failure 2021-06-05 15:50:05 +02:00
Valentin Lorentz aaca81c5b9 drivers: Fix log message when the STS policy uses the same port as configured. 2021-06-02 16:51:09 +02:00
Valentin Lorentz 43aada5b33 Store ignored hostmasks in Expiring HostmaskSet to prevent their pattern cache from expiring too soon
See e0fdcb67c0 for the rationale
(tl;dr: prevents triggering a degenerate case of the LRU cache when
there are over 1000 ignore masks)
2021-05-30 19:35:05 +02:00
Valentin Lorentz 698dfc2fca httpserver: Hide well-known callback from the index
Closes GH-1460.
2021-05-29 11:44:22 +02:00
Valentin Lorentz e0fdcb67c0 Store IrcUser hostmasks in HostmaskSet to prevent their pattern cache from expiring
When the number of hostmasks exceeds 1000 (the hardcoded size of
_patternCache and _hostmaskPatternEqualCache), this triggers
a pathological case in the LRU caches, that causes all calls to be
a cache miss.
This means that on every IRC message received, ircdb.checkIgnored triggers
a recompilation of *all* user hostmasks, which is very expensive
computationally.

This commit stores them in their own cache to prevent them from
expiring.
2021-05-27 22:15:52 +02:00
Valentin Lorentz a5cd870bd2 irclib: Refactor hostmaskPatternEqual to expose compileHostmaskPattern.
This clarifies the code a bit.

And I want to store compiled patterns in ircdb.User in a future
commit, so they can outlive the LRU cache.
2021-05-27 21:36:43 +02:00
Valentin Lorentz 711db72ad0 Socket: Fix crash non-TLS connections when using a SOCKS proxy 2021-05-27 01:08:33 +02:00
Valentin Lorentz 3bedc86479 Socket: Disable resolution when using a SOCKS proxy
1. Avoids leaking DNS queries
2. necessary for onion services
2021-05-27 00:08:02 +02:00
Valentin Lorentz cca1156b90 drivers: Log SOCKS proxy on connecting 2021-05-26 23:43:23 +02:00
Valentin Lorentz c3f39fc93b callbacks, Misc, Relay: Fix when the message prefix is just a nick
<user> and <host> are optional in https://datatracker.ietf.org/doc/html/rfc1459#section-2.3.1

Closes GH-1451.
2021-05-26 17:50:29 +02:00
Valentin Lorentz e59e0f6908 irclib: Fix target computation
It mistakenly used the bot's nick as target when the message is in private,
so 'more' after a private message always answered the user did not send
a command before (because said command was attributed to the bot)
2021-05-26 00:30:27 +02:00
Valentin Lorentz 710d16f301 Admin: Fix python 3.4 support. 2021-05-25 19:24:21 +02:00
Aminda Suomalainen fa3707ed66 src/conf.py: add :port to explanation of networks.X.socksproxy 2021-05-25 18:58:35 +02:00
Valentin Lorentz 1455a83391 Merge branch 'nopassword' into testing 2021-05-25 00:12:36 +02:00
Valentin Lorentz d91303271d ircdb: Fix 'IrcUser.nicks' on non-lowercase network names
This dict was filled with IrcString keys, which is hashed
as lowercase, so when queried with a non-lowercase string,
the key would not be found, and lead to very confusing errors.
2021-05-24 17:21:57 +02:00
Valentin Lorentz 742f4f825d Add the option to disable password login.
Actually resolves GH-1140, finally.
2021-05-23 15:33:38 +02:00
Valentin Lorentz 8559ea9848 httpserver: Fix pointless tracebacks
because doGet does not accept the 'form' argument
2021-05-17 18:24:21 +02:00
Valentin Lorentz db99332079 callback: Fix AssertionError when action is given with implicit noLengthCheck 2021-05-07 20:10:57 +02:00
Valentin Lorentz 177c20267c irclib: Fix crash caused by 4d2efc65ef.
dynamicScope inspects the stack looking for this variable, so we need to keep it.
2021-04-24 21:21:57 +02:00
David Macek 151fcf8caf Don't throttle when delay is exactly equal
If throttle time is set to 0.0 (e.g. in tests) and the reported time
since last dequeue is also 0.0 (maybe because the timer resolution is
too low), takeMsg would throttle.  This would make tests fail because
they expect messages immediately.

This issue was observed on Windows 10 20H2 with both a mingw-w64 Python
v3.8.9 from MSYS2 and several version of official Python v3.x.
Confirmed at least as far back as
aa98d987a7.

Since it's a non-integer comparison, the change shouldn't have any
observable effect in normal operation.
2021-04-24 21:04:42 +02:00
David Macek dce969e4b9 tests: Pass setUp arguments
ChannelPluginTestCase#setUp accepts the same parameters as its
super-method PluginTestCase#setUp, but it didn't pass the arguments down
the line.  Now it does.
2021-04-24 21:03:48 +02:00
David Macek 4d2efc65ef Remove some unused names 2021-04-24 21:01:40 +02:00
David Macek e740953068 Fix some typos 2021-04-24 21:01:07 +02:00
Valentin Lorentz c7d85e73d7 httpserver, Fediverse: show an error message on 404 2021-04-18 16:25:37 +02:00
Valentin Lorentz 3c1c4a69e9 callbacks: Make snarfers' output paginatable with @more
by moving the 'smart' reply() method from NestedCommandsIrcProxy
to ReplyIrcProxy.

There is no reason only commands should have a paginated output
and not snarfers defined in PluginRegexp.
2021-04-16 23:38:44 +02:00
Valentin Lorentz 24ca278b93 Reorganize the kwargs of _sendReply
I think it's more readable and modular this way
2021-04-16 23:11:02 +02:00
Valentin Lorentz ab3dbcfecd Fix computation of message overhead in multiline batches. 2021-04-16 23:09:01 +02:00
Valentin Lorentz 6b1460e160 Log loading times for each plugin
It makes it easier to debug long startup times.
2021-04-11 16:35:42 +02:00
Valentin Lorentz fd873fa0a5 httpserver: Don't serve_forever when documenting
1. it binds the port, which is useless while documenting
2. in some circumstances, it prevents supybot-plugin-doc from exiting
   (because the server threads are still running)
2021-04-11 13:24:10 +02:00
Valentin Lorentz 9869136289 src/setup: Add support for ReST readmes by default 2021-04-11 10:38:05 +02:00
Valentin Lorentz 4fc6580a09 callbacks: Document replySuccess() and error() 2021-04-05 22:47:44 +02:00
Valentin Lorentz f11684a1cf callbacks: Add/improve method docstrings 2021-04-05 20:34:49 +02:00
Valentin Lorentz 9323302704 Socket: Recover from socks.GeneralProxyError exceptions. 2021-04-05 13:41:29 +02:00
Valentin Lorentz c28d517d6f irclib: Copy messages before altering them at all
Fixes a bug triggered by Relay when it relays to a network that supports
labeled-response and an other that doesn't
2021-04-02 20:54:21 +02:00
Valentin Lorentz b030a7bb76 callbacks: Fix py 3.4 support 2021-04-01 20:10:17 +02:00
Valentin Lorentz d452a7051c callbacks: Don't write nick prefixes in the middle of multiline messages. 2021-04-01 20:10:17 +02:00
Valentin Lorentz 41c5589bea utils.str: Rely only on the documented methods of textwrap.TextWrapper.
'_split_chunks()' should be fine, but '_wrap_chunks()' is the only one explicitly
documented as overridable, so it's probably safer to use.
2021-04-01 20:10:17 +02:00
Valentin Lorentz bbc2e9de0d irclib: 'lift' message tags to the batch when sending a multiline batch 2021-04-01 20:10:17 +02:00
Valentin Lorentz 92399bb6a7 drivers: Don't strip all whitespaces before parsing messages.
They matter in multiline messages, and there isn't much point to
stripping them anyway.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 7cb3ae12da Misc: Make @more reply in batches when possible. 2021-04-01 20:10:17 +02:00
Valentin Lorentz bf90a7c94d callbacks: Make reply() send multiline batches when supybot.reply.mores.instant > 1 2021-04-01 20:10:17 +02:00
Valentin Lorentz d5d22985f5 irclib: Remove special-casing of the last instant message in _sendReply
There is no reason for it to be special; and this special-casing would be
annoying when we add support for outgoing multiline batches.
2021-04-01 20:10:17 +02:00
Valentin Lorentz de29218bdb irclib: Make NestedCommandsIrcProxy._replyOverhead count in bytes instead of chars
It was, once again, a bug to count characters, because they
might contain multi-byte characters, and truncation happens
after the 512th byte.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 8a3efe4379 irclib: Split _sendReply from NestedCommandsIrcProxy.reply
This function was getting uncomfortably big.
2021-04-01 20:10:17 +02:00
Valentin Lorentz 567618392f Add tests + fix bugs 2021-04-01 20:10:17 +02:00
Valentin Lorentz 99a6a7cde9 raise exceptions instead of logging, so we get a helpful traceback 2021-04-01 20:10:16 +02:00
Valentin Lorentz c7939d3eb5 [wip] typo 2021-04-01 20:10:16 +02:00
Valentin Lorentz 26f2d9a292 [wip] idea for ordering batches in Irc.queue 2021-04-01 20:10:16 +02:00
Valentin Lorentz e19436a4ba [wip] support outgoing batches 2021-04-01 20:10:16 +02:00
Valentin Lorentz 8b90884fa0 irclib: Copy messages before echoing them
Fixes a bug triggered by Relay when it relays between three or more networks.
2021-04-01 20:09:28 +02:00
Valentin Lorentz d60cc5c92a irclib: add method getClientTagDenied
To allow plugins to check if they should send a tag or not.
2021-03-18 19:56:30 +01:00
Valentin Lorentz 0919b9f58a ircmsgs: Fix function docstrings. 2021-03-11 19:46:05 +01:00
Valentin Lorentz cb7a395444 irclib: truncate outgoing messages to 512 bytes, not 512 chars 2021-03-11 19:02:55 +01:00
Valentin Lorentz 9bfa1458ef httpserver: improve error handling when binding ports
1. better error messages
2. try binding other ports even if one fails
2021-03-09 23:14:23 +01:00
Valentin Lorentz 11e79e44c5 Remove the '(XX more messages)' when the next message is sent immediately after
It's pointless and looks stupid.

It will look even more stupid when we enable multiline, because the
suffixes will be in the middle of the concatenated message.
2021-03-05 21:19:02 +01:00
Valentin Lorentz 4aca6e3d5a Add support for receiving commands from draft/multiline batches. 2021-03-04 21:34:25 +01:00
Valentin Lorentz f8c5221efd do43x: add the current nick to the list of tried nicks.
The original nick was never added to the list...
2021-03-04 21:29:23 +01:00
Valentin Lorentz 6f6dad8f7b Add proper support for nested batches. 2021-03-03 23:32:00 +01:00
Valentin Lorentz 9719bb799e irclib: Send the bot user mode when advertized by the server
It is enabld by default in order to be a good netizen,
but can be overridden by admins if needed.

References:
*
* https://github.com/ircv3/ircv3-specifications/pull/439
2021-02-28 11:11:29 +01:00
Valentin Lorentz e54751f9c9 Make the email regexp less strict.
The former one disallowed too many valid address, eg. those containing
quotes or square brackets.
2021-01-25 19:49:08 +01:00
Valentin Lorentz 772ec8d6a9 When getting STS policy over insecure connection, reuse the exact same IP address
Otherwise, if some IP addresses don't work (eg. all odd ones), the bot will
consecutively fail because it can't connect, then connect + get STS + reconnect,
then fail again, then connect + get STS, etc.
2021-01-11 23:22:21 +01:00
Valentin Lorentz ba77de0946 Try all IP addresses of a hostname.
Fixes a regression in ecc2c32950 that caused
Socket.py to ignore the IP address entirely after computing it, and
to call getSocket() and connect() with the hostname instead.
2020-12-05 14:26:52 +01:00
Valentin Lorentz 771c05c666 irclib: Fix test failure
FAIL: testMoreIsCaseInsensitive (Misc.test.MiscTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages/supybot/test.py, line 214, in runTest
    originalRunTest()
  File ./plugins/Misc/test.py, line 260, in testMoreIsCaseInsensitive
    self.assertNotError('more %s' % nick.upper())
  File /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages/supybot/test.py, line 355, in assertNotError
    m = self._feedMsg(query, **kwargs)
  File /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages/supybot/test.py, line 526, in _feedMsg
    response = self.irc.takeMsg()
  File /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages/supybot/log.py, line 368, in m
    return f(self, *args, **kwargs)
  File /home/travis/virtualenv/python3.8-dev/lib/python3.8/site-packages/supybot/irclib.py, line 1308, in takeMsg
    assert not msg.tagged('emulatedEcho')
AssertionError

I don't understand why it's happening or why it's only that specific test,
but there we go.
2020-11-10 09:47:36 +01:00
Valentin Lorentz c062787436 irclib: use a dedicated tag to detect emulated echo-messages, instead of receivedAt.
Plugins implementing inFilter do not set receivedAt, so incoming messages
rewritten by inFilter would be detected as echo messages, and ignored
by most plugins.
2020-11-04 11:32:20 +01:00
Valentin Lorentz 903eceebda Simplify checkIgnored code from previous commit 2020-11-04 11:32:20 +01:00