Commit Graph

254 Commits

Author SHA1 Message Date
Valentin Lorentz
dcca9f9637 RSS: Make feed announces actually network-specific
.getSpecific() always returned the  channel-specific but non-network-specific var,
unless the channel-specific and network-specific one was manually set.
2022-01-11 20:13:39 +01:00
Valentin Lorentz
ed9dffdd7f supybot-plugin-doc: Remove extra space + regenerate all READMEs. 2021-12-04 03:19:12 +01:00
Valentin Lorentz
828fff9cc0 supybot-plugin-doc: Add missing newlines + regenerate all READMEs. 2021-12-04 02:50:28 +01:00
Valentin Lorentz
63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz
dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz
ca624d7ae6 RSS: Fix nondeterministic test failures
Caused by 'rss announce add' triggering headline announces, that would
delay the execution of the 'remove' commands.

Thanks to @mapreri and @Unit193 for help in reproducing the issue
and confirming the patch.
2021-08-22 19:23:05 +02:00
Valentin Lorentz
0c8e455b4e RSS: When deleting a named feed, preserve anonymous feed of the same URL. 2021-08-22 19:21:28 +02:00
Valentin Lorentz
db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz
c008d1191c RSS: Fix crashes when the set of channels changed while iterating. 2021-07-31 14:33:34 +02:00
Valentin Lorentz
0af4af16d3 RSS: Fix random test failure
Closes GH-1479
2021-07-04 10:46:51 +02:00
Valentin Lorentz
2e2c243b8b RSS: Remove flaky test 2021-05-28 19:37:37 +02:00
Valentin Lorentz
b8852eb9f3 RSS: Fix nondeterministic test failure 2021-05-28 19:15:56 +02:00
Valentin Lorentz
758f9bee34 RSS: Catch errors from update_feed(), just in case
feedparser should always catch the error, but someone reported it doesn't
catch this error on TLS cert issues:

```
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 86, in newf
    f(*args, **kwargs)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 351, in update_feeds
    self.update_feed_if_needed(feed)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 337, in update_feed_if_needed
    self.update_feed(feed)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 311, in update_feed
    d = feedparser.parse(feed.url, etag=feed.etag,
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/api.py", line 214, in parse
    data = _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/api.py", line 114, in _open_resource
    return http.get(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/http.py", line 158, in get
    f = opener.open(request)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1393, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
```

So let's catch the error just in case, so it doesn't block all other
feeds.
2021-05-28 18:48:35 +02:00
Valentin Lorentz
3415cd08a1 RSS: Add test checking exception from a feed doesn't affect other feeds. 2021-05-28 18:38:06 +02:00
Valentin Lorentz
7c68b2cb53 RSS: Fix crash in @rss list 2021-05-28 17:56:59 +02:00
Valentin Lorentz
97a7aa304e RSS: Make @remove completely clean up feeds 2021-05-01 14:29:09 +02:00
Valentin Lorentz
556bd788b5 RSS: Default to setting announces per channel+network, instead of just per channel
The default behavior was to announce feeds on all channels with the same name,
which is rarely what was expected.
Instead, this limits it to the current network.
2021-05-01 13:31:34 +02:00
Valentin Lorentz
435f8de73a RSS: Fix bug in @remove that prevented it from fully deleting aliased feeds. 2021-05-01 13:27:56 +02:00
Valentin Lorentz
e96633c1e3 RSS: Forbid dots in feed names.
They confuse the registry
2021-04-28 20:04:58 +02:00
Valentin Lorentz
fec6959acb all plugins: regenerate READMEs 2021-04-22 00:29:15 +02:00
Valentin Lorentz
82fbfcffa4 all plugins: regenerate READMEs 2021-04-17 20:09:33 +02:00
Valentin Lorentz
cd02444f87 supybot-plugin-doc: Add labels for all command and config variables
So they can be referenced from elsewhere in the doc.
2021-04-06 00:33:07 +02:00
Valentin Lorentz
833af36b08 all plugins: auto-generate READMEs
Starting with this commit, there should be no "original" text
in READMEs.
Instead, the entire text should be in docstrings, and READMEs
should be automatically generated from these docstrings.

Motivation:

* The same help is available both via IRC and in the README
  (although the README can be more detailed, as only the first
  paragraph will be shown on IRC)
* This will allow auto-generating plugin help on docs.limnoria.net
  using the same content as the README, as it's sourced from the
  docstrings

Additionally, this converts all READMEs from Markdown to ReST,
because the documentation already uses ReST, and other docstrings
in the codebase are in ReST for this reason.
2021-04-05 17:42:31 +02:00
Valentin Lorentz
c04b31ae32 RSS: include channel name in 'feed is announced but does not exist' error log. 2020-10-10 11:51:56 +02:00
Valentin Lorentz
4bd5d8eb33 RSS: Fix announce removal to work with net+chan-specific config
It only removed the value from the chan-specific value.
2020-10-10 11:51:56 +02:00
Valentin Lorentz
0102d64cf4 RSS: fix tests with feedparser 6.
Mocks the urllib API it uses instead of an internal function.
2020-09-19 10:15:23 +02:00
Valentin Lorentz
2953126fca test: Make assertNoResponse default to a non-zero timeout for threaded plugins.
Else it doesn't reliably check there is no response.
2020-04-11 00:17:16 +02:00
Valentin Lorentz
e2d72c5a43 Fix msgfmt warnings. 2020-03-26 22:10:28 +01:00
Valentin Lorentz
2f49362510 RSS: If ids are missing, use <link> as id. 2020-01-31 20:59:42 +01:00
Valentin Lorentz
653afdd086 RSS: Show a meaningful error in case '@rss' is called with a non-URL/name.
Instead of letting feedparser try to parse the name.
2020-01-28 19:35:05 +01:00
Valentin Lorentz
9ab908759a Use importlib instead of imp.
importlib is deprecated.
2020-01-26 12:00:08 +01:00
Valentin Lorentz
9891b0fca4 Replace failUnless/failIf with assertTrue/assertFalse.
The old names are deprecated.
2020-01-26 11:14:18 +01:00
James Lu
9291fbd4a0 All plugins: set Limnoria contributors as maintainer 2019-11-16 11:41:13 -08:00
Valentin Lorentz
f85287771d RSS: Register feed config in config.py instead of plugin.py.
So they are not dropped if plugin.py fails to load before
feedparser is not installed.

Closes GH-1387.
2019-11-11 19:42:34 +01:00
James Lu
3c23faf1bd RSS: add 'announce channels' command to list channels a feed is announced to
Closes #1322.
2019-10-06 20:46:21 +02:00
Valentin Lorentz
702cfaaf97 RSS: Fix __repr__ formatter. 2019-10-05 15:57:28 +02:00
Valentin Lorentz
dcf55cf6de RSS: Use an explicit timeout for assertNoResponse, because it defaults to 0.
Which means there may be a command/update thread running and returning after assertNoResponse returns.
In particular, that's an issue when assertNoResponse is called right before
setting feedparser._open_resource, because the running thread might still
be holding the lock but have fetched the new version already, so the
thread announces the new feed elements earlier than expected.
2019-10-05 15:57:01 +02:00
Valentin Lorentz
a4665803fa News + RSS: Mock time.time() instead of using time.sleep() in tests.
Saves 40s over the ~5min test suite.
2019-10-05 12:22:30 +02:00
Valentin Lorentz
c1ae3f5c81 all plugins: Use msg.channel instead of msg.args[0] + give network name to self.registryValue. 2019-08-24 23:35:01 +02:00
Valentin Lorentz
696254ff1b Use irc.isChannel instead of ircutils.isChannel when possible.
It's aware of the network's 'chantypes'.
2019-08-04 21:52:40 +02:00
James Lu
c7716de887 RSS: add a quick test for a malformed feed that has no entries
(Only both of these combined gives new output so far.)
2018-10-14 12:42:15 -07:00
James Lu
2fdc433cb3 RSS: show soft errors set by feedparser when no entries are found
Hopefully this will ease debugging - e.g. a bad TLS certificate will now show
"Error: Couldn't get RSS feed. Parser error: <urlopen error
 [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)>"
instead of only a generic message.
2018-10-14 12:41:22 -07:00
Tasos Sahanidis
b900a369a4 RSS: Fix feeds with missing description or title
An AttributeError would be raised when a blacklist or a whitelist
would be set on feeds that miss either a title or a description.
2018-03-13 23:50:47 +01:00
Valentin Lorentz
3ab226c129 RSS: Only defer feed loading at plugin load while the bot is starting.
Defering it if the plugin is (re)loaded while already running is useless.
Closes GH-922.
2017-12-10 09:39:20 +01:00
James Lu
bdd0adfef5 RSS: add missing configuration help for feed URL options
This fixes confusing "That configuration group exists, but seems to have no help." errors on the URL
options for feeds, when these options are actually relevant and settable.
2017-12-09 23:03:42 -08:00
Valentin Lorentz
e08e899134 Fix test 2017-10-21 17:11:21 +02:00
Valentin Lorentz
e16faab805 RSS: On startup, only delay feeds loading for 10 seconds instead of the whole of supybot.plugins.RSS.waitPeriod. 2017-10-21 16:48:44 +02:00
Valentin Lorentz
eaa5a5523e RSS: Add var supybot.plugins.RSS.maximumAnnounceHeadlines. 2017-10-12 21:21:50 +02:00
Valentin Lorentz
d73de5e512 Fix potential bug due to mutability of lists as default argument. 2017-03-30 21:38:18 +00:00
Valentin Lorentz
3633a2b2e3 RSS: Only keep the most recent entries in the initial announce. 2017-01-27 21:00:58 +01:00