Commit Graph

155 Commits

Author SHA1 Message Date
Valentin Lorentz
c008d1191c RSS: Fix crashes when the set of channels changed while iterating. 2021-07-31 14:33:34 +02:00
Valentin Lorentz
758f9bee34 RSS: Catch errors from update_feed(), just in case
feedparser should always catch the error, but someone reported it doesn't
catch this error on TLS cert issues:

```
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 86, in newf
    f(*args, **kwargs)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 351, in update_feeds
    self.update_feed_if_needed(feed)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 337, in update_feed_if_needed
    self.update_feed(feed)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/supybot/plugins/RSS/plugin.py", line 311, in update_feed
    d = feedparser.parse(feed.url, etag=feed.etag,
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/api.py", line 214, in parse
    data = _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/api.py", line 114, in _open_resource
    return http.get(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/home/falso/virtualenv/limnoria/lib/python3.8/site-packages/feedparser/http.py", line 158, in get
    f = opener.open(request)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1393, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
```

So let's catch the error just in case, so it doesn't block all other
feeds.
2021-05-28 18:48:35 +02:00
Valentin Lorentz
7c68b2cb53 RSS: Fix crash in @rss list 2021-05-28 17:56:59 +02:00
Valentin Lorentz
97a7aa304e RSS: Make @remove completely clean up feeds 2021-05-01 14:29:09 +02:00
Valentin Lorentz
556bd788b5 RSS: Default to setting announces per channel+network, instead of just per channel
The default behavior was to announce feeds on all channels with the same name,
which is rarely what was expected.
Instead, this limits it to the current network.
2021-05-01 13:31:34 +02:00
Valentin Lorentz
435f8de73a RSS: Fix bug in @remove that prevented it from fully deleting aliased feeds. 2021-05-01 13:27:56 +02:00
Valentin Lorentz
e96633c1e3 RSS: Forbid dots in feed names.
They confuse the registry
2021-04-28 20:04:58 +02:00
Valentin Lorentz
833af36b08 all plugins: auto-generate READMEs
Starting with this commit, there should be no "original" text
in READMEs.
Instead, the entire text should be in docstrings, and READMEs
should be automatically generated from these docstrings.

Motivation:

* The same help is available both via IRC and in the README
  (although the README can be more detailed, as only the first
  paragraph will be shown on IRC)
* This will allow auto-generating plugin help on docs.limnoria.net
  using the same content as the README, as it's sourced from the
  docstrings

Additionally, this converts all READMEs from Markdown to ReST,
because the documentation already uses ReST, and other docstrings
in the codebase are in ReST for this reason.
2021-04-05 17:42:31 +02:00
Valentin Lorentz
c04b31ae32 RSS: include channel name in 'feed is announced but does not exist' error log. 2020-10-10 11:51:56 +02:00
Valentin Lorentz
4bd5d8eb33 RSS: Fix announce removal to work with net+chan-specific config
It only removed the value from the chan-specific value.
2020-10-10 11:51:56 +02:00
Valentin Lorentz
2f49362510 RSS: If ids are missing, use <link> as id. 2020-01-31 20:59:42 +01:00
Valentin Lorentz
653afdd086 RSS: Show a meaningful error in case '@rss' is called with a non-URL/name.
Instead of letting feedparser try to parse the name.
2020-01-28 19:35:05 +01:00
Valentin Lorentz
f85287771d RSS: Register feed config in config.py instead of plugin.py.
So they are not dropped if plugin.py fails to load before
feedparser is not installed.

Closes GH-1387.
2019-11-11 19:42:34 +01:00
James Lu
3c23faf1bd RSS: add 'announce channels' command to list channels a feed is announced to
Closes #1322.
2019-10-06 20:46:21 +02:00
Valentin Lorentz
702cfaaf97 RSS: Fix __repr__ formatter. 2019-10-05 15:57:28 +02:00
Valentin Lorentz
c1ae3f5c81 all plugins: Use msg.channel instead of msg.args[0] + give network name to self.registryValue. 2019-08-24 23:35:01 +02:00
Valentin Lorentz
696254ff1b Use irc.isChannel instead of ircutils.isChannel when possible.
It's aware of the network's 'chantypes'.
2019-08-04 21:52:40 +02:00
James Lu
2fdc433cb3 RSS: show soft errors set by feedparser when no entries are found
Hopefully this will ease debugging - e.g. a bad TLS certificate will now show
"Error: Couldn't get RSS feed. Parser error: <urlopen error
 [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)>"
instead of only a generic message.
2018-10-14 12:41:22 -07:00
Tasos Sahanidis
b900a369a4 RSS: Fix feeds with missing description or title
An AttributeError would be raised when a blacklist or a whitelist
would be set on feeds that miss either a title or a description.
2018-03-13 23:50:47 +01:00
Valentin Lorentz
3ab226c129 RSS: Only defer feed loading at plugin load while the bot is starting.
Defering it if the plugin is (re)loaded while already running is useless.
Closes GH-922.
2017-12-10 09:39:20 +01:00
James Lu
bdd0adfef5 RSS: add missing configuration help for feed URL options
This fixes confusing "That configuration group exists, but seems to have no help." errors on the URL
options for feeds, when these options are actually relevant and settable.
2017-12-09 23:03:42 -08:00
Valentin Lorentz
e16faab805 RSS: On startup, only delay feeds loading for 10 seconds instead of the whole of supybot.plugins.RSS.waitPeriod. 2017-10-21 16:48:44 +02:00
Valentin Lorentz
eaa5a5523e RSS: Add var supybot.plugins.RSS.maximumAnnounceHeadlines. 2017-10-12 21:21:50 +02:00
Valentin Lorentz
d73de5e512 Fix potential bug due to mutability of lists as default argument. 2017-03-30 21:38:18 +00:00
Valentin Lorentz
3633a2b2e3 RSS: Only keep the most recent entries in the initial announce. 2017-01-27 21:00:58 +01:00
Valentin Lorentz
d7099794bc Fix Python 2 support of last commit. 2016-09-20 23:22:13 +02:00
Valentin Lorentz
2cfe0f679f Fix previous commit on some Python versions. 2016-09-20 23:09:29 +02:00
Valentin Lorentz
d39647142c RSS: Fix shadowing of any/all. 2016-09-20 22:54:11 +02:00
Valentin Lorentz
4f280599e1 RSS: use the original FeedParserDict for substitution
instead of building a new keyword arguments dict by iteration.

'description' is a field in FeedParserDict, but is not present when iterating,
which made it unavailable as a substitution variable.
2016-09-08 21:42:31 +02:00
Valentin Lorentz
3776404628 RSS: Use HTTP(S) proxy. Closes GH-1234. 2016-08-16 10:56:29 +02:00
Valentin Lorentz
3f66dbc3d5 Channel & Dunno & Misc & RSS & Scheduler: Let the logging facility perform string templating to avoid extra format chars in argument. 2015-11-03 17:25:13 +01:00
Valentin Lorentz
fb52b16429 RSS: Allow @announce add to take URLs as feed names. 2015-10-10 20:40:25 +02:00
Valentin Lorentz
4498548cb3 RSS: Add supybot.plugins.RSS.notice. Closes GH-1151. 2015-09-23 11:27:20 +02:00
Valentin Lorentz
82332ff87c RSS: Add feed-specific waitPeriod. 2015-09-23 11:05:25 +02:00
Valentin Lorentz
1c218592af RSS: Add fallback on sorting by update timestamp + add outdatedFirst and updatedFirst sorts. Closes GH-1139. 2015-09-02 10:04:04 +02:00
Valentin Lorentz
29ca3eb5e4 RSS: Sort on time tuples instead of formatted dates. 2015-09-02 09:43:29 +02:00
Valentin Lorentz
c3a2c800f1 Remove need for 2to3. 2015-08-11 16:50:23 +02:00
Valentin Lorentz
216c5d213f Replace sys.version_info[0] usages with minisix.PY{2,3}. 2015-08-09 00:23:03 +02:00
Valentin Lorentz
7c4a28d7fd RSS: Remove illegal characters from entries. 2015-06-01 19:32:40 +02:00
Valentin Lorentz
ac0e6b86c0 RSS: Fix typo. 2015-05-23 11:48:32 +02:00
Valentin Lorentz
34138b9185 RSS: Don't crash on load if there is an invalid feed name. Closes GH-1116. 2015-05-23 11:40:26 +02:00
Valentin Lorentz
1650860032 RSS: Do no block commands calling while loading a feed. 2014-10-14 20:59:00 +00:00
Valentin Lorentz
d2046a81a2 RSS: Prevent errors if users do not use @announce add correctly. Closes GH-869. 2014-10-10 14:41:45 +00:00
Valentin Lorentz
44707374c0 RSS: Fix support of feeds that do no have guid fields. Closes GH845. 2014-09-14 10:06:45 +00:00
Valentin Lorentz
15fb16a8ae RSS: Make the cache bigger in order to prevent duplicated announces. 2014-09-12 22:37:06 +00:00
Valentin Lorentz
e51e45de4d RSS: Fix typo. 2014-08-13 17:04:35 +02:00
Valentin Lorentz
c08a480f11 RSS: Honor supybot.plugins.RSS.initialAnnounceHeadlines. 2014-08-13 16:43:43 +02:00
Valentin Lorentz
96ba33a296 RSS: Fix bad usage of safe_substitute. 2014-08-13 14:17:06 +00:00
Valentin Lorentz
1d4a1b13d8 RSS: Internationalize feed-specific format config doc. 2014-08-10 15:29:47 +02:00
Valentin Lorentz
3b2382d2f9 RSS: Remove debug output. 2014-08-05 11:49:13 +02:00