Commit Graph

130 Commits

Author SHA1 Message Date
Arnout Engelen
58e7e61d69 Only reset the cached feed when the response actually contains headlines
Some feeds, such as those from sourceforge.net, will sometimes show an error
page rather than a feed. In this case the feed cache used to be cleared for
that feed, causing all 'old' headlines to be flooded to the channel as soon
as the feed came back online.

This patch hopefully fixes that by only resetting the cache when the returned
page actually contains headlines.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2013-08-23 23:13:28 -04:00
Kill Your TV
b46a0dd6a2 Unicode fixes for python 2.x
These changes have been tested with Python 3.2.3 and Python 2.7.5.
2013-08-17 14:12:10 +00:00
Kill Your TV
56c80e62f1 RSS: fix initialAnnounceHeadlines not being obeyed 2013-08-15 22:32:19 +00:00
Valentin Lorentz
33a236079f RSS: Fall back to decoding the headline with 'utf8' if 'ascii' failed. 2013-07-31 09:33:03 +00:00
Andrew Hampe
75723e7e8c RSS: Remove b'' from around the RSS url. 2013-07-18 13:13:36 -05:00
Valentin Lorentz
071b61061a RSS: Honor configuration variable supybot.reply.format.url in announces. Closes GH-487. 2013-07-18 10:11:02 +00:00
Daniel Folkinshteyn
c27070895b RSS: add option display headline timestamp.
Conflicts:
	plugins/RSS/config.py
	plugins/RSS/plugin.py
2013-06-27 07:06:23 +02:00
Daniel Folkinshteyn
e4ddda4aeb RSS: add option to strip url redirects from headlines
Conflicts:
	plugins/RSS/config.py
2013-06-27 07:03:57 +02:00
Valentin Lorentz
20bef2dcd0 RSS: keep track of headlines over multiple feed fetches, with configurable expiration.
This is better at avoiding repeats than just keeping the last fetch, since some feeds
shuffle items around (like google news search).

Conflicts:
	plugins/RSS/config.py
2013-06-27 07:03:20 +02:00
Valentin Lorentz
ad25f17639 RSS: Add Python 3 support. 2013-05-29 15:16:23 +02:00
Valentin Lorentz
0537166502 RSS: Fix formating if newline character in an entry title. Closes GH-395. 2013-05-22 14:03:38 +02:00
Daniel Folkinshteyn
af1931b3db RSS: add option display headline timestamp. 2013-05-05 11:23:15 -04:00
Daniel Folkinshteyn
78659113c1 RSS: add option to strip url redirects from headlines 2013-05-04 00:01:52 -04:00
Daniel Folkinshteyn
bc0d16a4e1 RSS: keep track of headlines over multiple feed fetches, with configurable expiration.
This is better at avoiding repeats than just keeping the last fetch, since some feeds
shuffle items around (like google news search).
2013-05-03 23:39:34 -04:00
Valentin Lorentz
61eb0fd3f8 RSS: add supybot.plugins.RSS.announcementSeparator. 2013-04-06 16:57:30 +00:00
Valentin Lorentz
f880d5b6ef RSS: Fix typo. 2013-01-23 16:32:23 +01:00
Valentin Lorentz
3dba9088b0 Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	INSTALL
	plugins/ChannelLogger/README.txt
	plugins/ChannelStats/README.txt
	plugins/Google/plugin.py
	plugins/Google/test.py
	plugins/Plugin/test.py
	plugins/Web/test.py
	setup.py
	src/callbacks.py
	src/ircdb.py
	src/irclib.py
	src/utils/str.py
	test/test_irclib.py
2013-01-01 21:11:24 +01:00
James McCoy
aa4071fa68 RSS: Remove local/feedparser and add it to install_requires
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-14 12:13:00 -04:00
James McCoy
82ecf36fcd Update my name/contact information
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-01 10:16:48 -04:00
Valentin Lorentz
0c6320e9c1 RSS: Display error instead of traceback if fetching a feed failed. 2012-08-26 07:38:40 +00:00
Valentin Lorentz
242a54fbf2 Alias & RSS: remove third argument to types.MethodType.
It is useless in the case in point, and is not compatible with Python 3.
2012-08-05 14:08:04 +02:00
Valentin Lorentz
11df697b96 RSS: Be compatible with new versions of feedparser. 2012-08-04 20:43:11 +02:00
Valentin Lorentz
8b7d16c850 Alias & RSS: Use types.MethodType instead of new.instancemethod. 2012-08-04 18:46:16 +02:00
Sebastian Kayser
2486be4464 RSS: sort and display feed items by date 2012-05-28 19:32:24 +02:00
James McCoy
ff96b898f9 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>

Conflicts:

	plugins/RSS/plugin.py
2011-12-19 01:32:05 +01:00
James McCoy
b42b06fe79 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
(cherry picked from commit 964c73f591)

Signed-off-by: Daniel Folkinshteyn <nanotube@users.sourceforge.net>
2011-12-12 11:04:03 -05:00
James McCoy
8b73f78b3e Merge branch 'maint/0.83.4'
Conflicts:
	plugins/Misc/plugin.py
	src/ircutils.py

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2011-11-13 15:26:19 -05:00
James McCoy
964c73f591 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2011-10-22 15:23:56 -04:00
Valentin Lorentz
dda08e0648 RSS: Fix typo (fnor -> for). 2011-08-27 13:58:14 +00:00
Valentin Lorentz
08a28214b8 RSS: bug fix. (Bad import from gribble?) 2011-08-26 17:16:13 +02:00
Daniel Folkinshteyn
7c14992fe8 RSS: don't output anything if there are no headlines remaining after filtering. 2011-08-15 13:45:07 -04:00
Daniel Folkinshteyn
c270111c6b RSS: fix bug failing to define a variable under some conditions before referencing it. 2011-08-15 11:13:54 -04:00
Daniel Folkinshteyn
dd040f830d RSS: add channel-specific blacklist and whitelist.
also fix bug introduced with the initialannounce feature, which overwrote newheadlines
list when doing channel-specific things with it.
2011-08-14 10:53:58 +02:00
Daniel Folkinshteyn
af32d6bfd3 RSS: add channel-specific blacklist and whitelist.
also fix bug introduced with the initialannounce feature, which overwrote newheadlines
list when doing channel-specific things with it.
2011-08-14 01:42:08 -04:00
Daniel Folkinshteyn
7c3bc67c86 Add some sanity checks to RSS plugin:
First: if an rss feed is called without the number of headlines argument,
we now have a default number of items it will output.
before, it tried to stuff the whole rss feed into the channel, which is quite
floody, if more than one 'mores' is set, or if oneToOne is false.

Second: when adding a new feed to announce, it again, tried to stuff the
whole rss feed into the channel, which ran into the same floody conditions
as above. now we have a default number of feeds to output when there's no
cached history.
2011-02-25 19:15:41 +01:00
Valentin Lorentz
6e6fd58ea7 Fix RSS encoding problem 2011-01-01 17:24:13 +01:00
Valentin Lorentz
db977bf5d2 Fix Python2.5 issue (class decorators)
Issue S-fr#5
2010-10-26 09:32:12 +02:00
Valentin Lorentz
4aff05f361 RSS: fix mislocated decorator 2010-10-26 09:27:09 +02:00
Valentin Lorentz
aaa0c480af Internationalize Relay, Reply, RSS, Scheduler 2010-10-19 19:50:41 +02:00
James Vega
3f63917dc2 Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 0c62204809)
2010-08-24 19:52:09 -04:00
Daniel Folkinshteyn
e9d55d4bbd fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 40941e044a)
2010-08-24 19:48:42 -04:00
James Vega
23cca935cb Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 0c62204809)

Signed-off-by: Daniel Folkinshteyn <nanotube@users.sourceforge.net>
2010-06-25 00:42:02 -04:00
James Vega
0c62204809 Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-06-24 00:37:40 -04:00
Daniel Folkinshteyn
40941e044a fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-05-24 14:46:04 -04:00
Daniel Folkinshteyn
2bb3ba9443 fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.
2010-04-26 19:50:08 -04:00
Daniel Folkinshteyn
55eeb4a57b Add some sanity checks to RSS plugin:
First: if an rss feed is called without the number of headlines argument,
we now have a default number of items it will output.
before, it tried to stuff the whole rss feed into the channel, which is quite
floody, if more than one 'mores' is set, or if oneToOne is false.

Second: when adding a new feed to announce, it again, tried to stuff the
whole rss feed into the channel, which ran into the same floody conditions
as above. now we have a default number of feeds to output when there's no
cached history.
2010-04-25 00:34:31 -04:00
James Vega
eff20402ac Change canonize => normalize
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-08-27 16:41:34 -04:00
James Vega
54dda880d4 Add a pluginCommands kwarg to Commands.listCommands.
Plugins which implement their own listCommands method should pass their
non-typically discoverable commands to Commands.listCommands via this
mechanism.  This means that the de-duplication of commands is performed in one
spot instead of having each plugin implement it on their on in their
listCommands method.

This reverts commits 0ce829af6215b97e725f4d6d580d1151950be869 and
09fb0e6fc974445a3414fb03a94625f8538d4570.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-05-22 18:52:15 -04:00
James Vega
7b906fbf54 Fully qualify the call to universalImport so we don't get a NameError.
Also added some ImportError handlers to aid letting users of distribution
packages know if the distributions strip out the local module copies.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-03-19 20:33:17 -04:00
James Vega
3fd92e7f24 RSS: Move feedparser to the local directory
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-03-11 19:43:23 -04:00
Jeremy Fincher
5076241bcd Fixed RSS to include a feedparser which doesn't override the globally-installed one, using universalImport. 2009-03-11 14:06:16 -04:00
James Vega
62acb0c998 Make RSS.announce a nested plugin
This simplifies the interface for dealing with announced feeds by providing
separate "announce add", "announce remove", "announce list" commands as
opposed to the one "announce" command with confusing syntax.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-02-28 00:10:10 -05:00
James Vega
0e05fe67c7 No need to define RSS.listCommands
Doing so can lead to RSS feeds being listed multiple times.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-02-27 23:36:29 -05:00
James Vega
9cc917bd36 Guard feedparser import and warn the user if it's not available 2009-03-11 13:37:26 -04:00
James Vega
7bb3b1ec60 Merge the changes from the python25compat branch into the trunk 2009-03-11 13:37:24 -04:00
James Vega
8111c48a49 Switch from rssparser 3.3 to (the renamed) feedparser 4.1 in order to properly handle Bugzilla RSS feeds. 2007-05-09 02:30:18 +00:00
James Vega
4064da76aa plugins/RSS: UTF-8 encode the RSS' URL to prevent an encoding error when including the URL in the reply. 2006-08-28 13:18:12 +00:00
James Vega
73d7fd3a2d plugins/RSS/plugin.py: Refactor the character encoding detection and conversion so it can be used in more places. Hopepfully this will cause fewer UnicodeDecodeErrors. 2006-05-02 12:22:11 +00:00
James Vega
a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega
1edb4e13bf plugins/RSS: Colons are pretty. 2005-07-22 12:20:11 +00:00
James Vega
45c7f773ac plugins/RSS: Fix info so it works again. 2005-07-19 21:32:54 +00:00
James Vega
14dc0c0201 plugins/RSS: If the feed supplies an encoding, encode the headlines with it. 2005-07-07 14:13:53 +00:00
Jeremy Fincher
490fb0b140 Changed prefixName to prefixNick, which is more appropriate, and has always bothered me. Better now than later. 2005-06-01 21:08:30 +00:00
Jeremy Fincher
4da1f38235 Fixed the security issue nicktastic reported, and made sure that if, for some reason, an error doesn't raise an exception, it still stops the calling of the command function. 2005-06-01 20:13:33 +00:00
James Vega
34f5be5252 RSS: Use del instead of pop. 2005-05-30 02:27:16 +00:00
James Vega
7fdb72ab4f plugins/RSS: Stop using attributes for feeds. 2005-05-30 00:52:54 +00:00
James Vega
85e9035535 plugins/RSS: Fix a bug in RSS.remove. 2005-05-29 16:54:57 +00:00
James Vega
994cff5c05 Update a couple missed changeFunctionName calls. 2005-05-16 03:25:43 +00:00
James Vega
9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
James Vega
919f9e0915 Grammar nazi of DOOM! 2005-02-25 00:06:16 +00:00
James Vega
80c53dcea3 Make automatic announcement errors not: a) poll every time __call__ is called, b) respond with the errors. 2005-02-25 00:04:39 +00:00
James Vega
527a6e1834 Fix the problem with RSS complaining about invalid feed names. 2005-02-23 00:03:09 +00:00
James Vega
25ae2a52cf Use get to avoid a KeyError. 2005-02-22 14:28:45 +00:00
James Vega
3f94bf6dae Fix an exception in RSS.announce. 2005-02-22 14:27:59 +00:00
Jeremy Fincher
ee7ca1668c Fix for the new refactoring in callbacks. 2005-02-19 08:43:37 +00:00
Jeremy Fincher
99d723802e Changed the name of getCommand to getCommandMethod in preparation for a major refactoring. 2005-02-15 06:53:30 +00:00
Jeremy Fincher
b0cb616709 Changed callbacks.Privmsg to be callbacks.Plugin, and callbacks.PrivmsgCommandAndRegexp to be callbacks.Plugin. 2005-02-09 07:04:04 +00:00
Jeremy Fincher
571ff24155 Changed canonicalize to canonize -- thanks Skorobeus! 2005-02-04 03:19:05 +00:00
Jeremy Fincher
e94900c0eb Removed some debugging code, and added a bit of extra debug logging. 2005-02-02 06:12:25 +00:00
Jeremy Fincher
3f1b0335c2 Added RSS in the new plugin format. 2005-02-02 06:03:09 +00:00