Commit Graph

162 Commits

Author SHA1 Message Date
Valentin Lorentz
44707374c0 RSS: Fix support of feeds that do no have guid fields. Closes GH845. 2014-09-14 10:06:45 +00:00
Valentin Lorentz
15fb16a8ae RSS: Make the cache bigger in order to prevent duplicated announces. 2014-09-12 22:37:06 +00:00
Valentin Lorentz
e51e45de4d RSS: Fix typo. 2014-08-13 17:04:35 +02:00
Valentin Lorentz
c08a480f11 RSS: Honor supybot.plugins.RSS.initialAnnounceHeadlines. 2014-08-13 16:43:43 +02:00
Valentin Lorentz
96ba33a296 RSS: Fix bad usage of safe_substitute. 2014-08-13 14:17:06 +00:00
Valentin Lorentz
1d4a1b13d8 RSS: Internationalize feed-specific format config doc. 2014-08-10 15:29:47 +02:00
Valentin Lorentz
3b2382d2f9 RSS: Remove debug output. 2014-08-05 11:49:13 +02:00
Valentin Lorentz
8efcc4a009 RSS: Display nice error if an announced feed does not exist. 2014-08-04 17:01:30 +02:00
Valentin Lorentz
d431c2992b Factoids: Fix compatibility of web server with Python 3. Closes GH-784. 2014-08-04 13:21:40 +00:00
Valentin Lorentz
ab1ac2aa8e RSS: Remove use of deprecated property. Closes GH-807. 2014-08-04 15:02:46 +02:00
Valentin Lorentz
ddcbc151b6 RSS: Fix listing of commands (show names, not URLs). 2014-08-01 09:17:58 +00:00
Valentin Lorentz
f3b31e5a4b RSS: Add support for ETag and Last-Modified. 2014-08-01 01:12:19 +02:00
Valentin Lorentz
332164aa20 RSS: Fix Python2.6 compatibility. 2014-07-31 21:58:30 +00:00
Valentin Lorentz
057da44ab3 RSS: Make sure two different feeds cannot have the same URL. 2014-07-31 23:56:27 +02:00
Valentin Lorentz
3a127e884e RSS: Allow feed-specific format. Closes GH-104. 2014-07-31 23:50:43 +02:00
Valentin Lorentz
1265f9229a RSS: Open dump file in text mode (json uses text mode). 2014-07-31 20:57:47 +00:00
Valentin Lorentz
b5911f8489 RSS: Implement persistence of announced headlines. 2014-07-31 22:53:21 +02:00
Valentin Lorentz
e9b58f7820 RSS: Use queueMsg instead of sendMsg (flood…) 2014-07-31 19:17:27 +00:00
Valentin Lorentz
784b534a3d RSS: Remove deadlock and make announces work. 2014-07-31 20:50:12 +02:00
Valentin Lorentz
f35ece8147 RSS: Fix stupid bugs. 2014-07-31 20:14:09 +02:00
Valentin Lorentz
ca425b7b0a RSS: Fix checking of feed existancy. 2014-07-31 19:31:20 +02:00
Valentin Lorentz
0a6a4991a5 RSS: Make feed commands work. 2014-07-31 18:56:52 +02:00
Valentin Lorentz
219c47d1c8 RSS: Major rewriting. @rss works. 2014-07-31 18:44:49 +02:00
Valentin Lorentz
0c2ab9d83b RSS: Another attempt at fixing encoding issues (again). 2014-03-22 11:56:38 +00:00
Valentin Lorentz
06c83cbf14 RSS: Fix typo (again). 2014-03-22 11:47:28 +00:00
Valentin Lorentz
04cf2ca27a RSS: Fix typo. 2014-03-22 11:37:52 +00:00
Valentin Lorentz
933d4d3ffa RSS: Another attempt at fixing encoding issues. 2014-03-22 11:29:29 +00:00
Valentin Lorentz
bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
Valentin Lorentz
2fda69b4d6 Continue accelerating the 2to3 step (remove fix_raise). 2014-01-20 15:43:55 +01:00
Valentin Lorentz
4652c9ce51 Start accelerating the 2to3 step (remove fix_apply, fix_buffer, fix_callable, fix_exec, fix_execfile, fix_exitfunc, fix_filter, fix_funcattrs, fix_future, fix_getcwdu, and fix_has_key). 2014-01-20 14:49:47 +01:00
Valentin Lorentz
40675ffdfa Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	plugins/Admin/plugin.py
	plugins/BadWords/config.py
	plugins/Filter/plugin.py
	plugins/Google/plugin.py
	plugins/Math/plugin.py
	plugins/Misc/plugin.py
	plugins/Note/plugin.py
	plugins/RSS/plugin.py
	plugins/Seen/test.py
	plugins/ShrinkUrl/config.py
	plugins/ShrinkUrl/plugin.py
	plugins/ShrinkUrl/test.py
	plugins/Status/plugin.py
	plugins/String/config.py
	plugins/String/plugin.py
	plugins/Time/plugin.py
	plugins/Todo/plugin.py
	plugins/Web/plugin.py
	plugins/__init__.py
	scripts/supybot-botchk
	setup.py
	src/__init__.py
	src/callbacks.py
	src/commands.py
	src/conf.py
	src/drivers/Socket.py
	src/ircdb.py
	src/irclib.py
	src/ircutils.py
	src/questions.py
	src/registry.py
	src/schedule.py
	src/test.py
	src/utils/file.py
	src/utils/gen.py
	src/utils/net.py
	src/utils/web.py
	src/world.py
2013-08-24 11:28:29 +02:00
James McCoy
a483fef39d RSS: Ensure results is declared before trying to access it
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2013-08-24 00:04:29 -04:00
Arnout Engelen
58e7e61d69 Only reset the cached feed when the response actually contains headlines
Some feeds, such as those from sourceforge.net, will sometimes show an error
page rather than a feed. In this case the feed cache used to be cleared for
that feed, causing all 'old' headlines to be flooded to the channel as soon
as the feed came back online.

This patch hopefully fixes that by only resetting the cache when the returned
page actually contains headlines.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2013-08-23 23:13:28 -04:00
Kill Your TV
b46a0dd6a2 Unicode fixes for python 2.x
These changes have been tested with Python 3.2.3 and Python 2.7.5.
2013-08-17 14:12:10 +00:00
Kill Your TV
56c80e62f1 RSS: fix initialAnnounceHeadlines not being obeyed 2013-08-15 22:32:19 +00:00
Valentin Lorentz
33a236079f RSS: Fall back to decoding the headline with 'utf8' if 'ascii' failed. 2013-07-31 09:33:03 +00:00
Andrew Hampe
75723e7e8c RSS: Remove b'' from around the RSS url. 2013-07-18 13:13:36 -05:00
Valentin Lorentz
071b61061a RSS: Honor configuration variable supybot.reply.format.url in announces. Closes GH-487. 2013-07-18 10:11:02 +00:00
Daniel Folkinshteyn
c27070895b RSS: add option display headline timestamp.
Conflicts:
	plugins/RSS/config.py
	plugins/RSS/plugin.py
2013-06-27 07:06:23 +02:00
Daniel Folkinshteyn
e4ddda4aeb RSS: add option to strip url redirects from headlines
Conflicts:
	plugins/RSS/config.py
2013-06-27 07:03:57 +02:00
Valentin Lorentz
20bef2dcd0 RSS: keep track of headlines over multiple feed fetches, with configurable expiration.
This is better at avoiding repeats than just keeping the last fetch, since some feeds
shuffle items around (like google news search).

Conflicts:
	plugins/RSS/config.py
2013-06-27 07:03:20 +02:00
Valentin Lorentz
ad25f17639 RSS: Add Python 3 support. 2013-05-29 15:16:23 +02:00
Valentin Lorentz
0537166502 RSS: Fix formating if newline character in an entry title. Closes GH-395. 2013-05-22 14:03:38 +02:00
Daniel Folkinshteyn
af1931b3db RSS: add option display headline timestamp. 2013-05-05 11:23:15 -04:00
Daniel Folkinshteyn
78659113c1 RSS: add option to strip url redirects from headlines 2013-05-04 00:01:52 -04:00
Daniel Folkinshteyn
bc0d16a4e1 RSS: keep track of headlines over multiple feed fetches, with configurable expiration.
This is better at avoiding repeats than just keeping the last fetch, since some feeds
shuffle items around (like google news search).
2013-05-03 23:39:34 -04:00
Valentin Lorentz
61eb0fd3f8 RSS: add supybot.plugins.RSS.announcementSeparator. 2013-04-06 16:57:30 +00:00
Valentin Lorentz
f880d5b6ef RSS: Fix typo. 2013-01-23 16:32:23 +01:00
Valentin Lorentz
3dba9088b0 Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	INSTALL
	plugins/ChannelLogger/README.txt
	plugins/ChannelStats/README.txt
	plugins/Google/plugin.py
	plugins/Google/test.py
	plugins/Plugin/test.py
	plugins/Web/test.py
	setup.py
	src/callbacks.py
	src/ircdb.py
	src/irclib.py
	src/utils/str.py
	test/test_irclib.py
2013-01-01 21:11:24 +01:00
James McCoy
aa4071fa68 RSS: Remove local/feedparser and add it to install_requires
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-14 12:13:00 -04:00
James McCoy
82ecf36fcd Update my name/contact information
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-01 10:16:48 -04:00
Valentin Lorentz
0c6320e9c1 RSS: Display error instead of traceback if fetching a feed failed. 2012-08-26 07:38:40 +00:00
Valentin Lorentz
242a54fbf2 Alias & RSS: remove third argument to types.MethodType.
It is useless in the case in point, and is not compatible with Python 3.
2012-08-05 14:08:04 +02:00
Valentin Lorentz
11df697b96 RSS: Be compatible with new versions of feedparser. 2012-08-04 20:43:11 +02:00
Valentin Lorentz
8b7d16c850 Alias & RSS: Use types.MethodType instead of new.instancemethod. 2012-08-04 18:46:16 +02:00
Sebastian Kayser
2486be4464 RSS: sort and display feed items by date 2012-05-28 19:32:24 +02:00
James McCoy
ff96b898f9 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>

Conflicts:

	plugins/RSS/plugin.py
2011-12-19 01:32:05 +01:00
James McCoy
b42b06fe79 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
(cherry picked from commit 964c73f591)

Signed-off-by: Daniel Folkinshteyn <nanotube@users.sourceforge.net>
2011-12-12 11:04:03 -05:00
James McCoy
8b73f78b3e Merge branch 'maint/0.83.4'
Conflicts:
	plugins/Misc/plugin.py
	src/ircutils.py

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2011-11-13 15:26:19 -05:00
James McCoy
964c73f591 RSS._getConverter: Encode strings before handing them off to other functions
When the feed has a specified encoding, we'll be dealing with unicode objects
in the response from feedparser.parse().  To avoid possible UnicodeErrors, we
need to encode() before handing the string off to other functions, so the
other functions are always dealing with bytestrings instead of bytestrings and
unicode objects.  Mixing unicode and bytestrings will cause implicit
conversions of the unicode objects, which will most likely use the wrong
encoding.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2011-10-22 15:23:56 -04:00
Valentin Lorentz
dda08e0648 RSS: Fix typo (fnor -> for). 2011-08-27 13:58:14 +00:00
Valentin Lorentz
08a28214b8 RSS: bug fix. (Bad import from gribble?) 2011-08-26 17:16:13 +02:00
Daniel Folkinshteyn
7c14992fe8 RSS: don't output anything if there are no headlines remaining after filtering. 2011-08-15 13:45:07 -04:00
Daniel Folkinshteyn
c270111c6b RSS: fix bug failing to define a variable under some conditions before referencing it. 2011-08-15 11:13:54 -04:00
Daniel Folkinshteyn
dd040f830d RSS: add channel-specific blacklist and whitelist.
also fix bug introduced with the initialannounce feature, which overwrote newheadlines
list when doing channel-specific things with it.
2011-08-14 10:53:58 +02:00
Daniel Folkinshteyn
af32d6bfd3 RSS: add channel-specific blacklist and whitelist.
also fix bug introduced with the initialannounce feature, which overwrote newheadlines
list when doing channel-specific things with it.
2011-08-14 01:42:08 -04:00
Daniel Folkinshteyn
7c3bc67c86 Add some sanity checks to RSS plugin:
First: if an rss feed is called without the number of headlines argument,
we now have a default number of items it will output.
before, it tried to stuff the whole rss feed into the channel, which is quite
floody, if more than one 'mores' is set, or if oneToOne is false.

Second: when adding a new feed to announce, it again, tried to stuff the
whole rss feed into the channel, which ran into the same floody conditions
as above. now we have a default number of feeds to output when there's no
cached history.
2011-02-25 19:15:41 +01:00
Valentin Lorentz
6e6fd58ea7 Fix RSS encoding problem 2011-01-01 17:24:13 +01:00
Valentin Lorentz
db977bf5d2 Fix Python2.5 issue (class decorators)
Issue S-fr#5
2010-10-26 09:32:12 +02:00
Valentin Lorentz
4aff05f361 RSS: fix mislocated decorator 2010-10-26 09:27:09 +02:00
Valentin Lorentz
aaa0c480af Internationalize Relay, Reply, RSS, Scheduler 2010-10-19 19:50:41 +02:00
James Vega
3f63917dc2 Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 0c62204809)
2010-08-24 19:52:09 -04:00
Daniel Folkinshteyn
e9d55d4bbd fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 40941e044a)
2010-08-24 19:48:42 -04:00
James Vega
23cca935cb Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
(cherry picked from commit 0c62204809)

Signed-off-by: Daniel Folkinshteyn <nanotube@users.sourceforge.net>
2010-06-25 00:42:02 -04:00
James Vega
0c62204809 Use conf.registerGlobalValue to ensure generated values are properly setup.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-06-24 00:37:40 -04:00
Daniel Folkinshteyn
40941e044a fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-05-24 14:46:04 -04:00
Daniel Folkinshteyn
2bb3ba9443 fix bug in RSS.announce.list:
Because the 'channel' argument was declared optional, calling announce.list
off-channel without a channel argument caused an error.
2010-04-26 19:50:08 -04:00
Daniel Folkinshteyn
55eeb4a57b Add some sanity checks to RSS plugin:
First: if an rss feed is called without the number of headlines argument,
we now have a default number of items it will output.
before, it tried to stuff the whole rss feed into the channel, which is quite
floody, if more than one 'mores' is set, or if oneToOne is false.

Second: when adding a new feed to announce, it again, tried to stuff the
whole rss feed into the channel, which ran into the same floody conditions
as above. now we have a default number of feeds to output when there's no
cached history.
2010-04-25 00:34:31 -04:00
James Vega
eff20402ac Change canonize => normalize
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-08-27 16:41:34 -04:00
James Vega
54dda880d4 Add a pluginCommands kwarg to Commands.listCommands.
Plugins which implement their own listCommands method should pass their
non-typically discoverable commands to Commands.listCommands via this
mechanism.  This means that the de-duplication of commands is performed in one
spot instead of having each plugin implement it on their on in their
listCommands method.

This reverts commits 0ce829af6215b97e725f4d6d580d1151950be869 and
09fb0e6fc974445a3414fb03a94625f8538d4570.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-05-22 18:52:15 -04:00
James Vega
7b906fbf54 Fully qualify the call to universalImport so we don't get a NameError.
Also added some ImportError handlers to aid letting users of distribution
packages know if the distributions strip out the local module copies.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-03-19 20:33:17 -04:00
James Vega
3fd92e7f24 RSS: Move feedparser to the local directory
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-03-11 19:43:23 -04:00
Jeremy Fincher
5076241bcd Fixed RSS to include a feedparser which doesn't override the globally-installed one, using universalImport. 2009-03-11 14:06:16 -04:00
James Vega
62acb0c998 Make RSS.announce a nested plugin
This simplifies the interface for dealing with announced feeds by providing
separate "announce add", "announce remove", "announce list" commands as
opposed to the one "announce" command with confusing syntax.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-02-28 00:10:10 -05:00
James Vega
0e05fe67c7 No need to define RSS.listCommands
Doing so can lead to RSS feeds being listed multiple times.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-02-27 23:36:29 -05:00
James Vega
9cc917bd36 Guard feedparser import and warn the user if it's not available 2009-03-11 13:37:26 -04:00
James Vega
7bb3b1ec60 Merge the changes from the python25compat branch into the trunk 2009-03-11 13:37:24 -04:00
James Vega
8111c48a49 Switch from rssparser 3.3 to (the renamed) feedparser 4.1 in order to properly handle Bugzilla RSS feeds. 2007-05-09 02:30:18 +00:00
James Vega
4064da76aa plugins/RSS: UTF-8 encode the RSS' URL to prevent an encoding error when including the URL in the reply. 2006-08-28 13:18:12 +00:00
James Vega
73d7fd3a2d plugins/RSS/plugin.py: Refactor the character encoding detection and conversion so it can be used in more places. Hopepfully this will cause fewer UnicodeDecodeErrors. 2006-05-02 12:22:11 +00:00
James Vega
a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega
1edb4e13bf plugins/RSS: Colons are pretty. 2005-07-22 12:20:11 +00:00
James Vega
45c7f773ac plugins/RSS: Fix info so it works again. 2005-07-19 21:32:54 +00:00
James Vega
14dc0c0201 plugins/RSS: If the feed supplies an encoding, encode the headlines with it. 2005-07-07 14:13:53 +00:00
Jeremy Fincher
490fb0b140 Changed prefixName to prefixNick, which is more appropriate, and has always bothered me. Better now than later. 2005-06-01 21:08:30 +00:00
Jeremy Fincher
4da1f38235 Fixed the security issue nicktastic reported, and made sure that if, for some reason, an error doesn't raise an exception, it still stops the calling of the command function. 2005-06-01 20:13:33 +00:00
James Vega
34f5be5252 RSS: Use del instead of pop. 2005-05-30 02:27:16 +00:00
James Vega
7fdb72ab4f plugins/RSS: Stop using attributes for feeds. 2005-05-30 00:52:54 +00:00
James Vega
85e9035535 plugins/RSS: Fix a bug in RSS.remove. 2005-05-29 16:54:57 +00:00
James Vega
994cff5c05 Update a couple missed changeFunctionName calls. 2005-05-16 03:25:43 +00:00