Commit Graph

26 Commits

Author SHA1 Message Date
Daniel Folkinshteyn f3e5223f3f Undo the web title fix because it appears it broke more than it fixed. 2011-11-21 15:14:34 -05:00
Daniel Folkinshteyn b533290c7a Web: fix problems with title snarfer and unicode due to bug in HTMLParser in python 2.6+
Upstream bug: http://bugs.python.org/issue3932
Rather than override the unescape method with the patch posted, we just convert the page
text to unicode before passing it to the HTMLParser. UTF8 and Latin1 will eat just about
anything.
2011-10-11 13:06:27 -04:00
Daniel Folkinshteyn 2b708f034b Web: add 'timeout' config for web fetch, default 5 sec.
Otherwise, when a site would take a long time to respond, the thread would hang for quite a while.

also needed to mod src/utils/web.py to take the timeout arg.
2011-06-13 16:42:57 -04:00
James Vega 288d7c6e02 Update plugins to ignore all non-ACTION CTCP messages.
Also update commands.urlSnarfer to do the same, which allows us to revert
"Don't bother snarfing URLs from non-Action CTCP messages."

This reverts commit 3282e3407e.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-01-28 08:14:44 -05:00
Jeremy Fincher 3282e3407e Don't bother snarfing URLs from non-Action CTCP messages. 2010-01-28 06:35:53 -06:00
James Vega 25fc2de643 utils.web: Provide access to the raw httpUrlRe/urlRe strings
Using the compiled regexps for a PluginRegexp method's __doc__ doesn't work.

Closes Sourceforge #2879862

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-10-15 22:16:38 -04:00
James Vega ca917d3528 Use utils.web.httpUrlRe for the Web/ShrinkUrl snarfer regexes.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-10-04 21:41:05 -04:00
James Vega 0a4c66071b Adjust testTitle to take into account /.'s slight change 2009-03-11 13:37:26 -04:00
James Vega cbc91c6a26 Use a more appropriate message if the URL definitely has no title. 2009-03-11 13:37:25 -04:00
James Vega 74e06ea52a Catch the proper exception when parsing the title fails. 2009-03-11 13:37:24 -04:00
James Vega ee9aaa89d6 plugins/Web: Swtich the title parser back to HTMLParser sing sgmllib's parser spins on invalid input. 2006-09-13 19:40:51 +00:00
James Vega 855453a743 plugins/Web/test.py: Bump the timeout since we're fetching a lot of information from other sources, which may take a while. 2006-05-03 13:30:27 +00:00
James Vega a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega 9d48f2c879 plugins/Web: Update the exception handling for the change in parsers. 2005-09-20 19:06:35 +00:00
James Vega b375ea9792 plugins/Web: Fixed the title-retrieval parser to actually retrieve the entire title. 2005-07-19 13:55:37 +00:00
James Vega bc1451e898 plugins/Web: Encountering an HTMLParser exception doesn't mean the title hasn't already been snarfed. Don't bail right away. 2005-06-29 19:05:20 +00:00
Jeremy Fincher 490fb0b140 Changed prefixName to prefixNick, which is more appropriate, and has always bothered me. Better now than later. 2005-06-01 21:08:30 +00:00
James Vega 06800f9fc7 Correctly catch the HTMLParseError 2005-05-07 03:55:14 +00:00
James Vega 47179f8bc6 Catch HTMLParserErrors when we're trying to grab the <title>. 2005-05-07 03:24:10 +00:00
James Vega fcfda73f64 Bug #1190350, Don't grab fake title. 2005-04-30 12:53:42 +00:00
James Vega 9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
Jeremy Fincher 7e441285c7 Added the Web.fetch command. 2005-03-14 02:44:55 +00:00
Jeremy Fincher a2e2063d8b Added a callCommand to the Web plugin to catch utils.web.Error. 2005-03-09 07:26:32 +00:00
Jeremy Fincher b0cb616709 Changed callbacks.Privmsg to be callbacks.Plugin, and callbacks.PrivmsgCommandAndRegexp to be callbacks.Plugin. 2005-02-09 07:04:04 +00:00
James Vega 92839a94e7 Remove supybot.privmsgs imports. 2005-02-01 13:57:14 +00:00
Jeremy Fincher 0c2da03a67 Added the Web plugin (from pieces of Http, Fun, and URL) in the new plugin format. 2005-02-01 09:41:54 +00:00