Commit Graph

38 Commits

Author SHA1 Message Date
Valentin Lorentz
bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
Valentin Lorentz
2fda69b4d6 Continue accelerating the 2to3 step (remove fix_raise). 2014-01-20 15:43:55 +01:00
Valentin Lorentz
42b8a0676c utils/web.py: Attempt to fix handling of RSS feeds using broken entity references. 2013-11-26 15:15:07 +00:00
Valentin Lorentz
40675ffdfa Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	plugins/Admin/plugin.py
	plugins/BadWords/config.py
	plugins/Filter/plugin.py
	plugins/Google/plugin.py
	plugins/Math/plugin.py
	plugins/Misc/plugin.py
	plugins/Note/plugin.py
	plugins/RSS/plugin.py
	plugins/Seen/test.py
	plugins/ShrinkUrl/config.py
	plugins/ShrinkUrl/plugin.py
	plugins/ShrinkUrl/test.py
	plugins/Status/plugin.py
	plugins/String/config.py
	plugins/String/plugin.py
	plugins/Time/plugin.py
	plugins/Todo/plugin.py
	plugins/Web/plugin.py
	plugins/__init__.py
	scripts/supybot-botchk
	setup.py
	src/__init__.py
	src/callbacks.py
	src/commands.py
	src/conf.py
	src/drivers/Socket.py
	src/ircdb.py
	src/irclib.py
	src/ircutils.py
	src/questions.py
	src/registry.py
	src/schedule.py
	src/test.py
	src/utils/file.py
	src/utils/gen.py
	src/utils/net.py
	src/utils/web.py
	src/world.py
2013-08-24 11:28:29 +02:00
Valentin Lorentz
771b739af7 utils.web.getEncoding: use <meta charset /> if available. 2013-07-09 12:40:42 +00:00
Valentin Lorentz
b4402b28ed utils.web: Rename get_encoding to getEncoding for consistency. 2013-07-09 12:05:51 +00:00
Valentin Lorentz
d4df5de91d utils.web: Add function get_encoding. 2013-07-09 12:02:25 +00:00
Carsten Teibes
7129dc2697 Switch to charade, which is better maintained and works under 2 and 3 2013-06-27 19:36:44 +02:00
Valentin Lorentz
774158f2ec utils/web.py: Fix compatibility with Python 3. 2013-06-17 04:10:29 +00:00
Valentin Lorentz
cca1e6dba5 utils/web.py: Add some 'try' statements to make decoding as error-permissive as possible. 2013-06-13 17:22:33 +00:00
Valentin Lorentz
2eec980c8e utils/web.py: Python 3 should use decode() only if it is a bytes object. 2013-06-13 04:22:20 +00:00
Valentin Lorentz
a82ad9cb1a utils/web.py: Properly import chardet. 2013-06-12 17:38:50 +00:00
Valentin Lorentz
ad25f17639 RSS: Add Python 3 support. 2013-05-29 15:16:23 +02:00
Valentin Lorentz
8d95a424b0 Fix fetch of RSS feeds containing unicode. 2013-05-13 10:51:24 +02:00
Valentin Lorentz
77f614be86 utils.web.HtmlToText: Convert html entities. 2013-02-01 20:50:46 +01:00
Valentin Lorentz
3dba9088b0 Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	INSTALL
	plugins/ChannelLogger/README.txt
	plugins/ChannelStats/README.txt
	plugins/Google/plugin.py
	plugins/Google/test.py
	plugins/Plugin/test.py
	plugins/Web/test.py
	setup.py
	src/callbacks.py
	src/ircdb.py
	src/irclib.py
	src/utils/str.py
	test/test_irclib.py
2013-01-01 21:11:24 +01:00
James McCoy
90b7f3cd4e utils.web: Simplify getUrlFd by using standard parse methods
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-23 17:06:30 -04:00
Valentin Lorentz
85eb0b4b0d Fix 3a78300971. 2012-10-23 16:15:13 +00:00
Valentin Lorentz
db3746d122 Add support for authentication scheme.
This commit closes http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=690879

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-20 19:42:15 -04:00
Valentin Lorentz
7d87d422e1 Fix detection of .42 domains
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-20 19:15:05 -04:00
Valentin Lorentz
3a78300971 Add support for authentication scheme.
This commit closes http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=690879
2012-10-20 20:23:32 +02:00
Daniel Folkinshteyn
f310350335 Web: add 'timeout' config for web fetch, default 5 sec.
Otherwise, when a site would take a long time to respond, the thread would hang for quite a while.

also needed to mod src/utils/web.py to take the timeout arg.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-15 22:10:56 -04:00
James McCoy
cef93a6cfd Use relative imports for all packages under the supybot namespace
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-17 22:12:11 -04:00
James McCoy
82ecf36fcd Update my name/contact information
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-01 10:16:48 -04:00
Valentin Lorentz
3dc8e04195 utils.web.urlencode: Encode data into bytes. 2012-08-05 16:57:40 +02:00
Valentin Lorentz
c4dfa55d65 Use HTMLParser instead of deprecated sgmllib in utils.web. 2012-08-04 18:02:45 +02:00
Valentin Lorentz
4649188b96 Fix detection of .42 domains 2011-01-02 13:22:54 +01:00
James Vega
f03a3f6c85 utils/web.py: Only try catching socket.sslerror if built with SSL support
Closes: Sf#2998820

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2010-05-24 15:44:25 -04:00
James Vega
25fc2de643 utils.web: Provide access to the raw httpUrlRe/urlRe strings
Using the compiled regexps for a PluginRegexp method's __doc__ doesn't work.

Closes Sourceforge #2879862

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-10-15 22:16:38 -04:00
James Vega
c5896ff6cb utils.web: Import urllib's urlencode into our namespace.
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-09-29 19:54:07 -04:00
James Vega
de79e679f4 Add a data kwarg to getUrl(Fd) a la urllib2.Request.
This allows plugins to easily make POST requests using our getUrl(Fd)
functions.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
2009-09-16 01:17:33 -04:00
James Vega
3d5c877f80 Finally fix the XXX we had for httpUrlRe and urlRe 2009-03-11 13:37:26 -04:00
James Vega
a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega
9cb1669b5f src/utils/web: Add another "UNKNOWN_HOST" code to catch. 2005-10-14 12:59:47 +00:00
James Vega
9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
Jeremy Fincher
c598e4cca9 Added a few more UNKNOWN_HOST types. 2005-03-14 02:32:48 +00:00
Jeremy Fincher
5ca79f0f49 Hah, changed WebError to Error everywhere but the source :) 2005-02-02 14:07:20 +00:00
Jeremy Fincher
5fd6bbb52d Completely restructured our utils modules.
Tons of changes.  Here's the summary of things that matter most:

* There is no more supybot.fix.
* There is no more supybot.webutils; now there is supybot.utils.web.
  * It's no longer webutils.WebError, but just utils.web.Error.
* You shouldn't import itertools, ideally, but instead import utils.iter.
* No more using imap/ifilter in commands unless absolutely necessary.  It's
  premature optimization and annoying.
* utils.str.format isn't quite ready yet, but will be soon.  That'll be the
  next big thing to fix in our code.
2005-01-27 06:59:08 +00:00