Commit Graph

304 Commits

Author SHA1 Message Date
Valentin Lorentz
0e8bfecf37 utils.structures: Define __slots__ for more classes. 2015-12-02 12:22:24 +01:00
Valentin Lorentz
57b77a6725 utils.web: Rename getUrl to getUrlContent and add getUrlTargetAndContent. 2015-10-25 16:19:49 +01:00
Valentin Lorentz
e3ff413734 Web & core: Merge features of Web's title parser and utils.web.HtmlToText + don't unescape HTML twice. Closes GH-1176. 2015-10-23 07:41:36 +02:00
Jussi Timperi
df7689cc2e Web & utils.web: Force HTMLParser to process all buffered data.
Python issue 23144.
2015-10-22 16:56:53 +03:00
Valentin Lorentz
85ee37c162 Remove unbreakable space. 2015-09-29 17:38:07 +02:00
Valentin Lorentz
3047ea7acb Add warning in setup.py about Python 2.6. 2015-09-20 15:50:43 +02:00
Valentin Lorentz
879ce460e9 make utils.web.urlencode return a string, not bytes. 2015-09-09 22:02:19 +02:00
Valentin Lorentz
912d2e1538 core: Apply some suggestions of pyflakes. 2015-08-31 18:19:22 +02:00
Valentin Lorentz
c01a956a8b utils: Remove dependency on parent package. 2015-08-30 17:33:39 +02:00
Valentin Lorentz
c587900c6f Make normalizeWhitespace remove \r as well. 2015-08-30 15:44:07 +02:00
Valentin Lorentz
a905828b26 Fix Python 2.6 compatibility. 2015-08-26 17:59:50 +02:00
Valentin Lorentz
d1acdb6f92 Fix timezone handling in Python 2 and 3.2. 2015-08-26 17:42:42 +02:00
Valentin Lorentz
c3a2c800f1 Remove need for 2to3. 2015-08-11 16:50:23 +02:00
Valentin Lorentz
f305ff3c02 Remove fix_basestring. 2015-08-10 20:24:11 +02:00
Valentin Lorentz
2c380a4c3a Remove fix_xrange. 2015-08-10 19:36:07 +02:00
Valentin Lorentz
be6bc1a734 Remove need for fix_unicode. 2015-08-10 18:52:51 +02:00
Valentin Lorentz
c0ac84bb53 Remove need for fix_import, fix_types, and fix_urllib. 2015-08-10 17:55:25 +02:00
Valentin Lorentz
216c5d213f Replace sys.version_info[0] usages with minisix.PY{2,3}. 2015-08-09 00:23:03 +02:00
Valentin Lorentz
5a82cefd51 Remove use of fix_long fixer. 2015-08-08 23:08:49 +02:00
Valentin Lorentz
be118c3338 Remove use of fix_dict fixer. 2015-08-08 23:08:49 +02:00
Valentin Lorentz
eca7a036aa Socket: Move line decoding to utils.str. 2015-05-16 00:30:20 +02:00
Valentin Lorentz
ed37fb6646 String & core: Add support for G flag in matchers. Closes GH-1072. 2015-05-15 16:48:00 +02:00
Valentin Lorentz
3f9118a1a6 utils.str.format: Add support for sets for %L + add test case. Closes GH-1104 2015-05-14 07:14:29 +00:00
Valentin Lorentz
7ac053d8f1 Merge branch 'rewrite-rss' into testing 2014-07-31 22:53:44 +02:00
Valentin Lorentz
00e25f86d8 Implement TruncatableSet.__repr__. 2014-07-31 22:53:21 +02:00
Valentin Lorentz
5f582e3f52 Make AtomicFile support context managers. 2014-07-31 22:53:21 +02:00
Valentin Lorentz
4a3d39e747 Merge branch 'rewrite-rss' into testing
Conflicts:
	src/utils/str.py
2014-07-31 19:24:01 +00:00
Valentin Lorentz
2f327ab9c1 Fix Python 2.6 support (broken in the two last commits). 2014-07-31 18:06:20 +00:00
Valentin Lorentz
8be1671867 Fix unicode handling issue of utils.str.format on Python 2. 2014-07-31 19:57:26 +02:00
Valentin Lorentz
8010cd5ae2 Fix unicode handling issue of utils.str.format on Python 2. 2014-07-31 19:57:14 +02:00
Valentin Lorentz
c51876b4d6 Allow TruncatableSet to get no argument. 2014-07-31 18:43:40 +02:00
Valentin Lorentz
94fbea6266 Add utils.structures.TruncatableSet. 2014-07-31 14:24:47 +02:00
Valentin Lorentz
ab011a6618 ShrinkUrl & core: Fix encoding issue related to getUrlFd. 2014-07-16 05:42:46 +00:00
Valentin Lorentz
9210aa1a33 utils.web.getUrl: add the 'timeout' argument (and fix the doc of getUrlFd). 2014-04-26 12:43:45 +00:00
Valentin Lorentz
28536f0528 Fix utils.gen.safeEval to work with instances of ast.NameConstant (introduced in Pythn 3.4). 2014-04-19 10:17:14 +02:00
Valentin Lorentz
77e6de6bba Fix errors detected by PyLint. 2014-03-05 14:14:36 +01:00
Valentin Lorentz
dcb25db6e5 Add support for SRV records in domain name resolution. 2014-03-01 09:22:14 +01:00
Valentin Lorentz
108b0de0d1 Add support of IP-binding in non-IRC connections. 2014-03-01 09:06:21 +01:00
Valentin Lorentz
c872cd793f Fix compatibility with Pypy (does not support __closure__ attribute for functions). 2014-01-26 21:19:28 +01:00
Valentin Lorentz
06fdaa792f Continue accelerating the 2to3 step (remove fix_map). 2014-01-21 10:57:38 +01:00
Valentin Lorentz
1fbdedc7e0 Continue accelerating the 2to3 step (remove fix_funcattrs, fix_itertools, and fix_itertools_imports). 2014-01-21 10:50:55 +01:00
Valentin Lorentz
35a62b4e77 Continue accelerating the 2to3 step (remove fix_ws_comma, fix_xreadlines, and fix_zip). 2014-01-21 10:40:18 +01:00
Valentin Lorentz
ca419f6485 Continue accelerating the 2to3 step (remove fix_reduce, fix_renames, fix_repr, fix_set_literal, fix_standarderror, fix_sys_exc, fix_throw, and fix_tuple_params). 2014-01-20 16:04:53 +01:00
Valentin Lorentz
bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
Valentin Lorentz
2fda69b4d6 Continue accelerating the 2to3 step (remove fix_raise). 2014-01-20 15:43:55 +01:00
Valentin Lorentz
c1c23f66f4 Continue accelerating the 2to3 step (remove fix_print). 2014-01-20 15:31:09 +01:00
Valentin Lorentz
529b8f8d99 Continue accelerating the 2to3 step (remove fix_nonzero, fix_operator, and fix_paren). 2014-01-20 15:19:06 +01:00
Valentin Lorentz
d1649a44ac Continue accelerating the 2to3 step (remove fix_idioms, fix_imports2, fix_input, fix_intern, fix_isinstance, fix_ne, fix_next, and fix_raw_input). 2014-01-20 15:13:01 +01:00
Valentin Lorentz
4652c9ce51 Start accelerating the 2to3 step (remove fix_apply, fix_buffer, fix_callable, fix_exec, fix_execfile, fix_exitfunc, fix_filter, fix_funcattrs, fix_future, fix_getcwdu, and fix_has_key). 2014-01-20 14:49:47 +01:00
nyuszika7h
0e480b4e52 Fix file descriptor leaks 2014-01-03 17:44:01 +01:00
Valentin Lorentz
89fb4fb3b2 Fix stack frames display for objects without __dict__ attribute. 2013-12-25 16:08:07 +00:00
Valentin Lorentz
84cad71cf5 Fix inspection of the stack if the 'self' object has no __dict__. 2013-12-15 19:30:20 +00:00
Valentin Lorentz
f894983b55 Add inspection of self.* in debug locals dump. 2013-12-05 12:35:56 +00:00
Valentin Lorentz
42b8a0676c utils/web.py: Attempt to fix handling of RSS feeds using broken entity references. 2013-11-26 15:15:07 +00:00
Valentin Lorentz
40675ffdfa Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	plugins/Admin/plugin.py
	plugins/BadWords/config.py
	plugins/Filter/plugin.py
	plugins/Google/plugin.py
	plugins/Math/plugin.py
	plugins/Misc/plugin.py
	plugins/Note/plugin.py
	plugins/RSS/plugin.py
	plugins/Seen/test.py
	plugins/ShrinkUrl/config.py
	plugins/ShrinkUrl/plugin.py
	plugins/ShrinkUrl/test.py
	plugins/Status/plugin.py
	plugins/String/config.py
	plugins/String/plugin.py
	plugins/Time/plugin.py
	plugins/Todo/plugin.py
	plugins/Web/plugin.py
	plugins/__init__.py
	scripts/supybot-botchk
	setup.py
	src/__init__.py
	src/callbacks.py
	src/commands.py
	src/conf.py
	src/drivers/Socket.py
	src/ircdb.py
	src/irclib.py
	src/ircutils.py
	src/questions.py
	src/registry.py
	src/schedule.py
	src/test.py
	src/utils/file.py
	src/utils/gen.py
	src/utils/net.py
	src/utils/web.py
	src/world.py
2013-08-24 11:28:29 +02:00
James McCoy
88e4f73777 getSocket: Use returned family to create the socket
The existing code was parsing the passed in host to determine what type
of socket family to create.  getaddrinfo already provides this for us,
so there's no need to perform our own, potentially buggy, parsing.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2013-08-22 23:40:28 -04:00
Valentin Lorentz
e3acc03fd6 Another fix for b46a0dd6a2. 2013-08-17 19:27:52 +02:00
Valentin Lorentz
f3ecc33678 Fix commit b46a0dd (breaks tests for Alias and Aka). 2013-08-17 17:08:55 +02:00
Kill Your TV
b46a0dd6a2 Unicode fixes for python 2.x
These changes have been tested with Python 3.2.3 and Python 2.7.5.
2013-08-17 14:12:10 +00:00
Valentin Lorentz
dd37f8dd3f Socket: Forward DNS queries via the socks proxy (if any). 2013-08-17 15:47:39 +02:00
Valentin Lorentz
8d7b06efd1 utils.str.format: Replace %u with '' instead of '<>' if URL is empty. 2013-08-13 19:19:54 +00:00
Valentin Lorentz
1e050d7fa8 utils/str.py: Use already existing function utils.gen.timeElapsed instead of redundant one. 2013-08-11 12:08:01 +02:00
Valentin Lorentz
8b4b45ecf0 utils/str.py: Add the %T (Time delta) argument type. 2013-08-11 12:03:41 +02:00
Valentin Lorentz
c23f25545f Fix core tests for Python 3 (actually imported + fixes). 2013-08-10 15:18:04 +02:00
Valentin Lorentz
4a265a6ce9 Enhance representation of InsensitivePreservingDict objects. 2013-07-13 11:20:02 +02:00
Valentin Lorentz
771b739af7 utils.web.getEncoding: use <meta charset /> if available. 2013-07-09 12:40:42 +00:00
Valentin Lorentz
b4402b28ed utils.web: Rename get_encoding to getEncoding for consistency. 2013-07-09 12:05:51 +00:00
Valentin Lorentz
d4df5de91d utils.web: Add function get_encoding. 2013-07-09 12:02:25 +00:00
Carsten Teibes
7129dc2697 Switch to charade, which is better maintained and works under 2 and 3 2013-06-27 19:36:44 +02:00
Valentin Lorentz
774158f2ec utils/web.py: Fix compatibility with Python 3. 2013-06-17 04:10:29 +00:00
Valentin Lorentz
cca1e6dba5 utils/web.py: Add some 'try' statements to make decoding as error-permissive as possible. 2013-06-13 17:22:33 +00:00
Valentin Lorentz
2eec980c8e utils/web.py: Python 3 should use decode() only if it is a bytes object. 2013-06-13 04:22:20 +00:00
Valentin Lorentz
a82ad9cb1a utils/web.py: Properly import chardet. 2013-06-12 17:38:50 +00:00
Valentin Lorentz
e24ab4501d Fix bug with Python 3.3 (?) 2013-06-06 05:49:25 +00:00
Valentin Lorentz
2016983d35 Add supybot.utils.net.getAddressFromHostname() and improve Socket driver to try successively all IP addresses. 2013-05-31 17:21:10 +02:00
Valentin Lorentz
cefd91d4f1 Fix encoding issues with Alias containing non-ascii characters. 2013-05-31 15:42:57 +02:00
Valentin Lorentz
ad25f17639 RSS: Add Python 3 support. 2013-05-29 15:16:23 +02:00
Valentin Lorentz
6ffcf122a7 Remove code specific to Python 2.{3,4,5}. 2013-05-13 15:27:26 +02:00
Valentin Lorentz
8d95a424b0 Fix fetch of RSS feeds containing unicode. 2013-05-13 10:51:24 +02:00
Valentin Lorentz
caf255afd9 Internationalize string-handling functions in src/utils/.
This commit was supposed to be before the two previous ones, but I messed up with Git.
2013-04-21 16:00:31 +02:00
Valentin Lorentz
916a85c965 core: Various Python 3 compatibility fixes. 2013-03-25 19:15:08 +01:00
Valentin Lorentz
d753df9d0c Print extra debug data in the logs. 2013-02-03 17:46:39 +01:00
Valentin Lorentz
77f614be86 utils.web.HtmlToText: Convert html entities. 2013-02-01 20:50:46 +01:00
Valentin Lorentz
6c0497d34e Fix traceback when utils.str.dqrepr is given a unicode object. 2013-01-23 13:53:37 +01:00
Valentin Lorentz
fe39d35b2f core: Fix normalizeWhitespace handling of new lines. 2013-01-04 22:28:44 +01:00
Valentin Lorentz
28dc3dd3ac core: Fix normalizeWhitespace. 2013-01-04 20:25:25 +01:00
Valentin Lorentz
3dba9088b0 Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	INSTALL
	plugins/ChannelLogger/README.txt
	plugins/ChannelStats/README.txt
	plugins/Google/plugin.py
	plugins/Google/test.py
	plugins/Plugin/test.py
	plugins/Web/test.py
	setup.py
	src/callbacks.py
	src/ircdb.py
	src/irclib.py
	src/utils/str.py
	test/test_irclib.py
2013-01-01 21:11:24 +01:00
James McCoy
295f9b1f0d Make utils.str.soundex perform better when length is large
Closes: Sf patch#148
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-12-31 17:29:02 -05:00
Valentin Lorentz
cab1830140 utils.structures: Prevent a RingBuffer from being bigger than the new size, when calling resize(). 2012-12-27 12:22:14 +00:00
James McCoy
90b7f3cd4e utils.web: Simplify getUrlFd by using standard parse methods
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-23 17:06:30 -04:00
Valentin Lorentz
85eb0b4b0d Fix 3a78300971. 2012-10-23 16:15:13 +00:00
James McCoy
6ab807be43 utils.net: Use _ipAddr and _domain from utils.web to define emailRe
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-20 19:43:11 -04:00
Valentin Lorentz
db3746d122 Add support for authentication scheme.
This commit closes http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=690879

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-20 19:42:15 -04:00
Valentin Lorentz
7d87d422e1 Fix detection of .42 domains
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-20 19:15:05 -04:00
Valentin Lorentz
3a78300971 Add support for authentication scheme.
This commit closes http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=690879
2012-10-20 20:23:32 +02:00
Daniel Folkinshteyn
f310350335 Web: add 'timeout' config for web fetch, default 5 sec.
Otherwise, when a site would take a long time to respond, the thread would hang for quite a while.

also needed to mod src/utils/web.py to take the timeout arg.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-15 22:10:56 -04:00
Daniel Folkinshteyn
6025f7364c core: force inet_aton argument to string to prevent occasional error on reconnect.
it /should/ always be a string anyway, but sometimes things break with
a TypeError that it is an int instead of the expected string and hangs
up the bot.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-10-14 12:21:56 -04:00
Valentin Lorentz
156b3d93e1 Socket driver: Add support for Socks proxies. 2012-10-07 13:13:08 +02:00
James McCoy
cef93a6cfd Use relative imports for all packages under the supybot namespace
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-17 22:12:11 -04:00
James McCoy
82ecf36fcd Update my name/contact information
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-01 10:16:48 -04:00