Mikaela
/
Limnoria
mirror of https://github.com/Mikaela/Limnoria.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
11,856 Commits 3 Branches 123 Tags 40 MiB
Commit Graph

172 Commits

Author SHA1 Message Date
Valentin Lorentz 820113344c Web: Use utils.web.get_encoding for guessing charset. 2013-07-09 12:02:43 +00:00
Valentin Lorentz dbedcbb6c9 Web: Disable testWhitelist if --no-network is given. 2013-07-09 09:18:33 +02:00
Valentin Lorentz 5f1535447c Web: Use @title's utf8 decoding in the snarfer. 2013-07-02 13:42:53 +02:00
Daniel Folkinshteyn 944f9c3e3f Web: create a cofigurable url whitelist 
Prevent various forms of abuse that result via the Web plugin, such as fetching or titling
malicious content, or revealing bot IP.

Conflicts:
	plugins/Web/plugin.py
	plugins/Web/test.py
 2013-06-27 07:09:22 +02:00
George Miller 0150c79924 Added a way to have the urlsnarfer report exceptions (hotsnotnound, ...) 
(Should be possible changed to only ioExceptions)
Enable/Disable via 'supybot.plugins.Web.snarferReportIOExceptions'
 2013-04-05 10:05:00 +02:00
Valentin Lorentz 9ef83f70cf Web: Fix encoding in @title. 2013-03-06 12:11:46 +00:00
Valentin Lorentz 693e827340 Alias & ChannelLogger & MessageParser & MoobotFactoids & User & Web: Update l10n-fr. 2013-03-02 19:15:12 +01:00
Terje Hoås 2327317b33 Web: Fix fetch. Use getUrl instead of getUrlFd. 
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
 2013-01-08 20:32:43 -05:00
Valentin Lorentz 603f44129d Web: Fix Python 3 compatibility. 2013-01-06 17:06:26 +01:00
Valentin Lorentz 2177429618 Web: Remove netcraft (which does not seem to want bots). 2013-01-05 19:14:58 +01:00
Valentin Lorentz 5f1d7cc097 Web: Fix test of snarfing (Microsoft changes the title based on location of the computer). 2013-01-05 18:03:13 +01:00
Valentin Lorentz 918092a54d Web: Fix snarfing of titles with UTF-8 characters. 2013-01-05 18:02:35 +01:00
Valentin Lorentz 3dba9088b0 Merge remote-tracking branch 'supybot/master' into testing 
Conflicts:
	INSTALL
	plugins/ChannelLogger/README.txt
	plugins/ChannelStats/README.txt
	plugins/Google/plugin.py
	plugins/Google/test.py
	plugins/Plugin/test.py
	plugins/Web/test.py
	setup.py
	src/callbacks.py
	src/ircdb.py
	src/irclib.py
	src/utils/str.py
	test/test_irclib.py
 2013-01-01 21:11:24 +01:00
Valentin Lorentz 6cbc6a14dd Rename all 'locale' directories to 'locales'. 
Firstly, these directories contains more than one locale. Secondly, it conflicts with the 'locale' python module.
 2012-12-26 15:43:35 +01:00
Valentin Lorentz 943f3a0133 Web: Fix test for @title. 2012-12-12 19:24:23 +01:00
Valentin Lorentz 22febc4a20 Web: Fix encoding issues in title snarfing and @title. 2012-11-17 15:10:36 +00:00
Valentin Lorentz 6ea2d062b7 Web: Filter special chars in @title, and add --no-filter. 
I'm adding --no-filter just in case someone want to use @title to do this at purpose
 2012-10-31 16:35:51 +00:00
Daniel Folkinshteyn f310350335 Web: add 'timeout' config for web fetch, default 5 sec. 
Otherwise, when a site would take a long time to respond, the thread would hang for quite a while.

also needed to mod src/utils/web.py to take the timeout arg.

Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
 2012-10-15 22:10:56 -04:00
Terje Hoås cb623b2f4e Web: Fix fetch. Use getUrl instead of getUrlFd. 2012-10-02 18:19:53 +02:00
Valentin Lorentz ad3bf1302f Web: Fix compatibility with Python <= 2.6. 2012-09-22 17:43:59 +00:00
Valentin Lorentz a95546316f Web: Fix title snarfer with titles containing unicode characters. 2012-09-22 17:34:33 +00:00
Valentin Lorentz 841dc719e1 Web: Fix page decoding. 2012-09-19 15:08:52 +00:00
James McCoy d73a801d8f Web: Remove/update non-functional tests 
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
 2012-09-05 21:35:56 -04:00
James McCoy 82ecf36fcd Update my name/contact information 
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
 2012-09-01 10:16:48 -04:00
Valentin Lorentz 8507498be4 Web: make plugin py3k-friendly. 2012-08-04 22:23:40 +02:00
Valentin Lorentz 86f541e6db Revert "Web: Fix unicode problems for titleSnarfer." 
This reverts commit 3579701d2d.
 2012-05-11 18:10:23 +02:00
Terje Hoås 3579701d2d Web: Fix unicode problems for titleSnarfer. 2012-05-04 12:11:53 +02:00
Mika Suomalainen b688eabd17 Web: update l10n-de. 2012-04-27 15:47:28 +03:00
skizzhg aedfc07762 Update l10n-it. 2012-03-16 18:28:05 +01:00
Mika Suomalainen 837b7e89b1 Web: update l10n-fi. 2012-03-15 20:48:51 +08:00
Valentin Lorentz 44c10a0b49 Web: Update message.pot. 2012-02-16 17:17:23 +01:00
Valentin Lorentz 3a59601d29 Web: Fix supybot.plugins.Web.nonSnarfingRegexp and update l10n-fr and l10n-it. 2012-01-28 20:40:45 +01:00
Mika Suomalainen edf3851691 Web: add l10n-fi. 2011-12-22 15:24:20 +02:00
Tanner a913f4cf25 Web: Added a description to README.txt 2011-11-06 21:26:55 +08:00
Valentin Lorentz 8add66511e Update l10n-fr. 2011-10-29 16:52:29 +02:00
Valentin Lorentz eefddc76f0 Merge pull request #249 from fbesser/fbesser 
german translations
 2011-10-27 04:18:07 -07:00
skizzhg 51e02849ad Web: Fix l10n-it. 2011-10-27 10:47:25 +02:00
Florian Besser ffb19f72a6 Web: Add l10n-de 2011-10-26 12:10:30 +02:00
Florian Besser 74cd35f772 Web: localized 'Title:' in titleSnarfer 2011-10-26 11:25:51 +02:00
skizzhg 1af08c27d7 Admin & Channel & Games & Math & MessageParser & Misc & MoobotFactoids & Network & Nickometer & Owner & Praise & Relay & Reply & Topic & Unix & Web: update/add l10n-it. 2011-08-06 18:55:38 +02:00
Valentin Lorentz 6ed15bafe4 Web: Fix use of %S converter. 2011-07-01 17:07:54 +02:00
Valentin Lorentz f0571e434b AutoMode & BadWord & Config & Format & Karma & Later & Web: Add/update l10n-it. 2011-06-30 09:56:25 +02:00
Valentin Lorentz aef0b6865d Filter & Format & Later & Plugin & RSS & Reply & Status & Topic & Unix & Web: update French translation 2011-02-26 11:19:32 +01:00
Valentin Lorentz 0d7e3d3759 (Re)generate all plugin's .pot file 2011-02-26 10:40:50 +01:00
Valentin Lorentz 5e09a9c887 Merge branch 'l10n-fr' into testing 2010-10-30 12:11:29 +02:00
Valentin Lorentz df3459dc3b Localize Sucess, Time, Todo, Topic, Url, Unix, User, Utilities, and Web into French 2010-10-30 12:10:31 +02:00
Valentin Lorentz 8be4aaeff1 Merge branch 'readability' 2010-10-23 10:38:52 +02:00
Valentin Lorentz 6a9a8d81b8 Internationalize Time, Todo, Topic, URL, Unix, Utilities, and Web 2010-10-20 09:39:44 +02:00
Valentin Lorentz b47c7c92a3 Edit Status and Web plugins in order to use %S 2010-10-16 11:37:58 +02:00
James Vega 288d7c6e02 Update plugins to ignore all non-ACTION CTCP messages. 
Also update commands.urlSnarfer to do the same, which allows us to revert
"Don't bother snarfing URLs from non-Action CTCP messages."

This reverts commit 3282e3407e.

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
 2010-01-28 08:14:44 -05:00
Jeremy Fincher 3282e3407e Don't bother snarfing URLs from non-Action CTCP messages. 2010-01-28 06:35:53 -06:00
James Vega 25fc2de643 utils.web: Provide access to the raw httpUrlRe/urlRe strings 
Using the compiled regexps for a PluginRegexp method's __doc__ doesn't work.

Closes Sourceforge #2879862

Signed-off-by: James Vega <jamessan@users.sourceforge.net>
 2009-10-15 22:16:38 -04:00
James Vega ca917d3528 Use utils.web.httpUrlRe for the Web/ShrinkUrl snarfer regexes. 
Signed-off-by: James Vega <jamessan@users.sourceforge.net>
 2009-10-04 21:41:05 -04:00
James Vega 0a4c66071b Adjust testTitle to take into account /.'s slight change 2009-03-11 13:37:26 -04:00
James Vega cbc91c6a26 Use a more appropriate message if the URL definitely has no title. 2009-03-11 13:37:25 -04:00
James Vega 74e06ea52a Catch the proper exception when parsing the title fails. 2009-03-11 13:37:24 -04:00
James Vega ee9aaa89d6 plugins/Web: Swtich the title parser back to HTMLParser sing sgmllib's parser spins on invalid input. 2006-09-13 19:40:51 +00:00
James Vega 855453a743 plugins/Web/test.py: Bump the timeout since we're fetching a lot of information from other sources, which may take a while. 2006-05-03 13:30:27 +00:00
James Vega a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega 9d48f2c879 plugins/Web: Update the exception handling for the change in parsers. 2005-09-20 19:06:35 +00:00
James Vega b375ea9792 plugins/Web: Fixed the title-retrieval parser to actually retrieve the entire title. 2005-07-19 13:55:37 +00:00
James Vega bc1451e898 plugins/Web: Encountering an HTMLParser exception doesn't mean the title hasn't already been snarfed. Don't bail right away. 2005-06-29 19:05:20 +00:00
Jeremy Fincher 490fb0b140 Changed prefixName to prefixNick, which is more appropriate, and has always bothered me. Better now than later. 2005-06-01 21:08:30 +00:00
James Vega 06800f9fc7 Correctly catch the HTMLParseError 2005-05-07 03:55:14 +00:00
James Vega 47179f8bc6 Catch HTMLParserErrors when we're trying to grab the <title>. 2005-05-07 03:24:10 +00:00
James Vega fcfda73f64 Bug #1190350, Don't grab fake title. 2005-04-30 12:53:42 +00:00
James Vega 9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
Jeremy Fincher 7e441285c7 Added the Web.fetch command. 2005-03-14 02:44:55 +00:00
Jeremy Fincher a2e2063d8b Added a callCommand to the Web plugin to catch utils.web.Error. 2005-03-09 07:26:32 +00:00
Jeremy Fincher b0cb616709 Changed callbacks.Privmsg to be callbacks.Plugin, and callbacks.PrivmsgCommandAndRegexp to be callbacks.Plugin. 2005-02-09 07:04:04 +00:00
James Vega 92839a94e7 Remove supybot.privmsgs imports. 2005-02-01 13:57:14 +00:00
Jeremy Fincher 0c2da03a67 Added the Web plugin (from pieces of Http, Fun, and URL) in the new plugin format. 2005-02-01 09:41:54 +00:00