Commit Graph

103 Commits

Author SHA1 Message Date
Valentin Lorentz dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz 5195ff8e12 Web: Add new @location command, to follow HTTP redirects.
Useful to un-tinify URLs.
2020-10-13 22:28:52 +02:00
Valentin Lorentz 0b0da9716d callbacks: honor network-specificity of supybot.reply.whenAddressedBy.
A side-effect is that plugins should now pass 'irc' instead of 'irc.nick'
when they call 'callbacks.addressed()'.
2020-04-11 15:00:46 +02:00
Valentin Lorentz c457b52067 Deduplicate setting Accept-Language HTTP header.
This adds a new function conf.defaultHttpHeaders that can be used by plugins
to get all the default HTTP headers for a given network/channel.
2020-01-14 19:03:12 +01:00
Tasos Sahanidis ae5ad2ceab Web: Implement protocols.http.requestLanguage 2020-01-14 18:48:11 +01:00
Valentin Lorentz 8491d0b944 Web: Lower log level when title could not be found. 2019-12-15 18:43:51 +01:00
Valentin Lorentz dc2068deca Web: Remove leading space if the prefix is empty. 2019-12-15 17:27:47 +01:00
Valentin Lorentz 1a1707420b Web: Add early returns on exception when snarfing titles.
Closes GH-1390.
2019-11-22 18:17:53 +01:00
Rodrigo Nascimento Hernandez 7466058c8f Web: Catch more errors in getTitle. 2019-11-01 09:06:45 +01:00
Valentin Lorentz c1ae3f5c81 all plugins: Use msg.channel instead of msg.args[0] + give network name to self.registryValue. 2019-08-24 23:35:01 +02:00
Valentin Lorentz 0f82f89eec Web: Fix encoding issue on Python 2. Closes GH-1359. 2019-02-01 21:02:57 +01:00
James Lu 2242aadde9 Web: add trailing space for snarferPrefix at runtime
Before, the trailing space in the default snarferPrefix value disappears after a reload because spaces at the end of config lines are ignored.
2018-07-22 04:01:21 +00:00
Valentin Lorentz cd479717b8 Web: Add supybot.plugins.snarfMultipleUrls. Also, fix Web's test cases. 2018-04-14 21:50:32 +02:00
Valentin Lorentz e2180a1e08 Add variable supybot.plugins.Web.snarferPrefix. 2018-03-02 01:26:00 +01:00
Ken Spencer 76c73a57b9 Use a prefix-less help string, don't assume a '@' prefix (#1309)
* Use a prefix-less help string, don't assume a '@' prefix

* Nickometer: follow through on plugin.py with ` -> '
2017-10-25 21:19:37 +02:00
Tasos Sahanidis 8dbf37a173
Web: Fix exception raised due to lack of Content-Type 2017-09-20 04:57:47 +03:00
Johannes Löthberg 07f98d3619 Add timeout to `web title` command
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-08 10:11:15 +01:00
Valentin Lorentz 9fe4abec48 Web: Use a timeout to fetch pages. Closes GH-1275.
This is required because the sandbox is not used anymore,
since 9c57199838.
2016-12-08 00:48:11 +01:00
Valentin Lorentz b9b36d4de5 Improve decorator. 2016-12-08 00:37:12 +01:00
Valentin Lorentz 4acb692f17 Web: Use new-style command wrap (as a decorator). 2016-12-08 00:36:30 +01:00
Valentin Lorentz 9c57199838 Web: Disable the fetch sandbox on Python versions with the _MAXHEADERS fix.
Partial fix to GH-1271.
2016-11-11 12:13:02 +01:00
James Lu 66736b22d5 Web: optionally hide the domain in titleSnarfer
This adds a snarferShowDomain option to optionally hide the domain ("(at site.abc)" text) in titleSnarfer output. Closes #1236.
2016-08-09 11:22:00 -07:00
James Lu e2dedcc5a4 Web: normalize whitespace in titles
Sample link: http://googleblog.blogspot.com/2015/08/android-wear-now-works-with-iphones.html
Before: <bot> 'Title: \nOfficial Google Blog: Android Wear now works with iPhones\n (at googleblog.blogspot.com)'
After: <bot> Title: Official Google Blog: Android Wear now works with iPhones (at googleblog.blogspot.com)
2015-12-29 17:12:26 -08:00
Valentin Lorentz bc19a9fc7f Web: fix syntax. 2015-11-30 07:45:05 +00:00
Valentin Lorentz eaf9e40dc2 Web: increase subprocess memory limit and catch MemoryError appropriately. 2015-11-29 18:34:54 +00:00
Valentin Lorentz a070b658a0 Web: Fix title fetching. 2015-11-29 17:59:57 +00:00
Valentin Lorentz 1f57c31665 Web: Fix NameError with snarferShowTargetDomain. Closes GH-1177. 2015-10-25 16:20:31 +01:00
Valentin Lorentz e3ff413734 Web & core: Merge features of Web's title parser and utils.web.HtmlToText + don't unescape HTML twice. Closes GH-1176. 2015-10-23 07:41:36 +02:00
Jussi Timperi 5cf1b34f55 Web: Use title instead of parser.title. 2015-10-22 17:13:47 +03:00
Jussi Timperi df7689cc2e Web & utils.web: Force HTMLParser to process all buffered data.
Python issue 23144.
2015-10-22 16:56:53 +03:00
Valentin Lorentz 526ffb0ccb Web: Fix code factorization (576a96fb71). Closes GH-1173. 2015-10-17 15:41:20 +02:00
James Lu 6e96f8f8bf Web: actually return the whitespace-stripped title 2015-10-04 12:54:41 -07:00
Valentin Lorentz 20ef13ef9f Web: Ignore SVG titles. Closes GH-1147. 2015-08-29 21:08:35 +02:00
Valentin Lorentz 576a96fb71 Web: Factorize the code of the title snarfer and the title command. 2015-08-29 21:04:38 +02:00
Valentin Lorentz c3a2c800f1 Remove need for 2to3. 2015-08-11 16:50:23 +02:00
Valentin Lorentz 054953891f Web: check URL whitelist in snarfer. 2015-08-11 14:46:47 +00:00
Valentin Lorentz be6bc1a734 Remove need for fix_unicode. 2015-08-10 18:52:51 +02:00
Valentin Lorentz 6ceec0c541 Web: HTMLParseError is deprecated/unused since Python 3.3 and removed in Python 3.5. 2015-08-10 18:16:02 +02:00
Valentin Lorentz c0ac84bb53 Remove need for fix_import, fix_types, and fix_urllib. 2015-08-10 17:55:25 +02:00
Valentin Lorentz 216c5d213f Replace sys.version_info[0] usages with minisix.PY{2,3}. 2015-08-09 00:23:03 +02:00
Valentin Lorentz a81d3ddae6 Web: add option for having titlesnarfer immune to defaultignore. Closes GH-1101 2015-05-15 12:39:30 +02:00
Mikaela Suomalainen 64c0e38635 Web: fix unmatched parenthesis and add missing dot 2014-12-20 13:14:33 +02:00
Valentin Lorentz ba12692fb4 Web: Add support for charrefs. Closes GH-923. 2014-12-11 09:59:08 +01:00
Valentin Lorentz 8ab29fb291 Web: Add explicit error when page encoding cannot be guessed. 2014-10-13 01:13:15 +00:00
Valentin Lorentz 8cd0b4c1e3 Web: Increase timeout to 10 and improve error message. 2014-07-30 11:18:54 +00:00
Valentin Lorentz b8f31a3fca Web: disable threading in commands. (They are run in separated processes anyway…) 2014-04-06 14:05:40 +00:00
Valentin Lorentz 35a62b4e77 Continue accelerating the 2to3 step (remove fix_ws_comma, fix_xreadlines, and fix_zip). 2014-01-21 10:40:18 +01:00
Valentin Lorentz bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
nyuszika7h b5a9aee7a6 Web: Fix exception on timeout 2013-12-25 16:43:41 +01:00