Commit Graph

172 Commits

Author SHA1 Message Date
Valentin Lorentz 1a7c14f4b3 Web: Decode using the charset advertized in response headers
And fall back to the sniffing when not present
2022-11-26 09:06:47 +01:00
Valentin Lorentz 2cfc821203 Web: Allow configuring higher peekSize on Youtube 2022-10-28 14:18:52 +02:00
Valentin Lorentz eb6fc932d9 Web: Fix matching for youtube 2022-04-04 23:29:47 +02:00
Valentin Lorentz 66d986e820 Web: Add overrides to support Youtube and Reddit; remove Twitter from tests. 2022-03-03 22:16:53 +01:00
Valentin Lorentz 4b892c2b1d all: Update .pot and .po files 2022-02-06 00:14:47 +01:00
Valentin Lorentz ed9dffdd7f supybot-plugin-doc: Remove extra space + regenerate all READMEs. 2021-12-04 03:19:12 +01:00
Valentin Lorentz 828fff9cc0 supybot-plugin-doc: Add missing newlines + regenerate all READMEs. 2021-12-04 02:50:28 +01:00
Valentin Lorentz 63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz db05081ff6 commands: fix _checkUrl 2021-08-25 23:28:25 +02:00
Valentin Lorentz be88530fa4 commands: Disallow IRIs from 'url' and 'httpUrl' converters.
urllib doesn't support IRIs, and gives out a cryptic
'UnicodeEncodeError: 'ascii' codec can't encode character ...'
if we don't validate it.
2021-08-25 21:54:55 +02:00
Valentin Lorentz db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz fec6959acb all plugins: regenerate READMEs 2021-04-22 00:29:15 +02:00
Valentin Lorentz 82fbfcffa4 all plugins: regenerate READMEs 2021-04-17 20:09:33 +02:00
Valentin Lorentz cd02444f87 supybot-plugin-doc: Add labels for all command and config variables
So they can be referenced from elsewhere in the doc.
2021-04-06 00:33:07 +02:00
Valentin Lorentz 833af36b08 all plugins: auto-generate READMEs
Starting with this commit, there should be no "original" text
in READMEs.
Instead, the entire text should be in docstrings, and READMEs
should be automatically generated from these docstrings.

Motivation:

* The same help is available both via IRC and in the README
  (although the README can be more detailed, as only the first
  paragraph will be shown on IRC)
* This will allow auto-generating plugin help on docs.limnoria.net
  using the same content as the README, as it's sourced from the
  docstrings

Additionally, this converts all READMEs from Markdown to ReST,
because the documentation already uses ReST, and other docstrings
in the codebase are in ReST for this reason.
2021-04-05 17:42:31 +02:00
Valentin Lorentz 5195ff8e12 Web: Add new @location command, to follow HTTP redirects.
Useful to un-tinify URLs.
2020-10-13 22:28:52 +02:00
Valentin Lorentz 0b0da9716d callbacks: honor network-specificity of supybot.reply.whenAddressedBy.
A side-effect is that plugins should now pass 'irc' instead of 'irc.nick'
when they call 'callbacks.addressed()'.
2020-04-11 15:00:46 +02:00
Valentin Lorentz e2d72c5a43 Fix msgfmt warnings. 2020-03-26 22:10:28 +01:00
Valentin Lorentz 9ab908759a Use importlib instead of imp.
importlib is deprecated.
2020-01-26 12:00:08 +01:00
Valentin Lorentz 9891b0fca4 Replace failUnless/failIf with assertTrue/assertFalse.
The old names are deprecated.
2020-01-26 11:14:18 +01:00
Valentin Lorentz c457b52067 Deduplicate setting Accept-Language HTTP header.
This adds a new function conf.defaultHttpHeaders that can be used by plugins
to get all the default HTTP headers for a given network/channel.
2020-01-14 19:03:12 +01:00
Tasos Sahanidis ae5ad2ceab Web: Implement protocols.http.requestLanguage 2020-01-14 18:48:11 +01:00
Valentin Lorentz 8491d0b944 Web: Lower log level when title could not be found. 2019-12-15 18:43:51 +01:00
Valentin Lorentz dc2068deca Web: Remove leading space if the prefix is empty. 2019-12-15 17:27:47 +01:00
Valentin Lorentz 1a1707420b Web: Add early returns on exception when snarfing titles.
Closes GH-1390.
2019-11-22 18:17:53 +01:00
James Lu 9291fbd4a0 All plugins: set Limnoria contributors as maintainer 2019-11-16 11:41:13 -08:00
Rodrigo Nascimento Hernandez 7466058c8f Web: Catch more errors in getTitle. 2019-11-01 09:06:45 +01:00
Valentin Lorentz c1ae3f5c81 all plugins: Use msg.channel instead of msg.args[0] + give network name to self.registryValue. 2019-08-24 23:35:01 +02:00
Valentin Lorentz 0f82f89eec Web: Fix encoding issue on Python 2. Closes GH-1359. 2019-02-01 21:02:57 +01:00
James Lu 2242aadde9 Web: add trailing space for snarferPrefix at runtime
Before, the trailing space in the default snarferPrefix value disappears after a reload because spaces at the end of config lines are ignored.
2018-07-22 04:01:21 +00:00
Valentin Lorentz cd479717b8 Web: Add supybot.plugins.snarfMultipleUrls. Also, fix Web's test cases. 2018-04-14 21:50:32 +02:00
Valentin Lorentz e2180a1e08 Add variable supybot.plugins.Web.snarferPrefix. 2018-03-02 01:26:00 +01:00
Ken Spencer 76c73a57b9 Use a prefix-less help string, don't assume a '@' prefix (#1309)
* Use a prefix-less help string, don't assume a '@' prefix

* Nickometer: follow through on plugin.py with ` -> '
2017-10-25 21:19:37 +02:00
Tasos Sahanidis 8dbf37a173
Web: Fix exception raised due to lack of Content-Type 2017-09-20 04:57:47 +03:00
Johannes Löthberg 07f98d3619 Add timeout to `web title` command
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-08 10:11:15 +01:00
Valentin Lorentz 9fe4abec48 Web: Use a timeout to fetch pages. Closes GH-1275.
This is required because the sandbox is not used anymore,
since 9c57199838.
2016-12-08 00:48:11 +01:00
Valentin Lorentz b9b36d4de5 Improve decorator. 2016-12-08 00:37:12 +01:00
Valentin Lorentz 4acb692f17 Web: Use new-style command wrap (as a decorator). 2016-12-08 00:36:30 +01:00
Valentin Lorentz 9c57199838 Web: Disable the fetch sandbox on Python versions with the _MAXHEADERS fix.
Partial fix to GH-1271.
2016-11-11 12:13:02 +01:00
James Lu 66736b22d5 Web: optionally hide the domain in titleSnarfer
This adds a snarferShowDomain option to optionally hide the domain ("(at site.abc)" text) in titleSnarfer output. Closes #1236.
2016-08-09 11:22:00 -07:00
James Lu e2dedcc5a4 Web: normalize whitespace in titles
Sample link: http://googleblog.blogspot.com/2015/08/android-wear-now-works-with-iphones.html
Before: <bot> 'Title: \nOfficial Google Blog: Android Wear now works with iPhones\n (at googleblog.blogspot.com)'
After: <bot> Title: Official Google Blog: Android Wear now works with iPhones (at googleblog.blogspot.com)
2015-12-29 17:12:26 -08:00
Valentin Lorentz bc19a9fc7f Web: fix syntax. 2015-11-30 07:45:05 +00:00
Valentin Lorentz eaf9e40dc2 Web: increase subprocess memory limit and catch MemoryError appropriately. 2015-11-29 18:34:54 +00:00
Valentin Lorentz a070b658a0 Web: Fix title fetching. 2015-11-29 17:59:57 +00:00
Valentin Lorentz 1f57c31665 Web: Fix NameError with snarferShowTargetDomain. Closes GH-1177. 2015-10-25 16:20:31 +01:00
Valentin Lorentz e3ff413734 Web & core: Merge features of Web's title parser and utils.web.HtmlToText + don't unescape HTML twice. Closes GH-1176. 2015-10-23 07:41:36 +02:00
Jussi Timperi 1f96471d93 Web: Correct capitalization in testTitle. 2015-10-22 17:19:03 +03:00
Jussi Timperi 5cf1b34f55 Web: Use title instead of parser.title. 2015-10-22 17:13:47 +03:00
Jussi Timperi df7689cc2e Web & utils.web: Force HTMLParser to process all buffered data.
Python issue 23144.
2015-10-22 16:56:53 +03:00