Commit Graph

172 Commits

Author SHA1 Message Date
Valentin Lorentz
1a7c14f4b3 Web: Decode using the charset advertized in response headers
And fall back to the sniffing when not present
2022-11-26 09:06:47 +01:00
Valentin Lorentz
2cfc821203 Web: Allow configuring higher peekSize on Youtube 2022-10-28 14:18:52 +02:00
Valentin Lorentz
eb6fc932d9 Web: Fix matching for youtube 2022-04-04 23:29:47 +02:00
Valentin Lorentz
66d986e820 Web: Add overrides to support Youtube and Reddit; remove Twitter from tests. 2022-03-03 22:16:53 +01:00
Valentin Lorentz
4b892c2b1d all: Update .pot and .po files 2022-02-06 00:14:47 +01:00
Valentin Lorentz
ed9dffdd7f supybot-plugin-doc: Remove extra space + regenerate all READMEs. 2021-12-04 03:19:12 +01:00
Valentin Lorentz
828fff9cc0 supybot-plugin-doc: Add missing newlines + regenerate all READMEs. 2021-12-04 02:50:28 +01:00
Valentin Lorentz
63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz
dc79ab193a Update capitalization of my Github username 2021-09-14 20:30:47 +02:00
Valentin Lorentz
db05081ff6 commands: fix _checkUrl 2021-08-25 23:28:25 +02:00
Valentin Lorentz
be88530fa4 commands: Disallow IRIs from 'url' and 'httpUrl' converters.
urllib doesn't support IRIs, and gives out a cryptic
'UnicodeEncodeError: 'ascii' codec can't encode character ...'
if we don't validate it.
2021-08-25 21:54:55 +02:00
Valentin Lorentz
db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz
fec6959acb all plugins: regenerate READMEs 2021-04-22 00:29:15 +02:00
Valentin Lorentz
82fbfcffa4 all plugins: regenerate READMEs 2021-04-17 20:09:33 +02:00
Valentin Lorentz
cd02444f87 supybot-plugin-doc: Add labels for all command and config variables
So they can be referenced from elsewhere in the doc.
2021-04-06 00:33:07 +02:00
Valentin Lorentz
833af36b08 all plugins: auto-generate READMEs
Starting with this commit, there should be no "original" text
in READMEs.
Instead, the entire text should be in docstrings, and READMEs
should be automatically generated from these docstrings.

Motivation:

* The same help is available both via IRC and in the README
  (although the README can be more detailed, as only the first
  paragraph will be shown on IRC)
* This will allow auto-generating plugin help on docs.limnoria.net
  using the same content as the README, as it's sourced from the
  docstrings

Additionally, this converts all READMEs from Markdown to ReST,
because the documentation already uses ReST, and other docstrings
in the codebase are in ReST for this reason.
2021-04-05 17:42:31 +02:00
Valentin Lorentz
5195ff8e12 Web: Add new @location command, to follow HTTP redirects.
Useful to un-tinify URLs.
2020-10-13 22:28:52 +02:00
Valentin Lorentz
0b0da9716d callbacks: honor network-specificity of supybot.reply.whenAddressedBy.
A side-effect is that plugins should now pass 'irc' instead of 'irc.nick'
when they call 'callbacks.addressed()'.
2020-04-11 15:00:46 +02:00
Valentin Lorentz
e2d72c5a43 Fix msgfmt warnings. 2020-03-26 22:10:28 +01:00
Valentin Lorentz
9ab908759a Use importlib instead of imp.
importlib is deprecated.
2020-01-26 12:00:08 +01:00
Valentin Lorentz
9891b0fca4 Replace failUnless/failIf with assertTrue/assertFalse.
The old names are deprecated.
2020-01-26 11:14:18 +01:00
Valentin Lorentz
c457b52067 Deduplicate setting Accept-Language HTTP header.
This adds a new function conf.defaultHttpHeaders that can be used by plugins
to get all the default HTTP headers for a given network/channel.
2020-01-14 19:03:12 +01:00
Tasos Sahanidis
ae5ad2ceab Web: Implement protocols.http.requestLanguage 2020-01-14 18:48:11 +01:00
Valentin Lorentz
8491d0b944 Web: Lower log level when title could not be found. 2019-12-15 18:43:51 +01:00
Valentin Lorentz
dc2068deca Web: Remove leading space if the prefix is empty. 2019-12-15 17:27:47 +01:00
Valentin Lorentz
1a1707420b Web: Add early returns on exception when snarfing titles.
Closes GH-1390.
2019-11-22 18:17:53 +01:00
James Lu
9291fbd4a0 All plugins: set Limnoria contributors as maintainer 2019-11-16 11:41:13 -08:00
Rodrigo Nascimento Hernandez
7466058c8f Web: Catch more errors in getTitle. 2019-11-01 09:06:45 +01:00
Valentin Lorentz
c1ae3f5c81 all plugins: Use msg.channel instead of msg.args[0] + give network name to self.registryValue. 2019-08-24 23:35:01 +02:00
Valentin Lorentz
0f82f89eec Web: Fix encoding issue on Python 2. Closes GH-1359. 2019-02-01 21:02:57 +01:00
James Lu
2242aadde9 Web: add trailing space for snarferPrefix at runtime
Before, the trailing space in the default snarferPrefix value disappears after a reload because spaces at the end of config lines are ignored.
2018-07-22 04:01:21 +00:00
Valentin Lorentz
cd479717b8 Web: Add supybot.plugins.snarfMultipleUrls. Also, fix Web's test cases. 2018-04-14 21:50:32 +02:00
Valentin Lorentz
e2180a1e08 Add variable supybot.plugins.Web.snarferPrefix. 2018-03-02 01:26:00 +01:00
Ken Spencer
76c73a57b9 Use a prefix-less help string, don't assume a '@' prefix (#1309)
* Use a prefix-less help string, don't assume a '@' prefix

* Nickometer: follow through on plugin.py with ` -> '
2017-10-25 21:19:37 +02:00
Tasos Sahanidis
8dbf37a173
Web: Fix exception raised due to lack of Content-Type 2017-09-20 04:57:47 +03:00
Johannes Löthberg
07f98d3619 Add timeout to web title command
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-08 10:11:15 +01:00
Valentin Lorentz
9fe4abec48 Web: Use a timeout to fetch pages. Closes GH-1275.
This is required because the sandbox is not used anymore,
since 9c57199838.
2016-12-08 00:48:11 +01:00
Valentin Lorentz
b9b36d4de5 Improve decorator. 2016-12-08 00:37:12 +01:00
Valentin Lorentz
4acb692f17 Web: Use new-style command wrap (as a decorator). 2016-12-08 00:36:30 +01:00
Valentin Lorentz
9c57199838 Web: Disable the fetch sandbox on Python versions with the _MAXHEADERS fix.
Partial fix to GH-1271.
2016-11-11 12:13:02 +01:00
James Lu
66736b22d5 Web: optionally hide the domain in titleSnarfer
This adds a snarferShowDomain option to optionally hide the domain ("(at site.abc)" text) in titleSnarfer output. Closes #1236.
2016-08-09 11:22:00 -07:00
James Lu
e2dedcc5a4 Web: normalize whitespace in titles
Sample link: http://googleblog.blogspot.com/2015/08/android-wear-now-works-with-iphones.html
Before: <bot> 'Title: \nOfficial Google Blog: Android Wear now works with iPhones\n (at googleblog.blogspot.com)'
After: <bot> Title: Official Google Blog: Android Wear now works with iPhones (at googleblog.blogspot.com)
2015-12-29 17:12:26 -08:00
Valentin Lorentz
bc19a9fc7f Web: fix syntax. 2015-11-30 07:45:05 +00:00
Valentin Lorentz
eaf9e40dc2 Web: increase subprocess memory limit and catch MemoryError appropriately. 2015-11-29 18:34:54 +00:00
Valentin Lorentz
a070b658a0 Web: Fix title fetching. 2015-11-29 17:59:57 +00:00
Valentin Lorentz
1f57c31665 Web: Fix NameError with snarferShowTargetDomain. Closes GH-1177. 2015-10-25 16:20:31 +01:00
Valentin Lorentz
e3ff413734 Web & core: Merge features of Web's title parser and utils.web.HtmlToText + don't unescape HTML twice. Closes GH-1176. 2015-10-23 07:41:36 +02:00
Jussi Timperi
1f96471d93 Web: Correct capitalization in testTitle. 2015-10-22 17:19:03 +03:00
Jussi Timperi
5cf1b34f55 Web: Use title instead of parser.title. 2015-10-22 17:13:47 +03:00
Jussi Timperi
df7689cc2e Web & utils.web: Force HTMLParser to process all buffered data.
Python issue 23144.
2015-10-22 16:56:53 +03:00