Limnoria/plugins/URL
Valentin Lorentz 4da1291876 URL: Lazily deserialize records from the end in @last
Before this commit, the plugin first fetched a list of all
(deserialized) records in a list, then reversed the list, and iterated
on the reverse list.
This proved to be slow, with most of the time being spent in
`dbi.DB._newRecord` (which essentially deserializes one list of CSV).

After this commit, the list is reversed first, then the plugin iterates
on its generator, which calls `_newRecord` on records as they are
requested.

This means that when there are many URLs in the database, `@last` does
not need to waste time deserializing most records, when the result is
near the end (and if the result is the first record, then it does
exactly as much work as before).
2022-10-30 20:46:35 +01:00
..
locales all: Update .pot and .po files 2022-02-06 00:14:47 +01:00
README.rst supybot-plugin-doc: Remove extra space + regenerate all READMEs. 2021-12-04 03:19:12 +01:00
__init__.py Revert generic 'The Limnoria Contributors' in copyright notices 2021-10-17 09:57:55 +02:00
config.py Revert generic 'The Limnoria Contributors' in copyright notices 2021-10-17 09:57:55 +02:00
messages.pot all: Update .pot and .po files 2022-02-06 00:14:47 +01:00
plugin.py URL: Lazily deserialize records from the end in @last 2022-10-30 20:46:35 +01:00
test.py Revert generic 'The Limnoria Contributors' in copyright notices 2021-10-17 09:57:55 +02:00

README.rst

Documentation for the URL plugin for Supybot

Purpose

Keeps track of URLs posted to a channel, along with relevant context. Allows searching for URLs and returning random URLs. Also provides statistics on the URLs in the database.

Usage

This plugin records how many URLs have been mentioned in a channel and what the last URL was.

Commands

last [<channel>] [--{from,with,without,near,proto} <value>] [--nolimit]

Gives the last URL matching the given criteria. --from is from whom the URL came; --proto is the protocol the URL used; --with is something inside the URL; --without is something that should not be in the URL; --near is something in the same message as the URL. If --nolimit is given, returns all the URLs that are found to just the URL. <channel> is only necessary if the message isn't sent in the channel itself.

stats [<channel>]

Returns the number of URLs in the URL database. <channel> is only required if the message isn't sent in the channel itself.

Configuration

supybot.plugins.URL.nonSnarfingRegexp

This config variable defaults to "", is network-specific, and is channel-specific.

Determines what URLs are not to be snarfed and stored in the database for the channel; URLs matching the given regexp will not be snarfed. Give the empty string if you have no URLs that you'd like to exclude from being snarfed.

supybot.plugins.URL.public

This config variable defaults to "True", is not network-specific, and is not channel-specific.

Determines whether this plugin is publicly visible.