Commit Graph

50 Commits

Author SHA1 Message Date
Valentin Lorentz 4da1291876 URL: Lazily deserialize records from the end in @last
Before this commit, the plugin first fetched a list of all
(deserialized) records in a list, then reversed the list, and iterated
on the reverse list.
This proved to be slow, with most of the time being spent in
`dbi.DB._newRecord` (which essentially deserializes one list of CSV).

After this commit, the list is reversed first, then the plugin iterates
on its generator, which calls `_newRecord` on records as they are
requested.

This means that when there are many URLs in the database, `@last` does
not need to waste time deserializing most records, when the result is
near the end (and if the result is the first record, then it does
exactly as much work as before).
2022-10-30 20:46:35 +01:00
Valentin Lorentz a6aa5530dd Ensure files written with AtomicFile are read in UTF8
With some locale configurations (not that uncommon on CentOS), open() may
default to non-UTF8 encodings (eg. ANSI_X3.4-1968).

This is usually not an issue, because we use open() both for writing and
reading. However, AtomicFile implicitly enforces UTF8; which needs to be
mirrored when reading.
2022-10-06 09:49:42 +02:00
Valentin Lorentz 63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f025
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f025 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz 912d2e1538 core: Apply some suggestions of pyflakes. 2015-08-31 18:19:22 +02:00
Valentin Lorentz c3a2c800f1 Remove need for 2to3. 2015-08-11 16:50:23 +02:00
Valentin Lorentz f305ff3c02 Remove fix_basestring. 2015-08-10 20:24:11 +02:00
Valentin Lorentz be118c3338 Remove use of fix_dict fixer. 2015-08-08 23:08:49 +02:00
Valentin Lorentz bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
Valentin Lorentz 2fda69b4d6 Continue accelerating the 2to3 step (remove fix_raise). 2014-01-20 15:43:55 +01:00
Valentin Lorentz b44a08d2bd Fix previous merge (again). 2014-01-03 17:48:30 +00:00
nyuszika7h 0e480b4e52 Fix file descriptor leaks 2014-01-03 17:44:01 +01:00
Valentin Lorentz 40675ffdfa Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	plugins/Admin/plugin.py
	plugins/BadWords/config.py
	plugins/Filter/plugin.py
	plugins/Google/plugin.py
	plugins/Math/plugin.py
	plugins/Misc/plugin.py
	plugins/Note/plugin.py
	plugins/RSS/plugin.py
	plugins/Seen/test.py
	plugins/ShrinkUrl/config.py
	plugins/ShrinkUrl/plugin.py
	plugins/ShrinkUrl/test.py
	plugins/Status/plugin.py
	plugins/String/config.py
	plugins/String/plugin.py
	plugins/Time/plugin.py
	plugins/Todo/plugin.py
	plugins/Web/plugin.py
	plugins/__init__.py
	scripts/supybot-botchk
	setup.py
	src/__init__.py
	src/callbacks.py
	src/commands.py
	src/conf.py
	src/drivers/Socket.py
	src/ircdb.py
	src/irclib.py
	src/ircutils.py
	src/questions.py
	src/registry.py
	src/schedule.py
	src/test.py
	src/utils/file.py
	src/utils/gen.py
	src/utils/net.py
	src/utils/web.py
	src/world.py
2013-08-24 11:28:29 +02:00
James McCoy cef93a6cfd Use relative imports for all packages under the supybot namespace
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-17 22:12:11 -04:00
Valentin Lorentz 535593d555 src/dbi.py: Use open() instead of file(). 2012-08-05 16:56:07 +02:00
Valentin Lorentz 14200e002a Rename cdb.open to cdb.open_db and open cdb files in binary mode. 2012-08-05 15:47:30 +02:00
Valentin Lorentz 14609b3bb2 Add self as an argument to methos in src/dbi.py 2012-05-22 21:15:58 -04:00
Valentin Lorentz 1fe4cef61b Import os before using it. 2012-05-22 21:14:42 -04:00
Valentin Lorentz af460596d3 Add self as an argument to methos in src/dbi.py 2012-05-02 20:50:06 +02:00
Valentin Lorentz 776ce96d52 Import os before using it. 2012-05-02 20:48:49 +02:00
James Vega a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega 9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
Ali Afshar 4404cdc51a dbi cdb mapping trying to use a non-existent variable. 2005-03-21 15:53:13 +00:00
Jeremy Fincher cb05dc8484 Fixed a typo in a variable name. 2005-02-01 10:13:23 +00:00
James Vega ec045afd9c Removed extraneous whitespace, unused imports, and converted some sets.Set() calls to set(). 2005-01-31 15:24:36 +00:00
James Vega 0fde2393e6 Update usage of utils.iter functions. 2005-01-31 15:22:48 +00:00
James Vega 0a89623b70 Update a couple more imports for the utils split up and fix.py removal. 2005-01-30 15:39:39 +00:00
Jeremy Fincher 5fd6bbb52d Completely restructured our utils modules.
Tons of changes.  Here's the summary of things that matter most:

* There is no more supybot.fix.
* There is no more supybot.webutils; now there is supybot.utils.web.
  * It's no longer webutils.WebError, but just utils.web.Error.
* You shouldn't import itertools, ideally, but instead import utils.iter.
* No more using imap/ifilter in commands unless absolutely necessary.  It's
  premature optimization and annoying.
* utils.str.format isn't quite ready yet, but will be soon.  That'll be the
  next big thing to fix in our code.
2005-01-27 06:59:08 +00:00
Jeremy Fincher 5019d10a1b Updated license years. 2005-01-19 13:33:05 +00:00
Jeremy Fincher 8ca625d339 Initial import. 2005-01-19 13:14:38 +00:00
James Vega dbfec8afb9 Add supybot module 2007-04-22 20:10:26 +00:00
Jeremy Fincher 4624115909 No need to makeBackupIfSmaller in databases. That's more for configuration files. 2005-01-14 14:46:52 +00:00
Jeremy Fincher cbba7c6115 Fixed a potential bug. 2004-12-02 04:51:58 +00:00
Jeremy Fincher 9021c5e150 DirMapping is incomplete, but I need some feature additions for DB. 2004-12-01 21:47:02 +00:00
Jeremy Fincher 54b56a8a53 Raise the proper exception. 2004-11-30 04:42:20 +00:00
Jeremy Fincher 744944d8b2 Wrong variable name. 2004-10-29 19:27:25 +00:00
Jeremy Fincher 6ba2120f5f Optimization. 2004-10-04 16:11:08 +00:00
Jeremy Fincher e3c9464634 Return None if there's nothing in the database. 2004-10-01 21:25:26 +00:00
Jeremy Fincher 0e5cd79a1b eXXXtra eXXXtra, read all about it. 2004-10-01 08:22:48 +00:00
Jeremy Fincher 4cc1e4ba04 Abstracted for some future overriding. 2004-09-30 04:15:32 +00:00
Jeremy Fincher e01a9d3bc5 Forgot to vacuum, momma's gonna kill me! 2004-09-28 20:09:39 +00:00
Jeremy Fincher 5753195f45 Changed dbi.Record not to use a metaclass. 2004-09-24 20:05:34 +00:00
James Vega 50973a10fb Remove #! lines. 2004-09-20 03:01:50 +00:00
Jeremy Fincher acf0a3f3e5 Augmented the add method for Note. 2004-08-24 04:09:15 +00:00
Jeremy Fincher fbb348b6e7 Copyright updates. Again, more is left to be done, but this is my part. 2004-08-23 13:14:06 +00:00
James Vega 72f36efc18 Fix some problems in Infobot.py's sqlite implementation and genericize
some other errors to better fit in with db abstraction
2004-08-17 20:01:23 +00:00
James Vega 46c71a7521 Add dbi.NoRecordError and update Quotes to use it 2004-08-16 17:34:58 +00:00
Jeremy Fincher a1a703df32 Let's go to repr. Beware, people, this may invalidate certain databases. 2004-08-12 07:41:05 +00:00
Jeremy Fincher 12ee483f7a Slight bugfixes; returning from a remove is unpossible. 2004-08-11 05:48:55 +00:00
Jeremy Fincher e5e98fdc2f Initial checkin. 2004-08-11 05:14:15 +00:00