diff --git a/docs/ADVANCED_PLUGIN_CONFIG b/docs/ADVANCED_PLUGIN_CONFIG new file mode 100644 index 000000000..9134d2f71 --- /dev/null +++ b/docs/ADVANCED_PLUGIN_CONFIG @@ -0,0 +1,342 @@ +Advanced Plugin Config +---------------------- +This tutorial covers some of the more advanced plugin config features available +to Supybot plugin authors. + +What's This Tutorial For? +========================= + Brief overview of what this tutorial covers and the target audience. + +Want to know the crazy advanced features available to you, the Supybot plugin +author? Well, this is the tutorial for you. This article assumes you've read +the Supybot plugin author tutorial since all the basics of plugin config are +handled there first. + +In this tutorial we'll cover: + + * Using the configure function more effectively by using the functions + provided in supybot.questions + * Creating config variable groups and config variables underneath those + groups. + * The built-in config variable types ("registry types") for use with config + variables + * Creating custom registry types to handle config variable values more + effectively + +Using 'configure' effectively +============================= + How to use 'configure' effectively using the functions from + 'supybot.questions' + +In the original Supybot plugin author tutorial you'll note that we gloss over +the configure portion of the config.py file for the sake of keeping the +tutorial to a reasonable length. Well, now we're going to cover it in more +detail. + +The supybot.questions module is a nice little module coded specifically to help +clean up the configure section of every plugin's config.py. The boilerplate +config.py code imports the four most useful functions from that module: + + * "expect" is a very general prompting mechanism which can specify certain + inputs that it will accept and also specify a default response. It takes + the following arguments: + - prompt: The text to be displayed + - possibilities: The list of possible responses (can be the empty + list, []) + - default (optional): Defaults to None. Specifies the default value + to use if the user enters in no input. + - acceptEmpty (optional): Defaults to False. Specifies whether or not + to accept no input as an answer. + + * "anything" is basically a special case of except which takes anything + (including no input) and has no default value specified. It takes only + one argument: + - prompt: The text to be displayed + + * "something" is also a special case of except, requiring some input and + allowing an optional default. It takes the following arguments: + - prompt: The text to be displayed + - default (optional): Defaults to None. The default value to use if + the user doesn't input anything. + + * "yn" is for "yes or no" questions and basically forces the user to input + a "y" for yes, or "n" for no. It takes the following arguments: + - prompt: The text to be displayed + - default (optional): Defaults to None. Default value to use if the + user doesn't put anything. + +All of these functions, with the exception of "yn", return whatever string +results as the answer whether it be input from the user or specified as the +default when the user inputs nothing. The "yn" function returns True for "yes" +answers and False for "no" answers. + +For the most part, the latter three should be sufficient, but we expose expect +to anyone who needs a more specialized configuration. + +Let's go through a quick example configure that covers all four of these +functions. First I'll give you the code, and then we'll go through it, +discussing each usage of a supybot.questions function just to make sure you +realize what the code is actually doing. Here it is: + + def configure(advanced): + # This will be called by supybot to configure this module. advanced is + # a bool that specifies whether the user identified himself as an advanced + # user or not. You should effect your configuration by manipulating the + # registry as appropriate. + from supybot.questions import expect, anything, something, yn + WorldDom = conf.registerPlugin('WorldDom', True) + if yn("""The WorldDom plugin allows for total world domination + with simple commands. Would you like these commands to + be enabled for everyone?""", default=False): + WorldDom.globalWorldDominationRequires.setValue("") + else: + cap = something("""What capability would you like to require for + this command to be used?""", default="Admin") + WorldDom.globalWorldDominationRequires.setValue(cap) + dir = expect("""What direction would you like to attack from in + your quest for world domination?""", + ["north", "south", "east", "west", "ABOVE"], + default="ABOVE") + WorldDom.attackDirection.setValue(dir) + +As you can see, this is the WorldDom plugin, which I am currently working on. +The first thing our configure function checks is to see whether or not the bot +owner would like the world domination commands in this plugin to be available +to everyone. If they say yes, we set the globalWorldDominationRequires +configuration variable to the empty string, signifying that no specific +capabilities are necessary. If they say no, we prompt them for a specific +capability to check for, defaulting to the "Admin" capability. Here they can +create their own custom capability to grant to folks which this plugin will +check for if they want, but luckily for the bot owner they don't really have to +do this since Supybot's capabilities system can be flexed to take care of this. + +Lastly, we check to find out what direction they want to attack from as they +venture towards world domination. I prefer "death from above!", so I made that +the default response, but the more boring cardinal directions are available as +choices as well. + +Using Config Groups +=================== + A brief overview of how to use config groups to organize config variables + +Supybot's Hierarchical Configuration + +Supybot's configuration is inherently hierarchical, as you've probably already +figured out in your use of the bot. Naturally, it makes sense to allow plugin +authors to create their own hierarchies to organize their configuration +variables for plugins that have a lot of plugin options. If you've taken a look +at the plugins that Supybot comes with, you've probably noticed that several of +them take advantage of this. In this section of this tutorial we'll go over how +to make your own config hierarchy for your plugin. + +Here's the brilliant part about Supybot config values which makes hierarchical +structuring all that much easier - values are groups. That is, any config value +you may already defined in your plugins can already be treated as a group, you +simply need to know how to add items to that group. + +Now, if you want to just create a group that doesn't have an inherent value you +can do that as well, but you'd be surprised at how rarely you have to do that. +In fact if you look at most of the plugins that Supybot comes with, you'll only +find that we do this in a handful of spots yet we use the "values as groups" +feature quite a bit. + +Creating a Config Group +======================= + +As stated before, config variables themselves are groups, so you can create a +group simply by creating a configuration variable: + + conf.registerGlobalValue(WorldDom, 'globalWorldDominationRequires', + registry.String('', """Determines the capability required to access the + world domination commands in this plugin.""")) + +As you probably know by now this creates the config variable +supybot.plugins.WorldDom.globalWorldDominationRequires which you can access/set +using the Config plugin directly on the running bot. What you may not have +known prior to this tutorial is that that variable is also a group. +Specifically, it is now the WorldDom.globalWorldDominationRequires group, and +we can add config variables to it! Unfortunately, this particular bit of +configuration doesn't really require anything underneath it, so let's create a +new group which does using the "create only a group, not a value" command. + +Let's create a configurable list of targets for different types of attacks +(land, sea, air, etc.). We'll call the group attackTargets. Here's how you +create just a config group alone with no value assigned: + + conf.registerGroup(WorldDom, 'attackTargets') + +The first argument is just the group under which you want to create your new +group (and we got WorldDom from conf.registerPlugin which was in our +boilerplate code from the plugin creation wizard). The second argument is, of +course, the group name. So now we have WorldDom.attackTargets (or, fully, +supybot.plugins.WorldDom.attackTargets). + +Adding Values to a Group +======================== + +Actually, you've already done this several times, just never to a custom group +of your own. You've always added config values to your plugin's config group. +With that in mind, the only slight modification needed is to simply point to +the new group: + + conf.registerGlobalValue(WorldDom.attackTargets, 'air', + registry.SpaceSeparatedListOfStrings('', """Contains the list of air + targets.""")) + +And now we have a nice list of air targets! You'll notice that the first +argument is WorldDom.attackTargets, our new group. Make sure that the +conf.registerGroup call is made before this one or else you'll get a nasty +AttributeError. + +The Built-in Registry Types +=========================== + A rundown of all of the built-in registry types available for use with config + variables. + +The "registry" module defines the following config variable types for your use +(I'll include the 'registry.' on each one since that's how you'll refer to it in +code most often). Most of them are fairly self-explanatory, so excuse the +boring descriptions: + + * registry.Boolean - A simple true or false value. Also accepts the + following for true: "true", "on" "enable", "enabled", "1", and the + following for false: "false", "off", "disable", "disabled", "0", + + * registry.Integer - Accepts any integer value, positive or negative. + + * registry.NonNegativeInteger - Will hold any non-negative integer value. + + * registry.PositiveInteger - Same as above, except that it doesn't accept 0 + as a value. + + * registry.Float - Accepts any floating point number. + + * registry.PositiveFloat - Accepts any positive floating point number. + + * registry.Probability - Accepts any floating point number between 0 and 1 + (inclusive, meaning 0 and 1 are also valid). + + * registry.String - Accepts any string that is not a valid Python command + + * registry.NormalizedString - Accepts any string (with the same exception + above) but will normalize sequential whitespace to a single space.. + + * registry.StringSurroundedBySpaces - Accepts any string but assures that + it has a space preceding and following it. Useful for configuring a + string that goes in the middle of a response. + + * registry.StringWithSpaceOnRight - Also accepts any string but assures + that it has a space after it. Useful for configuring a string that + begins a response. + + * registry.Regexp - Accepts only valid (Perl or Python) regular expressions + + * registry.SpaceSeparatedListOfStrings - Accepts a space-separated list of + strings. + +There are a few other built-in registry types that are available but are not +usable in their current state, only by creating custom registry types, which +we'll go over in the next section. + +Custom Registry Types +===================== + How to create and use your own custom registry types for use in customizing + plugin config variables. + +Why Create Custom Registry Types? + +For most configuration, the provided types in the registry module are +sufficient. However, for some configuration variables it's not only convenient +to use custom registry types, it's actually recommended. Customizing registry +types allows for tighter restrictions on the values that get set and for +greater error-checking than is possible with the provided types. + +What Defines a Registry Type? + +First and foremost, it needs to subclass one of the existing registry types +from the registry module, whether it be one of the ones in the previous section +or one of the other classes in registry specifically designed to be subclassed. + +Also it defines a number of other nice things: a custom error message for your +type, customized value-setting (transforming the data you get into something +else if wanted), etc. + +Creating Your First Custom Registry Type + +As stated above, priority number one is that you subclass one of the types in +the registry module. Basically, you just subclass one of those and then +customize whatever you want. Then you can use it all you want in your own +plugins. We'll do a quick example to demonstrate. + +We already have registry.Integer and registry.PositiveInteger, but let's say we +want to accept only negative integers. We can create our own NegativeInteger +registry type like so: + + class NegativeInteger(registry.Integer): + """Value must be a negative integer.""" + def setValue(self, v): + if v >= 0: + self.error() + registry.Integer.setValue(self, v) + +All we need to do is define a new error message for our custom registry type +(specified by the docstring for the class), and customize the setValue +function. Note that all you have to do when you want to signify that you've +gotten an invalid value is to call self.error(). Finally, we call the parent +class's setValue to actually set the value. + +What Else Can I Customize? + +Well, the error string and the setValue function are the most useful things +that are available for customization, but there are other things. For examples, +look at the actual built-in registry types defined in registry.py (in the src +directory distributed with the bot). + +What Subclasses Can I Use? + +Chances are one of the built-in types in the previous section will be +sufficient, but there are a few others of note which deserve mention: + + * registry.Value - Provides all the core functionality of registry types + (including acting as a group for other config variables to reside + underneath), but nothing more. + + * registry.OnlySomeStrings - Allows you to specify only a certain set of + strings as valid values. Simply override validStrings in the inheriting + class and you're ready to go. + + * registry.SeparatedListOf - The generic class which is the parent class to + registry.SpaceSeparatedListOfStrings. Allows you to customize four + things: the type of sequence it is (list, set, tuple, etc.), what each + item must be (String, Boolean, etc.), what separates each item in the + sequence (using custom splitter/joiner functions), and whether or not + the sequence is to be sorted. Look at the definitions of + registry.SpaceSeparatedListOfStrings and + registry.CommaSeparatedListOfStrings at the bottom of registry.py for + more information. Also, there will be an example using this in the + section below. + +Using My Custom Registry Type + +Using your new registry type is relatively straightforward. Instead of using +whatever registry built-in you might have used before, now use your own custom +class. Let's say we define a registry type to handle a comma-separated list of +probabilities: + + class CommaSeparatedListOfProbabilities(registry.SeparatedListOf): + Value = registry.Probability + def splitter(self, s): + return re.split(r'\s*,\s*', s) + joiner = ', '.join + +Now, to use that type we simply have to specify it whenever we create a config +variable using it: + + conf.registerGlobalValue(SomePlugin, 'someConfVar', + CommaSeparatedListOfProbabilities('0.0, 1.0', """Holds the list of + probabilities for whatever.""")) + +Note that we initialize it just the same as we do any other registry type, with +two arguments: the default value, and then the description of the config +variable. + diff --git a/docs/ADVANCED_PLUGIN_TESTING b/docs/ADVANCED_PLUGIN_TESTING new file mode 100644 index 000000000..688426ae5 --- /dev/null +++ b/docs/ADVANCED_PLUGIN_TESTING @@ -0,0 +1,283 @@ +Advanced Plugin Testing +----------------------- + The complete guide to writing tests for your plugins. + +Why Write Tests? +================ + Why should I write tests for my plugin? Here's why. + +For those of you asking "Why should I write tests for my plugin? I tried it +out, and it works!", read on. For those of you who already realize that +Testing is Good (TM), skip to the next section. + +Here are a few quick reasons why to test your Supybot plugins. + + * When/if we rewrite or change certain features in Supybot, tests make + sure your plugin will work with these changes. It's much easier to run + supybot-test MyPlugin after upgrading the code and before even reloading + the bot with the new code than it is to load the bot with new code and + then load the plugin only to realize certain things don't work. You may + even ultimately decide you want to stick with an older version for a while + as you patch your custom plugin. This way you don't have to rush a patch + while restless users complain since you're now using a newer version that + doesn't have the plugin they really like. + + * Running the automated tests takes a few seconds, testing plugins in IRC + on a live bot generally takes quite a bit longer. We make it so that + writing tests generally doesn't take much time, so a small initial + investment adds up to lots of long-term gains. + + * If you want your plugin to be included in any of our releases (the core + Supybot if you think it's worthy, or our supybot-plugins package), it has + to have tests. Period. + +For a bigger list of why to write unit tests, check out this article: + + http://www.onjava.com/pub/a/onjava/2003/04/02/javaxpckbk.html + +and also check out what the Extreme Programming folks have to say about unit +tests: + + http://www.extremeprogramming.org/rules/unittests.html + +Plugin Tests +============ + How to write tests for commands in your plugins. + +Introduction + +This tutorial assumes you've read through the plugin author tutorial, and that +you used supybot-plugin-create to create your plugin (as everyone should). So, +you should already have all the necessary imports and all that boilerplate +stuff in test.py already, and you have already seen what a basic plugin test +looks like from the plugin author tutorial. Now we'll go into more depth about +what plugin tests are available to Supybot plugin authors. + +Plugin Test Case Classes + +Supybot comes with two plugin test case classes, PluginTestCase and +ChannelPluginTestCase. The former is used when it doesn't matter whether or +not the commands are issued in a channel, and the latter is used for when it +does. For the most part their API is the same, so unless there's a distinction +between the two we'll treat them as one and the same when discussing their +functionality. + +The Most Basic Plugin Test Case + +At the most basic level, a plugin test case requires three things: + + * the class declaration (subclassing PluginTestCase or + ChannelPluginTestCase) + * a list of plugins that need to be loaded for these tests (does not + include Owner, Misc, or Config, those are always automatically loaded) - + often this is just the name of the plugin that you are writing tests for + * some test methods + +Here's what the most basic plugin test case class looks like (for a plugin +named MyPlugin): + + class MyPluginTestCase(PluginTestCase): + plugins = ('MyPlugin',) + + def testSomething(self): + # assertions and such go here + +Your plugin test case should be named TestCase as you see above, though it +doesn't necessarily have to be named that way (supybot-plugin-create puts that +in place for you anyway). As you can see we elected to subclass PluginTestCase +because this hypothetical plugin apparently doesn't do anything +channel-specific. + +As you probably noticed, the plugins attribute of the class is where the list +of necessary plugins goes, and in this case just contains the plugin that we +are testing. This will be the case for probably the majority of plugins. A lot +of the time test writers will use a bot function that performs some function +that they don't want to write code for and they will just use command nesting +to feed the bot what they need by using that plugin's functionality. If you +choose to do this, only do so with core bot plugins as this makes distribution +of your plugin simpler. After all, we want people to be able to run your +plugin tests without having to have all of your plugins! + +One last thing to note before moving along is that each of the test methods +should describe what they are testing. If you want to test that your plugin +only responds to registered users, don't be afraid to name your test method +testOnlyRespondingToRegisteredUsers or testNotRespondingToUnregisteredUsers. +You may have noticed some rather long and seemingly unwieldy test method names +in our code, but that's okay because they help us know exactly what's failing +when we run our tests. With an ambiguously named test method we may have to +crack open test.py after running the tests just to see what it is that failed. +For this reason you should also test only one thing per test method. Don't +write a test method named testFoobarAndBaz. Just write two test methods, +testFoobar and testBaz. Also, it is important to note that test methods must +begin with test and that any method within the class that does begin with test +will be run as a test by the supybot-test program. If you want to write +utility functions in your test class that's fine, but don't name them +something that begins with test or they will be executed as tests. + +Including Extra Setup + +Some tests you write may require a little bit of setup. For the most part it's +okay just to include that in the individual test method itself, but if you're +duplicating a lot of setup code across all or most of your test methods it's +best to use the setUp method to perform whatever needs to be done prior to +each test method. + +The setUp method is inherited from the whichever plugin test case class you +chose for your tests, and you can add whatever functionality you want to it. +Note the important distinction, however: you should be adding to it and not +overriding it. Just define setUp in your own plugin test case class and it +will be run before all the test methods are invoked. + +Let's do a quick example of one. Let's write a setUp method which registers a +test user for our test bot: + + def setUp(self): + ChannelPluginTestCase.setUp(self) # important!! + # Create a valid user to use + self.prefix = 'foo!bar@baz' + self.feedMsg('register tester moo', to=self.nick, frm=self.prefix)) + m = self.getMsg() # Response to registration. + +Now notice how the first line calls the parent class's setUp method first? +This must be done first. Otherwise several problems are likely to arise. For +one, you wouldn't have an irc object at self.irc that we use later on nor +would self.nick be set. + +As for the rest of the method, you'll notice a few things that are available +to the plugin test author. self.prefix refers to the hostmask of the +hypothetical test user which will be "talking" to the bot, issuing commands. +We set it to some generically fake hostmask, and then we use feedMsg to feed +it a private message (using the bot's nick, accessible via self.nick) to +register the username "tester" with the password "moo". We have to do it this +way (rather than what you'll find out is the standard way of issuing commands +to the bot in test cases a little later) because registration must be done in +private. And lastly, since feedMsg doesn't dequeue any messages from the bot +after getting fed a message, we perform a getMsg to get the response. You're +not expected to know all this yet, but do take note of it since using these +methods in test-writing is not uncommon. These utility methods as well as all +of the available assertions are covered in the next section. + +So, now in any of the test methods we write, we'll be able to count on the +fact that there will be a registered user "tester" with a password of "moo", +and since we changed our prefix by altering self.prefix and registered after +doing so, we are now identified as this user for all messages we send unless +we specify that they are coming from some other prefix. + +The Opposite of Setting-up: Tearing Down + +If you did some things in your setUp that you want to clean up after, then +this code belongs in the tearDown method of your test case class. It's +essentially the same as setUp except that you probably want to wait to invoke +the parent class's tearDown until after you've done all of your tearing down. +But do note that you do still have to invoke the parent class's tearDown +method if you decide to add in your own tear-down stuff. + +Setting Config Variables for Testing + +Before we delve into all of the fun assertions we can use in our test methods +it's worth noting that each plugin test case can set custom values for any +Supybot config variable they want rather easily. Much like how we can simply +list the plugins we want loaded for our tests in the plugins attribute of our +test case class, we can set config variables by creating a mapping of +variables to values with the config attribute. + +So if, for example, we wanted to disable nested commands within our plugin +testing for some reason, we could just do this: + + class MyPluginTestCase(PluginTestCase): + config = {'supybot.commands.nested': False} + + def testThisThing(self): + # stuff + +And now you can be assured that supybot.commands.nested is going to be off for +all of your test methods in this test case class. + +Plugin Test Methods +=================== + The full list of test methods and how to use them. + +Introduction + +You know how to make plugin test case classes and you know how to do just +about everything with them except to actually test stuff. Well, listed below +are all of the assertions used in tests. If you're unfamiliar with what an +assertion is in code testing, it is basically a requirement of something that +must be true in order for that test to pass. It's a necessary condition. If +any assertion within a test method fails the entire test method fails and it +goes on to the next one. + +Assertions + +All of these are methods of the plugin test classes themselves and hence are +accessed by using self.assertWhatever in your test methods. These are sorted +in order of relative usefulness. + + * assertResponse(query, expectedResponse) - Feeds query to the bot as a + message and checks to make sure the response is expectedResponse. The + test fails if they do not match (note that prefixed nicks in the + response do not need to be included in the expectedResponse). + + * assertError(query) - Feeds query to the bot and expects an error in + return. Fails if the bot doesn't return an error. + + * assertNotError(query) - The opposite of assertError. It doesn't matter + what the response to query is, as long as it isn't an error. If it is + not an error, this test passes, otherwise it fails. + + * assertRegexp(query, regexp, flags=re.I) - Feeds query to the bot and + expects something matching the regexp (no m// required) in regexp with + the supplied flags. Fails if the regexp does not match the bot's + response. + + * assertNotRegexp(query, regexp, flags=re.I) - The opposite of + assertRegexp. Fails if the bot's output matches regexp with the + supplied flags. + + * assertHelp(query) - Expects query to return the help for that command. + Fails if the command help is not triggered. + + * assertAction(query, expectedResponse=None) - Feeds query to the bot and + expects an action in response, specifically expectedResponse if it is + supplied. Otherwise, the test passes for any action response. + + * assertActionRegexp(query, regexp, flags=re.I) - Basically like + assertRegexp but carries the extra requirement that the response must + be an action or the test will fail. + +Utilities + + * feedMsg(query, to=None, frm=None) - Simply feeds query to whoever is + specified in to or to the bot itself if no one is specified. Can also + optionally specify the hostmask of the sender with the frm keyword. + Does not actually perform any assertions. + + * getMsg(query) - Feeds query to the bot and gets the response. + +Other Tests +=========== + If you had to write helper code for a plugin and want to test it, here's + how. + +Previously we've only discussed how to test stuff in the plugin that is +intended for IRC. Well, we realize that some Supybot plugins will require +utility code that doesn't necessarily require all of the overhead of setting +up IRC stuff, and so we provide a more lightweight test case class, +SupyTestCase, which is a very very light wrapper around unittest.TestCase +(from the standard unittest module) that basically just provides a little +extra logging. This test case class is what you should use for writing those +test cases which test things that are independent of IRC. + +For example, in the MoobotFactoids plugin there is a large chunk of utility +code dedicating to parsing out random choices within a factoid using a class +called OptionList. So, we wrote the OptionListTestCase as a SupyTestCase for +the MoobotFactoids plugin. The setup for test methods is basically the same as +before, only you don't have to define plugins since this is independent of +IRC. + +You still have the choice of using setUp and tearDown if you wish, since those +are inherited from unittest.TestCase. But, the same rules about calling the +setUp or tearDown method from the parent class still apply. + +With all this in hand, now you can write great tests for your Supybot plugins! + diff --git a/docs/CAPABILITIES b/docs/CAPABILITIES new file mode 100644 index 000000000..4aeefba67 --- /dev/null +++ b/docs/CAPABILITIES @@ -0,0 +1,132 @@ +Introduction +------------ + +Ok, some explanation of the capabilities system is probably in order. With +most IRC bots (including the ones I've written myself prior to this one) "what +a user can do" is set in one of two ways. On the *really* simple bots, each +user has a numeric "level" and commands check to see if a user has a "high +enough level" to perform some operation. On bots that are slightly more +complicated, users have a list of "flags" whose meanings are hardcoded, and the +bot checks to see if a user possesses the necessary flag before performing some +operation. Both methods, IMO, are rather arbitrary, and force the user and the +programmer to be unduly confined to less expressive constructs. + +This bot is different. Every user has a set of "capabilities" that is +consulted every time they give the bot a command. Commands, rather than +checking for a user level of 100, or checking if the user has an 'o' flag, are +instead able to check if a user has the 'owner' capability. At this point such +a difference might not seem revolutionary, but at least we can already tell +that this method is self-documenting, and easier for users and developers to +understand what's truly going on. + +User Capabilities +----------------- +What the heck can these capabilities DO? + +If that was all, well, the capability system would be *cool*, but not many +people would say it was *awesome*. But it **is** awesome! Several things are +happening behind the scenes that make it awesome, and these are things that +couldn't happen if the bot was using numeric userlevels or single-character +flags. First, whenever a user issues the bot a command, the command dispatcher +checks to make sure the user doesn't have the "anticapability" for that +command. An anticapability is a capability that, instead of saying "what a +user can do", says what a user *cannot* do. It's formed rather simply by +adding a dash ('-') to the beginning of a capability; 'rot13' is a capability, +and '-rot13' is an anticapability. + +Anyway, when a user issues the bot a command, perhaps 'calc' or 'help', the bot +first checks to make sure the user doesn't have the '-calc' or the '-help' +(anti)capabilities before even considering responding to the user. So commands +can be turned on or off on a *per user* basis, offering fine-grained control +not often (if at all!) seen in other bots. This can be further refined by +limiting the (anti)capability to a command in a specific plugin or even an +entire plugin. For example, the rot13 command is in the Filter plugin. If a +user should be able to use another rot13 command, but not the one in the Format +plugin, they would simply need to be given '-Format.rot13' anticapability. +Similarly, if a user were to be banned from using the Filter plugin altogether, +they would simply need to be given the '-Format' anticapability. + +Channel Capabilities +-------------------- +What if #linux wants completely different capabilities from #windows? + +But that's not all! The capabilities system also supports *channel* +capabilities, which are capabilities that only apply to a specific channel; +they're of the form '#channel,capability'. Whenever a user issues a command to +the bot in a channel, the command dispatcher also checks to make sure the user +doesn't have the anticapability for that command *in that channel*, and if the +user does, the bot won't respond to the user in the channel. Thus now, in +addition to having the ability to turn individual commands on or off for an +individual user, we can now turn commands on or off for an individual user on +an individual channel! + +So when a user 'foo' sends a command 'bar' to the bot on channel '#baz', first +the bot checks to see if the user has the anticapability for the command by +itself, '-bar'. If so, it errors right then and there, telling the user that +he lacks the 'bar' capability. If the user doesn't have that anticapability, +then the bot checks to see if the user issued the command over a channel, and +if so, checks to see if the user has the antichannelcapability for that +command, '#baz,-bar'. If so, again, he tells the user that he lacks the 'bar' +capability. If neither of these anticapabilities are present, then the bot +just responds to the user like normal. + +Default Capabilities +-------------------- +So what capabilities am I dealing with already? + +There are several default capabilities the bot uses. The most important of +these is the 'owner' capability. This capability allows the person having it +to use *any* command. It's best to keep this capability reserved to people who +actually have access to the shell the bot is running on. It's so important, in +fact, that the bot will not allow you to add it with a command--you'll have you +edit the users file directly to give it to someone. + +There is also the 'admin' capability for non-owners that are highly trusted to +administer the bot appropriately. They can do things such as change the bot's +nick, cause the bot to ignore a given user, make the bot join or part channels, +etc. They generally cannot do administration related to channels, which is +reserved for people with the next capability. + +People who are to administer channels with the bot should have the +'#channel,op' capability--whatever channel they are to administrate, they +should have that channel capability for 'op'. For example, since I want +inkedmn to be an administrator in #supybot, I'll give him the '#supybot,op' +capability. This is in addition to his 'admin' capability, since the 'admin' +capability doesn't give the person having it control over channels. +'#channel,op' is used for such things as giving/receiving ops, kickbanning +people, lobotomizing the bot, ignoring users in the channel, and managing the +channel capabilities. The '#channel,op' capability is also basically the +equivalent of the 'owner' capability for capabilities involving +#channel--basically anyone with the #channel,op capability is considered to +have all positive capabilities and no negative capabilities for #channel. + +One other globally important capability exists: 'trusted'. This is a command +that basically says "This user can be trusted not to try and crash the bot." It +allows users to call commands like 'icalc' in the 'Math' plugin, which can +cause the bot to begin a calculation that could potentially never return (a +calculation like '10**10**10**10'). Another command that requires the 'trusted' +capability is the 're' command in the 'Utilities' plugin, which (due to the +regular expression implementation in Python (and any other language that uses +NFA regular expressions, like Perl or Ruby or Lua or ...) which can allow a +regular expression to take exponential time to process). Consider what would +happen if someone gave the bot the command 're [format join "" s/./ [dict go] +/] [dict go]' It would basically replace every character in the output of +'dict go' (14,896 characters!) with the entire output of 'dict go', resulting +in 221MB of memory allocated! And that's not even the worst example! + +Final Word +---------- + +From a programmer's perspective, capabilties are flexible and easy to use. Any +command can check if a user has any capability, even ones not thought of when +the bot was originally written. Plugins can easily add their own +capabilities--it's as easy as just checking for a capability and documenting +somewhere that a user needs that capability to do something. + +From an user's perspective, capabilities remove a lot of the mystery and +esotery of bot control, in addition to giving a bot owner absolutely +finegrained control over what users are allowed to do with the bot. +Additionally, defaults can be set by the bot owner for both individual channels +and for the bot as a whole, letting an end-user set the policy he wants the bot +to follow for users that haven't yet registered in his user database. It's +really a revolution! diff --git a/docs/USING_UTILS b/docs/USING_UTILS new file mode 100644 index 000000000..744ea278b --- /dev/null +++ b/docs/USING_UTILS @@ -0,0 +1,343 @@ +Using Supybot's utils module +---------------------------- +Supybot provides a wealth of utilities for plugin writers in the supybot.utils +module, this tutorial describes these utilities and shows you how to use them. + +str.py +====== +The Format Function + +The supybot.utils.str module provides a bunch of utility functions for +handling string values. This section contains a quick rundown of all of the +functions available, along with descriptions of the arguments they take. First +and foremost is the format function, which provides a lot of capability in +just one function that uses string-formatting style to accomplish a lot. So +much so that it gets its own section in this tutorial. All other functions +will be in other sections. format takes several arguments - first, the format +string (using the format characters described below), and then after that, +each individual item to be formatted. Do not attempt to use the % operator to +do the formatting because that will fall back on the normal string formatting +operator. The format function uses the following string formatting characters. + + * % - literal "%" + * i - integer + * s - string + * f - float + * r - repr + * b - form of the verb "to be" (takes an int) + * h - form of the verb "to have" (takes an int) + * L - commaAndify (takes a list of strings or a tuple of ([strings], and)) + * p - pluralize (takes a string) + * q - quoted (takes a string) + * n - n items (takes a 2-tuple of (n, item) or a 3-tuple of (n, between, item)) + * t - time, formatted (takes an int) + * u - url, wrapped in braces + +Here are a few examples to help elaborate on the above descriptions: + + >>> format("Error %q has been reported %n. For more information, see %u.", + "AttributeError", (5, "time"), "http://supybot.com") + + 'Error "AttributeError" has been reported 5 times. For more information, + see .' + + >>> i = 4 + >>> format("There %b %n at this time. You are only allowed %n at any given + time", i, (i, "active", "thread"), (5, "active", "thread")) + 'There are 4 active threads at this time. You are only allowed 5 active + threads at any given time' + + >>> i = 1 + >>> format("There %b %n at this time. You are only allowed %n at any given + time", i, (i, "active", "thread"), (5, "active", "thread")) + 'There is 1 active thread at this time. You are only allowed 5 active + threads at any given time' + + >>> ops = ["foo", "bar", "baz"] + >>> format("The following %n %h the %s capability: %L", (len(ops), "user"), + len(ops), "op", ops) + 'The following 3 users have the op capability: foo, bar, and baz' + +As you can see, you can combine all sorts of combinations of formatting +strings into one. In fact, that was the major motivation behind format. We +have specific functions that you can use individually for each of those +formatting types, but it became much easier just to use special formatting +chars and the format function than concatenating a bunch of strings that were +the result of other utils.str functions. + +The Other Functions + +These are the functions that can't be handled by format. They are sorted in +what I perceive to be the general order of usefulness (and I'm leaving the +ones covered by format for the next section). + + * ellipsisify(s, n) - Returns a shortened version of a string. Produces up + to the first n chars at the nearest word boundary. + - s: the string to be shortened + - n: the number of characters to shorten it to + + * perlReToPythonRe(s) - Converts a Perl-style regexp (e.g., "/abcd/i" or + "m/abcd/i") to an actual Python regexp (an re object) + - s: the regexp string + + * perlReToReplacer(s) - converts a perl-style replacement regexp (eg, + "s/foo/bar/g") to a Python function that performs such a replacement + - s: the regexp string + + * dqrepr(s) - Returns a repr() of s guaranteed to be in double quotes. + (Double Quote Repr) + - s: the string to be double-quote repr()'ed + + * toBool(s) - Determines whether or not a string means True or False and + returns the appropriate boolean value. True is any of "true", "on", + "enable", "enabled", or "1". False is any of "false", "off", "disable", + "disabled", or "0". + - s: the string to determine the boolean value for + + * rsplit(s, sep=None, maxsplit=-1) - functionally the same as str.split in + the Python standard library except splitting from the right instead of the + left. Python 2.4 has str.rsplit (which this function defers to for those + versions >= 2.4), but Python 2.3 did not. + - s: the string to be split + - sep: the separator to split on, defaults to whitespace + - maxsplit: the maximum number of splits to perform, -1 splits all + possible splits. + + * normalizeWhitespace(s) - reduces all multi-spaces in a string to a + single space + - s: the string to normalize + + * depluralize(s) - the opposite of pluralize + - s: the string to depluralize + + * unCommaThe(s) - Takes a string of the form "foo, the" and turns it into + "the foo" + - s: string, the + + * distance(s, t) - computes the levenshtein distance (or "edit distance") + between two strings + - s: the first string + - t: the second string + + * soundex(s, length=4) - computes the soundex for a given string + - s: the string to compute the soundex for + - length: the length of the soundex to generate + + * matchCase(s1, s2) - Matches the case of the first string in the second + string. + - s1: the first string + - s2: the string which will be made to match the case of the first + +The Commands Format Already Covers + +These commands aren't necessary because you can achieve them more easily by +using the format command, but they exist if you decide you want to use them +anyway though it is greatly discouraged for general use. + + * commaAndify(seq, comma=",", And="and") - transforms a list of items into + a comma separated list with an "and" preceding the last element. For + example, ["foo", "bar", "baz"] becomes "foo, bar, and baz". Is smart + enough to convert two-element lists to just "item1 and item2" as well. + - seq: the sequence of items (don't have to be strings, but need to + be 'str()'-able) + - comma: the character to use to separate the list + - And: the word to use before the last element + + * pluralize(s) - Returns the plural of a string. Put any exceptions to the + general English rules of pluralization in the plurals dictionary in + supybot.utils.str. + - s: the string to pluralize + + * nItems(n, item, between=None) - returns a string that describes a given + number of an item (with any string between the actual number and the item + itself), handles pluralization with the pluralize function above. Note + that the arguments here are in a different order since between is + optional. + - n: the number of items + - item: the type of item + - between: the optional string that goes between the number and the + type of item + + * quoted(s) - Returns the string surrounded by double-quotes. + - s: the string to quote + + * be(i) - Returns the proper form of the verb "to be" based on the number + provided (be(1) is "is", be(anything else) is "are") + - i: the number of things that "be" + + * has(i) - Returns the proper form of the verb "to have" based on the + number provided (has(1) is "has", has(anything else) is "have") + - i: the number of things that "has" + +structures.py +============= +Intro + +This module provides a number of useful data structures that aren't found in +the standard Python library. For the most part they were created as needed for +the bot and plugins themselves, but they were created in such a way as to be +of general use for anyone who needs a data structure that performs a like +duty. As usual in this document, I'll try and order these in order of +usefulness, starting with the most useful. + +The queue classes + +The structures module provides two general-purpose queue classes for you to +use. The "queue" class is a robust full-featured queue that scales up to +larger sized queues. The "smallqueue" class is for queues that will contain +fewer (less than 1000 or so) items. Both offer the same common interface, +which consists of: + + * a constructor which will optionally accept a sequence to start the queue + off with + * enqueue(item) - adds an item to the back of the queue + * dequeue() - removes (and returns) the item from the front of the queue + * peek() - returns the item from the front of the queue without removing + it + * reset() - empties the queue entirely + +In addition to these general-use queue classes, there are two other more +specialized queue classes as well. The first is the "TimeoutQueue" which holds +a queue of items until they reach a certain age and then they are removed from +the queue. It features the following: + + * TimeoutQueue(timeout, queue=None) - you must specify the timeout (in + seconds) in the constructor. Note that you can also optionally pass it a + queue which uses any implementation you wish to use whether it be one of + the above (queue or smallqueue) or if it's some custom queue you create + that implements the same interface. If you don't pass it a queue instance + to use, it will build its own using smallqueue. + * reset(), enqueue(item), dequeue() - all same as above queue classes + * setTimeout(secs) - allows you to change the timeout value + +And for the final queue class, there's the "MaxLengthQueue" class. As you may +have guessed, it's a queue that is capped at a certain specified length. It +features the following: + + * MaxLengthQueue(length, seq=()) - the constructor naturally requires that + you set the max length and it allows you to optionally pass in a sequence + to be used as the starting queue. The underlying implementation is + actually the queue from before. + * enqueue(item) - adds an item onto the back of the queue and if it would + push it over the max length, it dequeues the item on the front (it does + not return this item to you) + * all the standard methods from the queue class are inherited for this class + +The Other Structures + +The most useful of the other structures is actually very similar to the +"MaxLengthQueue". It's the "RingBuffer", which is essentially a MaxLengthQueue +which fills up to its maximum size and then circularly replaces the old +contents as new entries are added instead of dequeuing. It features the +following: + + * RingBuffer(size, seq=()) - as with the MaxLengthQueue you specify the + size of the RingBuffer and optionally give it a sequence. + * append(item) - adds item to the end of the buffer, pushing out an item + from the front if necessary + * reset() - empties out the buffer entirely + * resize(i) - shrinks/expands the RingBuffer to the size provided + * extend(seq) - append the items from the provided sequence onto the end + of the RingBuffer + +The next data structure is the TwoWayDictionary, which as the name implies is +a dictionary in which key-value pairs have mappings going both directions. It +features the following: + + * TwoWayDictionary(seq=(), **kwargs) - Takes an optional sequence of (key, + value) pairs as well as any key=value pairs specified in the constructor + as initial values for the two-way dict. + * other than that, no extra features that a normal Python dict doesn't + already offer with the exception that any (key, val) pair added to the + dict is also added as (val, key) as well, so the mapping goes both ways. + Elements are still accessed the same way you always do with Python + 'dict's. + +There is also a MultiSet class available, but it's very unlikely that it will +serve your purpose, so I won't go into it here. The curious coder can go check +the source and see what it's all about if they wish (it's only used once in our +code, in the Relay plugin). + +web.py +====== +The web portion of Supybot's utils module is mainly used for retrieving data +from websites but it also has some utility functions pertaining to HTML and +email text as well. The functions in web are listed below, once again in order +of usefulness. + + * getUrl(url, size=None, headers=None) - gets the data at the URL provided + and returns it as one large string + - url: the location of the data to be retrieved or a urllib2.Request + object to be used in the retrieval + - size: the maximum number of bytes to retrieve, defaults to None, + meaning that it is to try to retrieve all data + - headers: a dictionary mapping header types to header data + + * getUrlFd(url, headers=None) - returns a file-like object for a url + - url: the location of the data to be retrieved or a urllib2.Request + object to be used in the retrieval + - headers: a dictionary mapping header types to header data + + * htmlToText(s, tagReplace=" ") - strips out all tags in a string of HTML, + replacing them with the specified character + - s: the HTML text to strip the tags out of + - tagReplace: the string to replace tags with + + * strError(e) - pretty-printer for web exceptions, returns a descriptive + string given a web-related exception + - e: the exception to pretty-print + + * mungeEmail(s) - a naive e-mail obfuscation function, replaces "@" with + "AT" and "." with "DOT" + - s: the e-mail address to obfuscate + + * getDomain(url) - returns the domain of a URL + - url: the URL in question + +The Best of the Rest +==================== + Highlights the most useful of the remaining functionality in supybot.utils + +Intro + +Rather than document each of the remaining portions of the supybot.utils +module, I've elected to just pick out the choice bits from specific parts and +document those instead. Here they are, broken out by module name. + +supybot.utils.file - file utilities + + * touch(filename) - updates the access time of a file by opening it for + writing and immediately closing it + * mktemp(suffix="") - creates a decent random string, suitable for a + temporary filename with the given suffix, if + provided + * the AtomicFile class - used for files that need to be atomically + written, i.e., if there's a failure the original + file remains unmodified. For more info consult + file.py in src/utils + +supybot.utils.gen - general utilities + + * timeElapsed(elapsed, [lots of optional args]) - given the number of + seconds elapsed, returns a string with the English description of the + amount of time passed, consult gen.py in src/utils for the exact + argument list and documentation if you feel you could use this + function. + * exnToString(e) - improved exception-to-string function. Provides nicer + output than a simple str(e). + * InsensitivePreservingDict class - a dict class that is case-insensitive + when accessing keys + +supybot.utils.iter - iterable utilities + + * len(iterable) - returns the length of a given iterable + * groupby(key, iterable) - equivalent to the itertools.groupby function + available as of Python 2.4. Provided for + backwards compatibility. + * any(p, iterable) - Returns true if any element in the iterable satisfies + the predicate p + * all(p, iterable) - Returns true if all elements in the iterable satisfy + the predicate p + * choice(iterable) - Returns a random element from the iterable + +