Wednesday, August 30, 2006

Spyce will not waste your time: authentication

If you work in the web development area, or even dabble in it as a hobbyist, sooner or later you're going to write code for a project that needs authentication. Probably sooner than later.

For a feature that gets used so frequently, it's remarkable to me that nobody has really done this right. Here are some basic principles for a good solution:

  1. A minimum of customization to work out-of-the-box
  2. Gentle complexity slope when more sophisticated behavior is needed
  3. Play nice with others
  4. Don't try to solve world hunger

The first two are, I hope, no-brainers. The second two bear more explanation.

Play nice with others: not everyone wants to authenticate against a Users table in a relational database. (Fairly common alternatives are LDAP or Unix logins.) If you bake in assumptions like this too deep, it causes problems. It might be worth the problems if it were impossible to provide both generality and ease of use, but such is not the case.

Don't try to solve world hunger. By this I mean, don't invent a Grand Unified Authorization/Permissions scheme with Groups and Subgroups and Roles and ACLs that supposedly can handle any needed authorization requirements, but requires a week of studying your system to make it work. There have been several attempts by very smart people to make this work and it hasn't, yet. So unless you are a grad student or in some other position that leaves you with too much time on your hands, don't go there. Instead, make it easy for the application developer to implement whatever he needs on top of your authentication code.

With that introduction, here's what you need to do to enable authentication for your Spyce app:

  1. Invoke the spy:login (or spy:login_required) tag in the pages you wish to restrict to authenticated users
  2. Write a function that, given a username / password combination, returns a pickle-able identifier, or None; make this function the login_defaultvalidator in your Spyce config file. Like this one, used in the pyweboff demo (which does, as it happens, store user information in a database):
    def validator(login, password):
       user = db.users.selectone_by(name=login, password=password)
       if user:
       return None
    login_defaultvalidator = validator

That's it! Out of the box, Spyce handles the cookies, the login forms and handlers, all of the details you shouldn't have to worry about! The form rendering is done by active tags, which we talked about yesterday, so it's easy to customize when you get to that point. And if you want to restrict some pages to a subset of all users, you'd just write a validation function expressing those constraints, and pass that function to spy:login on those pages.

Here's a live example. (Login as spyce/spyce.)

If you'd like to compare this approach with other frameworks, here are some relevant links:

  • Ruby on Rails: "Don't start here until you 'get' it. Most of the authentication guides are broken or poorly documented." Good luck! Or maybe you'd like to buy our book!
  • Turbogears: (updated link) there is a lot to read here but Karl assures me that only a couple steps are actually necessary, besides the database setup.
  • Django: you have to do a lot of reading to find out what the real minimum work to do is, but I count 5 steps in 3 files.
  • I'll throw in ASP.NET for good measure: it's actually not too bad, but you're still on your own writing the login form and handler.

Tomorrow I'll talk about the Spyce db module, shown briefly above in the sample validation function. (And also mentioned in passing when I talked about Spyce handlers.)

Tuesday, August 29, 2006

Spyce will not waste your time: code reuse

Most frameworks today have pretty good support for code/markup reuse towards providing a common look to a site without the "include a header, include a footer" clunkers you used to see. (Spyce uses "parent tags," which are as elegant as any and more powerful than most.)

But what I want to talk about today is code/markup reuse in the small, at a level that corresponds to functions or methods in Python.

Most frameworks today still suck at this. Approaches vary, but the important thing they have in common is not letting you use the same techniques you use in the rest of the framework as well as generally being clunky.

For instance, with Rails, you define functions which you can then use from your views/templates/presentation layer, but within those functions you're back in 1996, stringing HTML together manually. Something like

def faq_html(question, answers)
    html = '''
    <table class="faq">
    ''' % question
    for a in answers:
        html += '<tr><td>A.</td><td>%s</td></tr>' % a
    html += '</table>'
    return html

(Edit: Rails also has a code reuse mechanism called "partials," which behave a lot more like normal rhtml code.)

In Spyce, you can still do things this old-fashioned way if you really want. But a much better way is defining "active tags," which take parameters like functions but are defined with the same tools you build the rest of your page with, i.e., Python and other tags. Such as:

[[.begin name=html singleton=True ]]
[[.attr name=question ]]
[[.attr name=answers ]]

<spy:table class="faq" data="(('A.', a) for a in answers)">
  <tr><th>Q.</th><th>[[= question ]]</th></tr>

[[.end ]]

In a tag library named "faq", this active tag could then be used as follows: (the "=" sign means, "evaluate this parameter as a Python expression instead of as a string)

<faq:html question="=question.body" answers="=[a.body for a in answers]" />

The more complex your code becomes, the more you'll appreciate the Spyce way. You'll find that with the increased programmer-friendliness active tags provide, you'll abstract more common code than you would have with clumsier tools, with the all the improved maintainability that entails over code-by-copy-and-paste.

But what's really cool is you can encapsulate handlers with your tags, creating self-contained bundles of render and control logic. This is crucial: there's no way to do this with the traditional "helper function" approach. Splitting functionality out into active tag components like this allows both reuse and encapsulation, just as classes provide similar benefits in pure Python code.

(We talked about Spyce handlers yesterday. Briefly: Spyce allows you to attach a python function to a form submit action.)

To continue our FAQ example, let's add a "Create a new FAQ" handler, like this:

def faq_new(self, api, question, answer):
    q = api.db.questions.insert(body=question)
    api.db.answers.insert(, body=answer)

[[.begin name=faq_html singleton=True ]]
[[.attr name=question ]]
[[.attr name=answers ]]

<spy:table class="faq" data="(('A.', a) for a in answers)">
  <tr><th>Q.</th><th>[[= question ]]</th></tr>

<h2>New FAQ</h2>
<div><f:text name=question label="Question:" /></div>
<div><f:text name=answer label="Answer:" /></div>
<div><f:submit handler=self.faq_new value="Create" /></div>
[[.end ]]

Spyce provides friendlier and more powerful tools for code reuse than either other view-oriented frameworks or MVC frameworks. Give it a try; you won't want to go back.

Tomorrow I'll explain the Spyce authentication system.

Monday, August 28, 2006

Spyce will not waste your time: controllers/handlers

Traditional view-oriented frameworks (such as PHP, or Spyce 1.3) do not handle control logic well. Today I'll show how Spyce 2 solves this problem with "active handlers," and tomorrow I'll show how this lets Spyce provide much more elegant code-reuse tools than either other view-oriented frameworks or MVC frameworks like Turbogears and Django.
Control logic is the code that decides what happens next in response to a user action. Say we have a simple form to create new to-do lists, like this:
<input type=text name=name value="">
<input type=submit>
In a traditional view-oriented framework (like Spyce 1.3), you would process this form with code that looks something like this:
[[name = request['name']
That wasn't too bad. (Especially since I cheated and used the modern Spyce db api, AKA SqlSoup from SQLAlchemy.) But even with a simple, one-element form like this, a couple things are clear:
  • code like "name = request['name']" must be written for each form element. Irritating!
  • putting this chunk of code in the page with the form on it sucks (because you have to "if" it out if you're not actually processing the form POST)
  • putting this code on another page that doesn't actually do anything else sucks too (because then you have a view-that's-not-really-a-view, which is ugly -- to say nothing of existentially disturbing)
The Spyce solution is to allow you to specify Python functions, called handlers, that run on form submission. So your submit will now look like
<f:submit handler=list_new />
(note the Spyce form submit active tag; the vanilla html submit tag won't understand the handler parameter, of course), and you can define your handler as
def list_new(api, name):
This takes care of the redundant code to access form variables -- Spyce inspects the function definition and pulls the corresponding values out of the request for you -- and it also solves the problem of how to organize your code.
Earlier, I said you could write "handler=list_new." Actually, you either need to write "handler=self.list_new" and define list_new in a class chunk in the current page (as demonstrated in this example), or you can write "handler=somemodule.list_new," and put list_new in somemodule. (In the spirit of Not Wasting Your Time, Spyce will take care of importing somemodule as necessary.)
Pretty simple stuff, but I needed to explain handlers before I get into how Spyce uses them with active tags to provide unique code re-use tools. That will be our topic next time! Until then, if you missed the entry on form processing, go read it, since it touches on some of the more advanced features of handlers.

Friday, August 25, 2006

Spyce will not waste your time: form processing

The revamed Spyce website announces "Spyce will not waste your time."

Most web frameworks today waste your time with busywork at least some of the time. This is unacceptable; boilerplate code is tedious to write, an obstacle to good maintenance, and a distraction from real productivity. All the development of Spyce 2.0 and 2.1 has been towards eliminating common sources of busywork in web development.

Today I'll give a couple concrete examples of how this applies to creating forms for user input.

Let's say you want to have a select box that remembers what the user's last selection was. Pretty much any form these days needs logic like this when giving feedback inline. Many frameworks make you write this code out by hand; if you've ever developed in one of these, you know how quickly this gets old.

Here's what you'd have to write in Spyce 2.1, given a list of options named options:

<f:select name="foo" data="options" />

That's it.

For comparison, here's what this would have looked like in Spyce version 1.2, some time in 2002. Many frameworks still make you do it much the same way today:

<select name="foo">
  [[ for name, value in options:{ ]]
     <option value="[[= value ]]"[[ if value == request['foo']:{ ]] selected[[ } ]]>[[= name ]]</option>
  [[ } ]]

Going back to the modern Spyce: notice the data attribute? All Spyce tags that you would expect to use a for loop with, now take a data attribute that can be any Python iterable. You can still render the contents of a select or a ul or a table manually, but Spyce saves you time for the common case.

Spyce also takes the boilerplate out of giving feedback to users. I'm unaware of any other framework that makes this as easy as Spyce; perhaps the closest is ASP.NET, but even that seems pretty clunky to me after using Spyce for a while.

All you need to do is raise a HandlerError from your handler, and Spyce knows how to render it with the associated form with no further action on your part:

    raise HandlerError('choice', 'None selected')

The rendering of errors is easily customized, and you can raise CompoundHandlerError if there's more than one problem.

There's a live example over here; click on "Run this code" and see what it does when you ask for the square root of a negative number.

I'll leave comparing this with the hoops your current framework makes you jump through as an exercise for the reader.

Thursday, August 24, 2006

Spyce 2.1.2 released

Just some bug fixes this time:

  • fix support for threadless python builds
  • fix using compiled tags within other tags
  • fix f:textarea
  • fix spyceUtil.extractValue for incomplete dict work-alikes
  • fix session1 backwards compatibility

Amazon EC2: How much less userfriendly can you make it?

This morning I read about the new Amazone Elastic Compute Cloud service. It's basically a cluster of Xen VPSes, done right.

At least that's what I thought until I actually tried to use it.

Let's see:

  1. Sign up for AWS
  2. Sign up for S3
  3. Create certs
  4. Download tools
  5. Export 3 EC2 environment variables
  6. Oh, hell. Tools are in java. Switch to windows box since I don't have the patience to figure out installing Java 1.5 on debian right now.
  7. Repeat steps 4-5
  8. Export JAVA_HOME
  9. Run ec2-describe-images
  10. Exception in thread "main" java.lang.NoClassDefFoundError: com/amazon/aes/webservices/client/cmd/DescribeImages

Looks like I get to try to fix the classpath for them. How retro-cool can you get? It's just like 1999!

Bah. Next time, use Python, guys.

update:: Apparently they didn't even bother testing on windows and their script was just plain broken. Way to go.

Monday, August 14, 2006

Hell yes: Google codejam does Python

For the first time, codejam 2006 features Python as a language option.

I don't follow topcoder closely enough to know if this is the first contest they've done with Python or not, but this smells of Google influence to me.

I was about resigned to not bothering with this year's code jam, with my C# even rustier than last year, but now I'm totally stoked. Go Python!

(Oh yeah, link to main code jam page.)

Update: for the curious, topcoder added Python support about 10 days ago

Sunday, August 13, 2006

Spyce 2.1 released

Check out What's new:

  • Login tags
  • SQLAlchemy integration
  • Validation in handlers
  • More powerful form tags
  • Improved handler integration

Pauli Virtanen has contributed debian scripts, so we'll have a .deb to go with the rpm and sourceball releases pretty soon.

Thursday, August 10, 2006

Postgresql: don't rely on autovacuum

I have a pg database that just suffered through a multiple-hour vacuum of one of its tables. It was pretty painful. Autovacuum is configured to run every 12 hours, but for whatever reason it didn't see fit to vacuum this very busy table for substantially longer than that. (Not sure exactly how long. Over a week is my guess.)

The problem is that one size does not fit all. On smaller tables, the default autovacuum_vacuum_scale_factor of 0.4 is just fine. On this table waiting that long is unacceptable.