Spyced

Earlier this year I wrote an introduction to SqlSoup , the SQLAlchemy extension that leverages SQLAlchemy's excellent introspection, mapping, and sql construction to provide a database interface that is both simple and powerful. Here's what SqlSoup has added since then (continuing with the books/loans/users example tables from pyweboff ). Full SqlSoup documentation is on the SQLAlchemy wiki . Set operations The introduction covered updating and deleting rows that had been mapped to Python objects. You can also perform updates and deletes directly to the database. >>> db.loans.insert(book_id=book_id, user_name=user.name) MappedLoans(book_id=2,user_name='Bhargan Basepair',loan_date=None) >>> db.flush() >>> db.loans.delete(db.loans.c.book_id==2) >>> db.loans.update(db.loans.c.book_id==2, book_id=1) >>> db.loans.select_by(db.loans.c.book_id==1) [MappedLoans(book_id=1,user_name='Joe Student',loan_date=datetime....

I don't follow Ruby development too closely (most of the info on it is still in Japanese, after all), but the US RubyConf was held recently so there's been an unusual number of English posts on Ruby, among them David Pollack's The Impending Ruby Fracture. David's article seems to consist of these points: Matz is uninterested in adding static bondage & discipline features to Ruby (true, as far as I know) "Enterprise" users won't be satisfied without said features (more on this below) There are a lot of Ruby runtimes out there right now (the most interesting part of the article) Therefore some Enterprise will co-opt one of the runtimes to fork Ruby and add the B&D features (wtf?) Summarized this way it looks faintly ridiculous, and yet nobody over on the programming reddit has called this out. Maybe I'm taking excessive liberties with David's article, but I don't think I am. The possibility of forking is part of what makes open sou...

I'm pleased that I made it into the round of 1000 in Google's Codejam . The Python support was much appreciated! (Actually, I'd be curious what the breakdown was by language of the contestants and qualifiers. I have no real stats but from the 50+ submissions I glanced at, all C#, Java, and C++, I would guess that topcoder's "Python might be too slow" disclaimer scared a lot of people away from Python. Or, like VB.NET, it just isn't that popular with this group. The ignomy!) I solved the 250 pt problem (problem set 5) very quickly, which is good because I didn't end up solving the 750 pt one at all. The 250 pt problem was You are given a tuple (integer) f that describes a set of values from a function f. The x-th element (zero-indexed) is the value of f(x). The function is not convex at a particular x value if there exist y and z values such that y Here was my brute force solution: class point: def __init__(self, x=0, y=0): self....

If you work in the web development area, or even dabble in it as a hobbyist, sooner or later you're going to write code for a project that needs authentication. Probably sooner than later. For a feature that gets used so frequently, it's remarkable to me that nobody has really done this right. Here are some basic principles for a good solution: A minimum of customization to work out-of-the-box Gentle complexity slope when more sophisticated behavior is needed Play nice with others Don't try to solve world hunger The first two are, I hope, no-brainers. The second two bear more explanation. Play nice with others: not everyone wants to authenticate against a Users table in a relational database. (Fairly common alternatives are LDAP or Unix logins.) If you bake in assumptions like this too deep, it causes problems. It might be worth the problems if it were impossible to provide both generality and ease of use, but such is not the case. Don't try to solve wo...

Most frameworks today have pretty good support for code/markup reuse towards providing a common look to a site without the "include a header, include a footer" clunkers you used to see. (Spyce uses "parent tags," which are as elegant as any and more powerful than most.) But what I want to talk about today is code/markup reuse in the small, at a level that corresponds to functions or methods in Python. Most frameworks today still suck at this. Approaches vary, but the important thing they have in common is not letting you use the same techniques you use in the rest of the framework as well as generally being clunky. For instance, with Rails, you define functions which you can then use from your views/templates/presentation layer, but within those functions you're back in 1996, stringing HTML together manually. Something like def faq_html(question, answers) html = ''' <table class="faq"> <thead> <tr...

Traditional view-oriented frameworks (such as PHP, or Spyce 1.3) do not handle control logic well. Today I'll show how Spyce 2 solves this problem with "active handlers," and tomorrow I'll show how this lets Spyce provide much more elegant code-reuse tools than either other view-oriented frameworks or MVC frameworks like Turbogears and Django. Control logic is the code that decides what happens next in response to a user action. Say we have a simple form to create new to-do lists, like this: <form> <input type=text name=name value=""> <input type=submit> </form> In a traditional view-oriented framework (like Spyce 1.3), you would process this form with code that looks something like this: [[name = request['name'] db.todo_lists.insert(name=name) api.db.flush() ]] That wasn't too bad. (Especially since I cheated and used the modern Spyce db api, AKA SqlSoup from SQLAlchemy.) But even with a simple, one-element for...

The revamed Spyce website announces "Spyce will not waste your time." Most web frameworks today waste your time with busywork at least some of the time. This is unacceptable; boilerplate code is tedious to write, an obstacle to good maintenance, and a distraction from real productivity. All the development of Spyce 2.0 and 2.1 has been towards eliminating common sources of busywork in web development. Today I'll give a couple concrete examples of how this applies to creating forms for user input. Let's say you want to have a select box that remembers what the user's last selection was. Pretty much any form these days needs logic like this when giving feedback inline. Many frameworks make you write this code out by hand; if you've ever developed in one of these, you know how quickly this gets old. Here's what you'd have to write in Spyce 2.1, given a list of options named options : <f:select name="foo" data="options...

Just some bug fixes this time: fix support for threadless python builds fix using compiled tags within other tags fix f:textarea fix spyceUtil.extractValue for incomplete dict work-alikes fix session1 backwards compatibility

This morning I read about the new Amazone Elastic Compute Cloud service. It's basically a cluster of Xen VPSes, done right. At least that's what I thought until I actually tried to use it. Let's see: Sign up for AWS Sign up for S3 Create certs Download tools Export 3 EC2 environment variables Oh, hell. Tools are in java. Switch to windows box since I don't have the patience to figure out installing Java 1.5 on debian right now. Repeat steps 4-5 Export JAVA_HOME Run ec2-describe-images Exception in thread "main" java.lang.NoClassDefFoundError: com/amazon/aes/webservices/client/cmd/DescribeImages Looks like I get to try to fix the classpath for them. How retro-cool can you get? It's just like 1999! Bah. Next time, use Python, guys. update: : Apparently they didn't even bother testing on windows and their script was just plain broken. Way to go.

For the first time, codejam 2006 features Python as a language option. I don't follow topcoder closely enough to know if this is the first contest they've done with Python or not, but this smells of Google influence to me. I was about resigned to not bothering with this year's code jam, with my C# even rustier than last year , but now I'm totally stoked. Go Python! (Oh yeah, link to main code jam page .) Update: for the curious, topcoder added Python support about 10 days ago

Check out What's new : Login tags SQLAlchemy integration Validation in handlers More powerful form tags Improved handler integration Pauli Virtanen has contributed debian scripts, so we'll have a .deb to go with the rpm and sourceball releases pretty soon.

I have a pg database that just suffered through a multiple-hour vacuum of one of its tables. It was pretty painful. Autovacuum is configured to run every 12 hours, but for whatever reason it didn't see fit to vacuum this very busy table for substantially longer than that. (Not sure exactly how long. Over a week is my guess.) The problem is that one size does not fit all. On smaller tables, the default autovacuum_vacuum_scale_factor of 0.4 is just fine. On this table waiting that long is unacceptable.

I'm losing patience with sqlite. I've been working on Spyce examples using postgresql, but now that I'm getting close to releasing Spyce 2.1, I figured I'd better convert the examples to use sqlite since that's such a no-brainer to set up. It has been a frustrating experience. Weird-assness I've run into includes "int" is not the same as "integer" (always use the latter to avoid unpleasant surprises) It's impossible to get useful information about column DEFAULTs programmatically There's a bug in the parsing of some three-way joins And I didn't think I was doing anything very complicated! My examples have three tables at most! Really my overall impression is one of a "0.9" product at best. I'm amazed that so many people appear to use this festering pile of gotchas in production.

Apparently PragDave had the temerity to suggest at RailsConf that Rails could stand some improvement in some areas, such as supporting composite primary keys in ActiveRecord. Naturally, the first reaction of a huge Rails fan like Martin Fowler is to get to work figuring out how to implement this. Whoops, sorry, no, that would be in some alternate universe where fanboyism isn't the most important technical prinicple for some people. Martin's real reaction was to write a rebuttal , the gist of which is, if Rails doesn't already support it, it can't be important. It's sad when someone who's done some good work puts on the fanboy blinders. You used to get the same schtick from mysql.com back in the 3.x days. Remember the rants in their docs about how foreign keys and transctions were for wimps and real programmers didn't want them anyway? Of course, today even mysql corporate admits that these are important features, even though one can be forgiven for ...

Andrew Smith pointed out that according to Indeed.com, Python is about a factor of 3 times more popular than Ruby and is maintaining that lead as both graphs trend upwards. I'd like to add just a couple things that I noticed. One is that, like Django , Rails is a term with multiple meanings, and the Ruby framework only accounts for a small fraction of jobs that Indeed pulls up for that term. (I'm impressed that Indeed allows you to nest arbitrarily complex boolean expressions here...) Another is that although Python looks pretty popular vs Ruby or Lisp, it's a good thing that popularity doesn't really reflect how good a language is, because ye olde statically compiled languages are still seven to twenty times more popular than python . Even PHP and Perl are more popular. (Although the trend on Perl is definitely down-sloping, for which we can all give thanks. C++ also has a noticable downward trend.)

I wrote a relatively simple multithreaded script to automate some cleanup work in my database. I used psycopg1, because it was conveniently packaged for the version of debian the server had. (And also because psycopg2's bundled pooling mechanism kind of sucks.) My script ran for a couple minutes, and segfaulted. I upgraded to the latest version of psycopg1, to no avail. You'd think that after 20+ "stable" releases this wouldn't be a problem anymore. Sigh. I ran it in gdb to see where it was segfaulting, and sure enough psycopg was dereferencing a null pointer. Unfortunately it was far from obvious how to fix the problem, at least to someone unfamiliar with the code. I bit the bullet and upgraded to psycopg2, which apparently got its first non-beta release earlier this month. For less-sucky pooling I used sqlalchemy's pool module. No more segfaults.

Greg Mullane has an excellent post on updating unique columns . A simple problem, but one that can be troublesome in practice: [T]here is one circumstance when [unique constraints] can be a real pain: swapping the values of the rows. In other words, a transaction in which the column values start unique, and end unique, but may not be so in the middle. Read his article -- I wouldn't have thought of his "reversing the polarity" method. Clever! But my first thought when I read this was, "Aha, Greg missed one." Surely the easiest way is to simply create a deferrable constraint (where you can elect to have the constraint only checked at the end of the transaction, instead of at the end of each statement)! So I gave it a try: => CREATE TABLE foo ( i int CONSTRAINT foo_pk PRIMARY KEY DEFERRABLE ); ERROR: misplaced DEFERRABLE clause At first I thought this indicated a syntax error, but my syntax was correct. After some g...

Python database tools have tended to suffer from the the 80% problem . (Open-source hackers tend to come up with solutions that solve 80% of a problem. Then someone else comes along and covers a different 80% of the same problem. And so on, so you end up with different solutions that attack the same problem, none of which are general enough for others to build on.) SQLAlchemy is making this a thing of the past, thanks to Mike Bayer's hard work. And, increasingly, others. SQLAlchemy made its second major release today, the big zero-dot-two-oh. (Mike is conservative with version numbers; most projects would call this 0.9 if not 1.0.) There's also a migration guide for porting 0.1-based code. SQLAlchemy lives up to its billing as The Python SQL Toolkit and Object Relational Mapper. (Emphasis mine.) This is possible is because of the extensive under-the-hood effort Mike has expended keeping dependencies to a minimum; you really can build on its functionality at any level...

My friend Paul introduced Noodle at the Utah python user group a week ago. (Yeah, sorry about the not-exactly-breaking-news.) Paul's a Lisp expert -- I think from before he was a Python expert, but I'm actually not sure of the chronology there -- and he wrote Noodle to create a pythonic Lisp dialect. Noodle combines Lisp syntax and features like macros with Python-ish syntax for lists, dicts, and tuples, and compiles to Python bytecode so it can easily leverage all the Python libraries. This bears repeating: there are a lot of projects out there that try to produce Java bytecode or CIL, but Noodle is the first I've heard of that produces Python bytecode. Pretty cool, if you ask me. His slides are linked from his blog, but basically his conclusion so far is that it turns out to be harder to integrate python-style syntax into Lisp than he'd hoped. Not hard as in implementation, but hard as in making it non-clunky to use. The warts are small small individually bu...

I presented on PyGame at the utah python user group last night. (When I don't get someone else lined up to speak in advance, I end up doing it myself. You'd think that would be enough motivation to not procrastinate.) I had a lot of fun preparing this. I'd never used PyGame before, but as a teenager I spent a lot of time in the same space. (Anyone remember YakIcons?) So the general concepts were familiar to me, and I was pleasantly surprised by how good a job PyGame did at making things easy for me. Here are my pygame slides , and my pyquest game skeleton is here . (PyQuest is of course inspired by Crystal Quest -- the mac game, not the XB 360 remake -- and the graphics and sound files are from Solar Wolf, which I guess makes PyQuest LGPL. This caused Paul Cannon some serious mental trauma at the meeting, seeing and hearing solar-wolf-and-yet-not-solar-wolf.)