Spyced

Sure, some things about Python bug me. But that's not what this is about. I wanted to react to Jacob Kaplan-Moss's gripes instead of promulgating my own. Specifically, his problem with Python's interfaces, or lack thereof. I think I can keep this brief: interfaces are a hack that Java uses because Gosling et al thought multiple inheritance was too confusing and/or dangerous. (I believe I've read something recently where Gosling said that this was one decision he might do differently if he were re-designing Java now with the benefit of hindsight, but I can't find the source. Anyone remember seeing that?) Python has MI. It doesn't need interfaces. I'm a little baffled that someone on the django core team would cite this as a problem with Python. Jacob's precise objection is, I shouldn’t need to care care about the difference between something that pretends to be a list and something that really is a list. That's just it! You don't...

I'll be giving a (very!) introductory Python workshop at the Utah Geographic Information Council conference in April. After my 90 minutes, Kevin Bell -- also of the utah python user group -- will present on specific GIS applications. (Apparently Python is particularly big in GIS these days because one of the big vendors, ERSI, takes Python pretty seriously .)

I represented Spyce for the web frameworks panel. It was pretty cool looking out at the standing-room-only crowd, even though let's face it, most people were not there because of Spyce. :) James Bennett and Matt Harrison have notes posted online.

I uploaded my slides for my Sunday SqlSoup talk, so they're linked in the schedule now. I also uploaded them here . Update: the profile.py module I showed is here .

I'll demonstrate writing a simple app with Spyce, including the most painless Ajax you have ever seen. Come check it out at 3:40 in the Bent Tree II room.

My SQLAlchemy tutorial went pretty well for the most part. It was a fast pace but most people kept up pretty well. If I did it again I would add more of an intro to ORM in general for people who had never used one, but over half the attendees had used SO or django's or tried SA already. I would also paste more code from my slides into the samples download to save people typing during the exercises (I had some, but I would do more next time). I think most people liked it; the main exception was one fellow who was in way way over his head and visibly pissed about it. (I used a list comprehension at one point and he had no idea what it was.) The slides are here. (The .py files referred to in the slides have also been moved to the jellis/ subdirectory.)

I'll be representing Spyce as a late addition to the Web Frameworks panel. I'm also planning a lightning talk on Ajax in Spyce 2.2 (which will be released as soon as I finish getting the docs in shape) and an open-space Introduction to Spyce. See you there!

My wife got me a PSP for Valentine's Day, so I'm looking for videos to put on it. Since Google makes it so easy to get a PSP-compatible version, I thought I'd start with theirs... Recommendations?

I presented on SQLAlchemy at the Utah python user group last Thursday; slides are linked here . In retrospect, for a shorter presentation like this I should probably spend more time talking about the ORM features, and less about the SQL layer. Although the SQL layer is useful on its own, and essential for doing advanced mapping, I don't think it has the sex appeal that the ORM has. (Although I do think the first part, about why ORMs should allow you to take advantage of your database's strengths rather than being limited to a MySQL 3 feature set, was useful.)

ActiveState has released Komodo IDE 4 . Perhaps more interesting, if you're not already a Komodo user, is the release of Komodo Edit , which is very similar to the old Komodo IDE Personal edition, only instead of costing around $30, Komodo Edit is free. The mental difference between "free" and "$30" is much more than the relatively small amount of money; it will be interesting to see what happens in the IDE space now. After a brief evaluation I would say Edit is perhaps the strongest contender for "best free python IDE." The only serious alternative is PyDev, which on its Eclipse foundation provides features like svn integration that Edit doesn't. PyDev also includes a debugger, another feature ActiveState would like to see you upgrade to the full IDE for. But Komodo is stronger in other areas such as call tips and, well, not being based on Eclipse. I also think its code completion is better, although this impression is preliminary. It...

See you there!

I was pretty happy with the old version of blogger, but I upgraded today so I can use the new API against my own blog. So far I have 4 spam comments (captcha is still on) versus about that number for the entire life of my blog under the old blogger. Bleh. Could just be a coincidence. I hope so. (Update Feb 26: A month later, I've had just one more spam comment. So it probably really was just coincidence.)

In December, Fujitsu made available a video of Gavin Sherry speaking on Advanced PostgreSQL . (Where's part 2, guys?) Here's some of the topics Gavin addresses, and the approximate point at which they can be found in the video. [start] wal_buffers: "at least 64"; when it's ok to turn fsync off [not very often]; how hard disk rpm limits write-based transaction rate, even with WAL 00:12: wal_sync_method = fdatasync is worth checking out on Linux 00:13: FSM [free space map], MVCC, and vacuum; how to determine appropriate FSM size; why this is important to avoid VACUUM FULL 00:22: vaccum_cost_delay 00:26: background writer 00:30: history of buffer replacement strategies 00:37: scenarios where bgwriter is not useful 00:41: how random_page_cost affects planner's use of indexes 00:47: effective_cache_size 00:49: logging; how to configure syslog to not hose your performance 00:52: linux file system configuration 00:58: solaris fs...

One of the reasons ORM tools have a spotted reputation is that it's really, really easy to write a dumb ORM that works fine for simple queries but performs like molasses once you start throwing real data at it. Let me give an example of a situation where, to my knowledge, only SQLAlchemy of the Python (or Ruby) ORMs is really able to handle things elegantly, without gross hacks like "piggy backing." Often you'll see a one-to-many relationship where you're not always interested in all of the -many side. For instance, you might have a users table, each associated with many orders. In SA you'd first define the Table objects, then create a mapper that's responsible for doing The Right Thing when you write "user.orders." (I'm skipping connecting to the database for the sake of brevity, but that's pretty simple. I'm also avoiding specifying columns for the Tables by assuming they're in the database already and telling SA to a...

Vadim Tkachenko posted an interesting benchmark of MyISAM vs InnoDB vs Falcon datatypes. (Falcon is the new backend that MySQL started developing after Oracle bought InnoDB.) For me the interesting part is not the part with the alpha code -- Falcon is competitive for some queries but gets absolutely crushed on others -- but how InnoDB is around 30% faster than MyISAM. And these are pure selects, supposedly where MyISAM is best. Of course this is a small benchmark and YMMV, but this is encouraging to me because it suggests that if I ever have to use MySQL, I can use a backend with transactions, real foreign key support, etc., without sucking too badly performance-wise. (It also suggests that people who responded to the post on postgresql crushing mysql in a different benchmark by saying, "well, if they wanted speed they should have used MyISAM," might want to reconsider their advice.)

I thought I was pretty used to SQL's three-valued logic by now, but this still caused me a minute of scratching my head: # select count(*) from _t; count ------- 1306 (1 row) # select count(*) from _t2; count ------- 19497 (1 row) Both _t and _t2 are temporary tables of a single column I created with SELECT DISTINCT. # select count(*) from _t where userhash in (select userhash from _t2); count ------- 982 (1 row) # select count(*) from _t where userhash not in (select userhash from _t2); count ------- 0 (1 row) Hmm, 982 + 0 != 1306... Turns out there was a null in _t2; X in {set containing null} evaluates to null, not false, and negating null still gives null. (The rule of thumb is, any operation on null is still null.) ................. I'm giving a tutorial on Advanced Databases with SQLAlchemy at PyCon in February. Feel free to let me know if there is anything you'd like me to cover specifically.

My thinkpad R52's screen died a couple days ago. I decided that this time I was going to be a man and install Linux on my new machine: all our servers run Debian, and "apt-get install" is just so convenient vs manual package installation on Windows. And it looks like qemu is a good enough "poor man's vmware" that I could still test stuff in IE when necessary. Alas, it was not to be. My new laptop is an HP dv9005, and although ubuntu's livecd mode ran fine, when it actually installed itself to the HDD and loaded X it did strange and colorful things to the LCD. Things that didn't resemble an actual desktop. When I told it to start in recovery mode instead it didn't even finish booting. That was all the time I had to screw around, so I reinstalled Windows to start getting work done again. Which brings me (finally!) to this advice on tortoisesvn : it really puts teh snappy back in the tortoise. Thanks annonymous progblogger!

I don't blog about my day job at Mozy much, but I can't pass this one up: Walt Mossberg reviewed Mozy vs Carbonite in today's Wall Street Journal, and he concluded "of the two products, I prefer Mozy." "The Walt effect" makes the Digg effect look like small potatoes.

As of Komodo 4.0 beta 2 , Komodo no longer includes ActiveState's Tk-based GUI builder (with support for non-tcl Tk bindings like Python's Tkinter). They've released it as open source to the SpecTcl project, from which it was apparently forked long ago. Re-integration with Komodo 4 as a plugin is planned eventually, but it doesn't look likely before the 4.0 release. I wonder how much of the impetus behind this is the increased amount of web-based development done today, and how much is due to Tk's increasingly dated look and the increased popularity of more modern toolkits such wxwidgets.

I needed to scan some pretty massive gzipped text files, so my first try was the obvious "for line in gzip.open(...)." This worked but seemed way slower than expected. So I wrote "pyzcat" as a test and ran it against a file with 100k lines: #!/usr/bin/python import sys, gzip for fname in sys.argv[1:]: for line in gzip.open(fname): print line, Results: $ time zcat testzcat.gz > /dev/null real 0m0.329s $ time ./pyzcat testzcat.gz > /dev/null real 0m3.792s 10x slower -- ouch! Well, if zcat is so much better, let's try using zcat to do the reads for us: def gziplines(fname): from subprocess import Popen, PIPE f = Popen(['zcat', fname], stdout=PIPE) for line in f.stdout: yield line for fname in sys.argv[1:]: for line in gziplines(fname): print line, Results: $ time ./pyzcat2 testzcat.gz |wc real 0m0.750s So, reading from a zcat subprocess is 5x faster than using the gzip module. cGzipFile anyo...