class foo(list):
def __eq__(self, other):
raise 'foo.__eq__ called'
>>> help(list.__eq__)
Help on wrapper_descriptor:
__eq__(...)
x.__eq__(y) <==> x==y
>>> [].__eq__(foo())
True
>>> [] == foo()
Traceback (most recent call last):
File "", line 1, in ?
File "", line 3, in __eq__
foo.__eq__ called
Help says those two statements should be equivalent. Why aren't they?
One of the ways to extend Spyce is with a "Spyce module." This is a historical term, and a little unfortunate because some people have assumed when I talk about "modules" I automatically mean the Spyce variety rather than the vanilla python variety.
A Spyce module is simply a class that extends spyceModule.spyceModule. That's it.
Spyce modules may be used in a .spy page with
[[.import name="modulename"]]
which instructs the compiler to create code that instantiates an instance of the given spyceModule at the beginning of each request for this page. It also automatically invokes the instance's finish method when the request finishes.
What can you do with Spyce modules? A common reason to write a Spyce module -- perhaps the most common -- is is to provide some sort of resource pooling. Here's the spydurus module which does exactly that:
spydurus.py
CONNECTIONS = 3 # max connections to put in the pool
import Queue
from spyceModule import spyceModule
from durus.client_storage import ClientStorage
from durus.connection import Connection
q = Queue.Queue()
for i in range(CONNECTIONS):
q.put(Connection(ClientStorage()))
class spydurus(spyceModule):
def start(self):
self.conn = None
try:
self.conn = q.get(timeout=10)
except Queue.Empty:
raise 'timeout while getting durus connection'
else:
self.conn.abort() # syncs connection
def finish(self, err):
q.put(self.conn)
That's it! (Well, minus some convenience methods dealing with self.conn that are elided for concision here.) Note that finish is always called, even if your code raises an exception, even if you redirect to another page, even if another module's finish method raises.
Most modules can also be usefully used as a python module -- for instance, tododata.py in the Spyce to-do demo uses spydurus.q to get one of the pooled connections in a non-.spy context.
spydurus doesn't need it, but spyceModule also provides a hook into the spyce server in the form of self._api. You can do just about anything from this. I've seen a module that implements cgi_buffer functionality, a module that provides a Struts-style controller, and more. And of course there's all the standard modules.
(Updated links to point to the sf.net site.)
I was reminded of this one while working on the Spyce/Durus demo to make it a better example of using Durus in a real application.
What is wrong with the following code?
A.py:
import time, threading
finished = False
def foo():
import sys
sys.stderr.write('testing')
global finished
finished = True
threading.Thread(target=foo).start()
while not finished:
time.sleep(1)
B.py:
import A print 'finished'
Update: fixed A.py so there was only one problem.
There won't be an official beta this week after all; Rimon wants to review the code and docs over the weekend before we take that step.
However, as an unofficial beta, Spyce 2.0 is available from subversion:
svn co http://svn-hosting.com/svn/spyce/trunk/spyce
The featureset is solid at this point. If you start playing around now, you won't get burned by changes of that sort. You may well find bugs, of course, which I'll fix as soon as I can.
The docs are up here, including a pretty good introduction as to Why Spyce 2.0 Rocks, if I say so myself. (That is, I say the intro is pretty good. But also that Spyce2 rocks, for that matter.) The to-do demo and an enhanced version of the chatbox demo are both included.
Update: forgot a link to the changelog.
You've taken spyce off my "disgusting template engines" list. This is how I'd want to see apps structured, exactly.
-- Tim Lesher
As a followup to my Durus notes, I've put up a new Spyce + Durus demo showing off the features of Spyce 2.0.
This time I allowed myself more than two files, and split the action logic into a separate module. (To-do lists seem to be the new standard for demoing your web toolkit, and I dare not be less than trendy.)
I tried to comment the code (linked from the demo pages) so it speaks for itself. If you have questions after looking at that and the language constructs page, please feel free to ask in the comments here. I'd like the docs to be in good shape for the 2.0 release.
Update: I've modified the demo to include spydurus, a Durus connection-pooling module for Spyce, and to use ClientStorage instead of FileStorage. The demo was popular enough that single-threading it got to be annoying.
(Updated links to point to the spyce sf.net site.)
The README that comes with Durus is missing a couple pieces of information that are critical if you actually want to write a program that uses Durus, and the only other documentation appears to be the 2005 Durus pycon presentation, which gives an admirable description of the technical underpinnings but doesn't fill in the blanks of how to use it, either.
Specifically, as far as code samples go, the README gives you this:
Example using FileStorage to open a Connection to a file:
from durus.file_storage import FileStorage
from durus.connection import Connection
connection = Connection(FileStorage("test.durus"))
And this:
# Assume mymodule defines A as a subclass of Persistent.
from mymodule import A
x = A()
root = connection.get_root() # connection set as shown above.
root["sample"] = x # root is dict-like
connection.commit() # Now x is stored.
That's all you get.
Okay, in this situation it's clear how to retrieve x -- look it up by the key we've hardcoded. But what if x is a list of Persistent objects, and I want to retrieve one of those. Do I have to keep track of its index in x? Do I need to generate my own unique IDs? (Note that the linked code has an obvious race condition in a multithreaded environment.)
No, you don't. Durus assigns each Persisent object an attribute called _p_oid. (P object id. Not sure if P stands for Persistent, or Private, or something else entirely.)
[in persistent.py]
def __new__(klass, *args, **kwargs):
[...]
instance._p_oid = None # <-- this is the oid that get() cares about
def _p_format_oid(self):
return format_oid(self._p_oid)
_p_oid is a four-byte binary string, so for passing around as a GET or POST variable in a web application, the format function (which turns it into a string representation of the oid number) is handy.
Now that you're passing oids around, Durus gives you an easy way to retrieve the object it identifies:
[in connection.py]
def get(self, oid):
"""(oid:str|int|long) -> Persistent | None
Return object for `oid`.
The object may be a ghost.
"""
Note that if you do pass around the formatted id, you'll need to turn it into a Python int (or long) before sending it to get; if you pass the string '123' get will assume it's a valid (binary) oid and not autoconvert it.
Now that the code diving is out of the way, I'm enjoying Durus a lot. Next post I'll give a short Spyce demo using Durus for persistence.
I had the question,
What do you mean by "compiling" tag libraries?
I mean, that Spyce compiles that chatbox.spy into in a more-or-less 1.3-legal python module. (The classcode/handlers/exports features aren't in 1.3, but you get the idea.) The compiled result looks like this (output courtesy of the Spyce -c option):
class boxlet(spyceTagPlus):
name='boxlet'
buffer=False
exports=1
classcode=((7,4),(10,83),"def addLine(self):\n # (use get() in case server restarted)\n request._api.getServerGlobals().get('chatlines', []).append(request['newline'])",'test-chatbox.spy')
handlers=[('e73068ffe2d1ead1793b6790ec48f92a04fd18641','self.addLine')]
def syntax(self):
self.syntaxSingleOnly()
def begin(self,width='300',lines='5'):
pool=self._api._startModule('pool',None,None)
taglib=self._api.getModules()['taglib']
taglib.load('form','form.py','f')
self._out.writeStatic('\n\n')
pool.init()
self._out.writeStatic('\n<div width="')
self._out.writeExpr(width)
self._out.writeStatic('">\n')
i=-int(lines)
line=None# for first export
for line in pool.setdefault('chatlines',[])[i:]:
self._out.writeStatic(' <div>')
self._out.writeExpr(line)
self._out.writeStatic('</div>\n')
self._out.writeStatic(' <div>\n <f:text name=newline />')
taglib.tagPush('f','text',locals(),{'name':'newline'},False)
try:
taglib.tagBegin()
taglib.tagBody()
taglib.tagEnd()
finally:taglib.tagPop()
self._out.writeStatic('\n <f:submit handler=\'self.addLine\' value="Send" />')
taglib.tagPush('f','submit',locals(),{'_handlerid':'e73068ffe2d1ead1793b6790ec48f92a04fd18641','value':'Send'},False)
try:
taglib.tagBegin()
taglib.tagBody()
taglib.tagEnd()
finally:taglib.tagPop()
self._out.writeStatic('\n <f:submit value="Refresh" />')
taglib.tagPush('f','submit',locals(),{'_handlerid':'e73068ffe2d1ead1793b6790ec48f92a04fd18642','value':'Refresh'},False)
try:
taglib.tagBegin()
taglib.tagBody()
taglib.tagEnd()
finally:taglib.tagPop()
self._out.writeStatic('\n </div>\n</div>\n\n')
self.line=line
def export(self):
return{'last':self.line}
class spyceTagcollection(spyceTagLibrary):
tags=[boxlet]
You can see why not many of these got written. :)
(Updated with a little more explanation as to what's going on.)
The Spyce Active Tag compiler is coming along nicely. Spyce has had active tags since 1.3.0 (current active release is 1.3.13; it's very, very stable by now), but writing a tag library shares a lot of the problems that JSP 1.x tag libraries had; it takes a lot of code to get something done.
Now, I've updated the Spyce compiler to be able to compile tag libaries, and tied it in to the active handler feature as a bonus. Meaning, tags can wrapping their control logic together with the view so all the user has to write is a single tag, like "<chat:boxlet />" below. (You could put the controller logic into another module and write handler="foo.addLine" instead of self, but I'm keeping it simple here.)
Here's a simple example that defines and uses a chatbox component. Chat state is stored in the server globals area for simplicity. This code is running (for the next few days at least) over here. Update: original demo is down, but a slightly more sophisticated version (showing two chatboxes on the same page) is up at the spyce 2.0 prerelease site.
(I hope to get a beta of the next Spyce release (1.4? 2.0?) out later this week.)
(chatbox.spy)
[[.tagcollection ]]
[[.begin name=boxlet singleton=True ]]
[[.attr name=width default=300 ]]
[[.attr name=lines default=5 ]]
[[\ # this code runs when user hits Send
def addLine(self):
# (use get() in case server restarted)
request._api.getServerGlobals().get('chatlines', []).append(request['newline'])
]]
[[.import names="pool"]] [[-- creates pool alias for request._api.getServerGlobals() --]]
<div width="[[= width ]]">
[[ i = -int(lines)
line = None # for first export
]]
[[ for line in pool.setdefault('chatlines', [])[i:]:{ ]]
<div>[[= line ]]</div>
[[ } ]]
<div>
<f:text name=newline />
<f:submit handler='self.addLine' value="Send" />
<f:submit value="Refresh" />
</div>
</div>
[[.export var=line as=last ]] [[-- sends this variable to the calling scope --]]
[[.end]]
(demo.spy)
[[.taglib as='chat' from='chatbox.spy']]
<html>
<head>
<title>Active tag handler test</title>
</head>
<body>
<f:form>
<chat:boxlet />
The last line in chat is: [[= last ]]
</f:form>
</body>
</html>
(whitenoise added to keep blogger from screwing up my formatting...)
I'm looking for a free HTML editor suitable for end-users that are less comfortable entering html in a textarea. The most advanced open-source projects that fit this description seem to be xinha, a fork of the discontinued HTMLArea, and FCKEditor. Both suffer from problems with corrupting the browser history; bug reports are here and here (and here too). I note though that blogger's proprietary editor doesn't have this problem.
Breaking my back button is a serious usability problem. I note that both of these projects have huge amounts of features, far more than Blogger does. This would actually be a drawback to me if I were to deploy one, since I'd have to figure how to turn all the bloat off. Perhaps a project that focused less on features and more on usability could succeed better here.
(Please, if there's one out there that I missed, point me to it.)
At my place of employment we have a table that looks like this:
CREATE TABLE permissions (
id character(40) NOT NULL,
userid character(40) REFERENCES USERS(id),
objectid character(40) NOT NULL,
permission character varying(15) NOT NULL
);
Note that objectid isn't a foreign key to anything. (Before I joined there was no foreign key declared for userid, either.) That is because in this schema, everything gets a unique char(40) ID, and for flexibility the designer wanted to use the same permissions storage for all tables in the database.
(The fix for this, BTW, would involve creating an "objects" table that simply held all the IDs in the system and have that referenced by both the objects tables and any table like this one that wants to be able to reference "any object." I haven't done that yet, but I'll be moving that up on my priority list now.)
About a month ago, after suitable testing, I ran an upgrade script against our live database that went something like this:
CREATE TABLE alerttypes_old AS SELECT * FROM alerttypes; DROP TABLE alerrtypes; -- create new table -- insert into new table massaged data from _old
As it happens, alerttypes is one of the tables that we are interested in permissions for. I forgot to delete the appropriate entries corresponding to the old rows, but worse, I forgot to create new ones for the new alerttypes.
What makes this more surprising is that the developer who reviewed the script missed this too. But that is what happens when you don't have proper integrity constraints: sooner or later, you're going to be restoring from backup. Even if you're a smart guy. Even if you test first (on several machines). Even if you have code reviews.
Incidently, I have seen people leave out FK constraints (or drop the ones I added -- grr!) to accomodate series of statements that temporarily violate the constraint, but eventually (in theory) leave things in a correct state. The correct course here is to put your related statements in a transaction (a good idea anyway), and tell your database to check constraints when the transaction ends, not before. For postgresql, that looks like this:
acs=# begin work; BEGIN acs=# SET CONSTRAINTS ALL DEFERRED; SET CONSTRAINTS -- ...
import os
def foo(s):
f = lambda a: a + "; print '%s'" % os.getcwd()
exec(f(s))
foo("print 'asdf'")
What error does this give? Why?
This one is pretty tough. I'll put a simpler illustration that gives the same error in the comments as a hint.
(Updated to fix inadvertent SyntaxError in the exec'd string, pointed out by Ian Bicking. Python bails with the error I intended before reaching that point, though.)
My current employer keeps a lot of application metadata as xml in the database, and I needed to make a simple update to some of that. Writing a client program to do this would be immense overkill, as well as not playing nicely with my database auto-upgrade script. The update could be easily handled with a regular expression substitution, but although postgresql has a decent regular expression implementation built-in, it has no function to do replacement with them. [Update: regex_replace was added in version 8.1, six months after this post.)
An underused feature of PostgreSQL is its ability to define functions in just about any language you like. Java, Perl, Python, TCL, among others, as well as PL/PGSQL. Most developers, unfortunately, are a bit leery of defining custom functions in the database. Whether this is because of the learning curve (which turns out to be quite shallow), or because they are too used to inferior databases that don't allow such things, I couldn't say.
Anyway, creating a function that did what I needed was trivial.
-- pattern, original, replacement, flags
CREATE OR REPLACE FUNCTION regsub (VARCHAR, VARCHAR, VARCHAR, VARCHAR)
RETURNS VARCHAR AS '
import re
flags = 0
for flagchar in args[3]:
flags |= getattr(re, flagchar)
p = re.compile(args[0], flags)
return re.sub(p, args[2], args[1])
' LANGUAGE 'plpythonu'
WITH (ISCACHABLE);
-- pattern, original, replacement
CREATE OR REPLACE FUNCTION regsub (VARCHAR, VARCHAR, VARCHAR) RETURNS VARCHAR AS '
select regsub($1, $2, $3, '''');
' LANGUAGE 'sql'
WITH (ISCACHABLE);
Some things to note:
>>> import inspect
>>> help(inspect.getargspec)
Help on function getargspec in module inspect:
getargspec(func)
Get the names and default values of a function's arguments.
A tuple of four things is returned: (args, varargs, varkw, defaults).
'args' is a list of the argument names (it may contain nested lists).
'varargs' and 'varkw' are the names of the * and ** arguments or None.
'defaults' is an n-tuple of the default values of the last n arguments.
What conditions would you guess might cause getargspec's args list to contain nested lists?
Now, this doesn't mean I cloned Tiles in Spyce. I took the usual Spyce approach of adding functionality found in the JSP world without adding all the complexity. (Actually, in this case it's actually more influenced by the OpenACS master and slave tags, but how many people would that mean anything to? :)
Specifically I added a way to create consistent site layout templates in the reverse-include style familiar to users of all modern web frameworks. (No, ASP.NET isn't modern by this definition. But ASP.NET 2.0 will be. They call it "Master pages," and googling for that is an excellent way to find a lot of people talking about how this is the best thing since sliced bread. This entry is long enough without me reiterating the reasons why this is a Good Thing.)
Instead of including header and footer in your content page, your content page declares that it belongs to a parent page, which defines all your common markup and a placeholder for the child. Simple example:
child
<spy:parent title="child title" /> Child content
parent
<html>
<head>
<title>[[= child['title'] ]]</title>
</head>
<body>[[= child['_body'] ]]</body>
</html>
which results in the final html of
<html>
<head>
<title>child title</title>
</head>
<body>Child content</body>
</html>
This is both recursive and dynamic.
Recursive, in that one parent template can itself be the child of another. This is useful wherever you have shared content or navigation on a group of pages; you can make them children of parent1 which is a child of your master parent, so site-wide changes still only need to be made in one place, and changes affecting only this group are also only made in one place.
Dynamic, in that all the arguments to spy:parent are evaluated at runtime, even the src argument. Here's an example that doesn't do much except demonstrate this:
[[ import os cwd = os.getcwd() s = '/parenttable.spy' ]] <spy:parent src="=s" foo="=cwd" />This results in the template located at /parenttable.spy being used instead of the default, and an argument named foo being passed with the result of the getcwd evaluation. (Changing which parent template to use at runtime is primarily useful for localizing different languages; passing other parameters that are evaluated at runtime is something you will use frequently even if you only care about English.)
In my opinion it's a phenomenal example of how robust Rimon's Spyce framework is that the diff for this changeset is only about 70 lines. Spyce is one of those rare projects that's a real pleasure to work on because the original author found just the right balance between thinking for future extensibility and overengineering. Tough line to walk, and Spyce does it well.
import os, threading
def foo(s):
print s
for dir in os.listdir(os.getcwd()):
threading.Thread(target=lambda: foo(dir)).start()
What bug may cause this to print something other than the contents of the current directory?
This post seems to indicate that this was fixed for 8.0, although it may have gotten into one of the later 7.x releases (CB is still running the ancient-by-postgresql-standards 7.4.2 release, just over a year old) as well. Changelogs don't mention it specifically though.
I've rearranged disk usage so this won't happen again, but looking at the list of bugfixes through 7.4.7 I guess I should schedule some downtime to upgrade. Now that 8.0 has its first point release I should probably just bite the bullet and dump/reload to that. Unfortunately, Carnage Blender runs two multi-gig databases now, each with several circular dependencies that make necessary manual fiddling with the dumps before restoring.
On the other hand, I did create an rss feed for the python job board several months ago. I guess it didn't occur to me that anyone would want to know about non-python jobs. :)
If you're into dynamic languages, maybe you're like me and you think of mixins when you hear the phrase "partial classes." Wouldn't be the first time Microsoft gave old technology a new name and called it innovation, right?
Unfortunately for C# developers, partial classes have nothing to do with refactoring common behavior into a single reusable code fragment. All it is is a way to break a class up into multiple files for the benefit of code generation tools like Studio 2005's GUI designer. This is something that could easily be done with annotations/decorators/attributes (Java/Python/C# terms for pretty-close-to-the-same-thing). In fact, this is the approach the NetBeans Java IDE has been taking for years with their Swing designer (called "guarded [blocks|areas|code|chunks]"); it works fine.
My friend says that he could see this being useful to other, third party code generators, but I have a hard time seeing that carrying enough weight with Microsoft to actually spawn a new language feature, even if it were the only way to accomplish IDE support for code generation, which it isn't.
Some of the more rabid .NET fanboys have suggested that this will be cool because it lets you physically split a class among different developers. This is a lousy idea on several levels. First, if your classes are that large you probably need to rethink your design. Second, if you really do have an ironclad reason for this monster class, remember that one of the main goals of OO design is that someone who only wants to consume your class only needs to know the public API. People advocating splitting classes up into multiple human-edited files are forgetting that the other side of that coin is, the implementors of this class do need to know everything about it, or you'll be in a situation far worse than ordinary conflict merging.
Maybe I'm missing something. It really does baffle me that the C# team would go to the trouble (and it is trouble) to add a feature with only one real use case, and a flimsy one at that. Most of C# design I can see good reasons behind, if you take as a starting point "Like Java, only with the benefit of several years of hindsight. Oh yeah, and Sun doesn't own it." Some of the decisions are arguable but this one makes me scratch my head.
Fortunately, Oliver just recently put up a a pdf of his openlazlo slides. Unfortunately there's a lot of blank slides, and rather more diagrams than code. (I think they were limited to 30 minutes, though. Ouch!) What I got out of it was,
