I have been lucky to be able to fill our recent open positions with people who know Python as well as Java so now we are up to half the (6 person) company in that category and preferring Python, and 2 of the others have played with Python and liked it at least well enough to not object. So the boss has conceded that it makes sense to go the Python route for our next project.
We're going to be doing a web, "next gen" version of our existing client-server project, which is mostly simple CRUD but does have 1000+ tables in its current incarnation. So we really need something that can autogenerate 90+% of the CRUD or we will go insane.
The trouble is, I still don't really like any of the Python web options 100%. (I like the web options in other languages less, but I'm a perfectionist.)
Django is well documented, its admin app is something everyone else envies, and newforms looks decent, but the ORM blows and I'm not fond of the template engine either. (Pre-emptive pedantry: yes, I know I can "import sqlalchemy." Please stop saying that like it means something; I'm not interested in defining models twice -- once for real work with SA, and once for interop with the rest of django.) Apparently django-sqlalchemy got far enough in PyCon sprints that it's kinda usable so working on that would be an option. Of course even then there is no guarantee the django core would accept it into mainline, and maintaining it as a "vendor branch" would proably suck. If django used a dscm like Mercurial I might be willing to do that, but svn is just too painful so that is a real risk.
I don't see a way to generate a page containing just a CRUD interface for table X with the django admin app. The admin app really is a monolithic application, not something you can easily re-use pieces of.
Regexps suck for url mapping.
Pylons is not well documented and after keeping an eye on this for something like 18 months I don't think this is a problem that will be solved, for whatever reasons. On the other hand, SA + mako is a very sane default, and both of those are well documented so it's really only core Pylons that suffers from doc crapitude, and core Pylons is fairly small. IRC responsiveness mitigates this further.
Pylons still doesn't have a good CRUD (or even high-level manual form generation) solution, which has bugged me for even longer than the docs. I can't fathom how people can tolerate writing this kind of boilerplate in 2008. Formalchemy gets about 30% of the way there. DBMechanic requires TG2 atm, although apparently hacking it to run on Pylons may not be too much effort; I would guess around 20% of the effort to get the django-sa project really usable.
TG2 is of course very bleeding edge and although I like genshi's syntax in theory, in practice XML templates irritate the hell out of me. (Very verbose, xinclude sucks compared to "inheritance," and incorporating rich dynamic content -- i.e., user-generated, like forum posts, that needs to include html tags -- is a PITA. Not to mention that having to write "a > b" when you mean "a > b" bugs me all out of proportion to the actual inconvenience it inflicts on me.) Still, better than the django templates.
I'm skeptical that TG2 is a big enough value add to want to add it (in its unfinished state) as a dependency vs rolling our own on Pylons. But DBMechanic does look like it could be exactly what I want in a CRUD generator.
web.py seems like more of a tech demo than a real product. I don't see any signs of a CRUD or form generator. reddit, probably the largest web.py site at least in terms of page views, moved to Pylons.
Zope 3 is alone in being really production ready without running from svn. Grok does do a good job of smashing zcml and z3c.form looks okay but lives up to the Zope reputation of complexity. (Field managers, widget managers -- are these the same things? -- widget modes, ...) AFAIK relational dbs are still second-class citizens in zope, and with all due respect to zodb it is no postgresql. OTOH there is z3c.sqlalchemy which gives me hope. Finally: you have to manually restart zope (per the Grok tutorial) after changing your .py files? Seriously?
Bottom line, Zope might actually be a decent option if we had a Zope expert on staff but we do not and I am not willing to tackle the learning curve alone.
Nevow: form handling is in flux. The new hotness is "pollenation forms," but that is svn-only and the api "will probably change."
Zope and Nevow both have their own xml-based templates predecessing but similar to genshi. Something like Nevow's Stan is obviously useful for programmatic template generation but it's not yet clear if that's going to be something we need. Probably only if we have to write our own form generator. If so, I suspect ripping a standalone Stan out of Nevow would be straightforward.
(Spyce of course never really got any traction to speak of. It's time for me to let it go quietly into the night and leverage someone else's framework.)
Conclusion: I think porting DBMechanic to Pylons is our best option. DBMechanic seems designed to be more flexible than the django admin app. Django would be my second choice.
Corrections? Thoughts?
Comments
These are two very different things, and you should not try to implement anything on top of or in the django admin. Creating some generic CRUD views is not that difficult, but if you do not like the ORM or the template engine, and have no need for the admin (which I doubt you do), then django really is not a good fit as you have come to realize.
I would recommend just going with bare WSGI or Pylons and SQLAlchemy. It does not sound like you need any of the features that the current 'frameworks' provide.
1. django-sqlalchemy not being included in core is not a problem since it's not a branch or a modification to Django itself. It's actually implemented using mostly the 3rd-party database backend library that is currently used to determine between sqlite, mysql, and postgresql. Using django-sqlalchemy gives you a fourth, sqlalchemy option, and that's where all the hooks into Django take place.
2. Why do you need Django itself to support your dscm of choice? Why not use one of the many excellent bridges that exist between svn and your dscm of choice? Especially since no real winner has been declared in this space, it's tough to see this as a real problem.
3. For the autogeneration stuff, I think you're looking for a combination of "django-admin.py inspectdb" (which generates a models.py file from tables in the database) and generic views. There are generic views for creation, updating, and deleting, as well as a host of reading operations--list views and date-based views. Since templates are always going to be slightly different for each user, that's really going to be the only thing you spend much time on, if you use generic views correctly.
4. The newforms-admin branch is making great steps in transforming the admin into something that is less monolithic. It's fairly stable and I'd highly recommend checking it out.
5. Why don't you like using regexes for url patterns? Trying to match a{n}b{n}? :) Honestly though, why don't you like it?
By the way, thought your talk on PyTriton was fascinating. Kudos on all that hard work!
A while ago when I was looking into crafting well-designed URLs I was trying to come up with a simple notation for defining a URL schema. Then I realised it already existed in the form of Regular Expressions.
1) What is the big win for you of SQLAlchemy over Django ORM?Aggregation and other more complex queries? Or is there something more deepseated you don't like about the Django ORM here?
2) Although I agree that saying 'just import sqlalchemy' doesn't cut it, the same is not true for alternate templating engines. It's dead easy to use Django with Genshi/Jinja/others and unlike with the ORM you don't really lose any cohesion by doing so.
A few notes about Grok: Grok ships with integration of zope.formlib (not z3c.form), and less intimidating than z3c.form. z3c.form is indeed classic Zope 3 "power over simplicity", though I think we'll be able to tame it someday with Grok. Our formlib integration isn't as well documented as we'd like, but documentation can be found here.
On SQLAlchemy integration: I've recently done a project with Grok and SQLAlchemy and I'm quite pleased with how easy the two integrate (ZODB and PostgreSQL in the same application, sharing the same transaction!). SQLAlchemy plays very well with Grok. Hopefully we'll be able to start sharing a good default story on this soon. It's really of importance to the Grok project to have this work well out of the box for people.
Phillip von Weitershausen's book "Web component development with Zope 3" is also great.
http://mdp.cti.depaul.edu/
Doug: well, the django admin is all about automating a certain use case of CRUD, right? So I think a certain amount of confusion is natural for a first-timer. But yes, getting this kind of confusion cleared up by people who _are_ experts where I am not is the main reason I posted this. :)
Eric: thanks for the clarification about django-sa, and good point about the dscm/svn bridges, although I think only git's is really first-class, and I don't care for some other things about git. I'm not just picky about web frameworks! :)
It does look like generic views + formpreview does a substantial amount of what I want. Too bad inspectdb is static rather than dynamic.
Sean: SA can handle all kinds of mapping and queries that django cannot. I do have production experience with Rails + ActiveRecord and I ran into "dammit, AR can't do this either?" frustrations literally daily. I honestly have no idea how people can be satisfied with such limited solutions. I imagine that your brain eventually starts to think in terms of what your ORM can do instead of what the database can do, but why cripple your thinking when a best-in-class ORM does exist? The Django ORM and AR are roughly at the same level with AR probably having a slight edge. See my "why SA impresses me" post for one example (although there are easier ways to accomplish this in modern SA). Additionally, the unit of work pattern is much better than manually saving every updated object.
Yes, the templates thing bugs me a whole lot less than the ORM one since as you say there really is no obstacle to just using something else. But as Doug says why bother with django at that point? :)
Martijn: thanks for the good news about SA + grok.
All: I've written a lot of regular expressions and while I like to think I'm pretty good at doing so, they've always stayed slightly "write-only" for me. The thought of poring over dozens of re's in a big app trying to figure out exactly why my url isn't being mapped where I expected does not appeal to me. I find Routes more readable.
I've just started messing with Django and right off the bat, the single key pk thing bit me.
Take a look at their routing system as well. (I couldn't post an example of an url_map here because angle brackets aren't allowed.)
Oh, and of course, as the templating language I use jinja which was also written by the authors of werkzeug. Jinja is an enhanced version of django-templates.
I note that you say that DBMechanic is TG2 specific -- I don't think that's the case really. Chris has reported success running it with Pylons and Grok, and the goal is to support all the major frameworks.
I think you personally at this moment may be best off with Pylons and not TG2, but I also think it's important to mention that TG2 provides things like DBSprockets integration, automatic form handling, and the like out of the box.
We can do those things because we have a known core of standard components, while Pylons gives you the freedom to choose your own.
We've got more plans along those lines, from user registration, to other larger site-components. TG2 will provide a lot of things that Pylons just can't.
With that said, we fully intend to support two options in one place. We've decided to anoint Mako as our "second" templating language -- because sometimes you just need the performance.
But, Pylons will offer something that TG2 can't -- the abilty to swap out components. I mean you could swap out the ORM in TG2, but you'll loose all the integration featurs that we've added on top of Pylons.
The way I work is using Django ORM for the obvious stuff: queries to retrieve sets of objects; and then stored procs and/or hand-coded SQL for anything more complex. I've never worked with an ORM that made sense to me for anything beyond basic querying, but then I've never tried SQLAlchemy. You've convinced me I should give it a good look, especially as the django-sa bridge seems to be coming along nicely, so using it with my existing Django projects may soon be fairly straightforward.
switch to rails or merb
web2py is the only framework that participated to the www.flourishconf.com rumble (develop a web app in 24 hours) and won because all the other frameworks chickened out. This https://mdp.cti.depaul.edu/SurveyRumble was developed in web2py in 24hrs (actually 12hrs because I slept and ate too meanwhile).
SQLAlchemy is a great ORM but was not appropriate for web2py since its components required a tight integration that could not have been achieved without modifying SQLAlchemy. Anyway, it is not like it is difficult to write an ORM or learning to use one so it was not much time I wasted. btw, the web2py ORM does migrations, supports Oracle (including limitby), does left joins, and its is much much much easier to use than SQLAlchemy.
web2py is based on wsgi and until version 1.18 it was based on the httpserver from Paste. Turned out that the cherrypy 3.0 wsgiserver was faster so I switched to that. All my users supported this switch and noticed an improvement.
It seems pointless to me to criticize web2py because "you do not like it" and because "it did not use your favourite project". Tell me instead which functionality (you think) is missing in web2py or which benchmark you performed and have found web2py slower than another framework, if you can.
I am happy to report my ModelWebHelpers idea is being included in the next release. That is, you can use a special variant of the input tag webhelpers that will automatically fill in values from the model record, if it's present. Check out http://paste.lisp.org/display/58562 for a sample of something close to how it's going to work. (We're using the 'special object' approach variant at the bottom.)
The default dispatcher is Object/Method based, and I quite enjoy it. It has a Routes based dispatcher which is quite popular as well.
I need to get these uploaded to http://cherrypy.org, but here are a bunch of case studies for CherryPy:
http://groups.google.com/group/cherrypy-users/browse_thread/thread/1774049cc582a595
Its an excellent choice for people whose needs don't fit the cookie-cutter megaframeworks.
(Full disclosure: I also use and quite enjoy Django. "Right tool for the job" and all that. ;) )
Using Django as an example, nothing stops you from building Models from a DB specification; it'll take one person who understands a bit more about Python than most people, one person who understands how metaclasses work, but if you autogenerate classes from the DB, it'll work.
(The default ORM will probably still not be sufficient for your needs; I'm just using an example.)
Thanks, now I know that staying away from web2py was a good decision. :)
(Sure, it's easy to write a simplistic one that doesn't let you take advantage of what relational databases are good at. But I'm not interested in using those. Again, see Why SQLAlchemy Impresses Me for one example.)
As a matter of fact, the start of the port for DBMechanic to pylons has already been done. We got this up and running at Pycon last month. Here is a how-to: http://code.google.com/p/dbsprockets/wiki/DBMechanic (look at the bottom for pylons)
No, it's not polished, but it does work.
TG2 offers automatic crud based on dbsprockets. The general template will generate a listing page, and edit page, and a create page. This is a pretty useful tool for stubbing out a website, and something that you don't get with pylons.
TG2 does support Mako, by the way, and it is not hard to integrate. Personally, I think you get a lot of bang for your buck with TG2, and I wouldn't shy away from it based on a notion of flexibility, or value added.
About Django-admin. A few weeks ago we got a proposal for GSoC which basically took the Django Admin syntax and laid it on top of dbsprockets. After looking at it, I realized I could probably implement it in about a weekend. However, I thought it violated the DRY paradigm, so I refrained. Basically, I am saying we can make a pretty simple django-admin type interaface for TG2, but it doesn't *feel* right at this time. Maybe someone will change my mind.
Lastly, we are working on the docs. I have been working with one of our GSoC perspectives on fleshing out the ToscaWidgets documentation, adding ajax widgets, and creating a bunch of how-tos to help get people up and running.
http://docs.turbogears.org/2.0/RoughDocs/ToscaWidgets
Others have been working very hard on the TG2 docs as well. We realize that docs are a problem, most people in the wsgi world don't have a large publishing company behind them, we are mostly engineering-types who just want things to work, and provide our users with well-tested releases (as opposed to living with hot_action). Our wiki pages are all freely editable, and I would hope that developers using TG would contribute more in the future.
In short, dbsprockets supports a lot more frameworks than you might think, (even django is on my list to get working) and TG2 offers many more integrated tools out of the box than you might think. Feel free to email me and ask me about either.
cheers.
-chris
we2py's orm not bad at all.
why are you blame it? only beacause it isn't sqlalchemy? :)
This is really not a deal unless you plan on writing some hideous urls. Especially if you plan on using a simple rest style, your urls.py classes will look succinct and easy to understand.
I was a little worried about the regex thing too, since I don't care for them, but it has not been an issue at all, and it's damn handy to have them there when you need them. I wouldn't trade them at all now that I have used them.
In a previous post, you wrote:
++ Poor documentation of core Pylons ... I had to use the source several times. ... The first tutorial overcomplicated things, showing how to configure things to handle semi-obscure requirements, without explaining those requirements or simpler alternatives.
Is this still the core of your objection to Pylons docs? If you have additional observations to make, I would be very interested to them (the email address below is valid for a week).
Cheers,
Graham Higgins
gjh-dated-1208191473.090b82@bel-epa.com
and maybe a link of zodb vs postgres benchmark ... http://www.upfrontsystems.co.za/Members/roche/where-im-calling-from/zodb-benchmarks-revisited
enjoy ...
You can define a 'traverse' method on content objects (or if you want, against existing objects defined in completely other packages thanks to the power of the component architecture), which takes a name (which is a URL step), and returns either an object to traverse into, or None (traverser doesn't know what to do, fall back on whatever is there for this object; it could for instance be a container or have views).
It also turns out to be quite easy to use things like zope.location.location.locate() to give database-driven objects a 'location' in URL space, meaning you can very easily generate URLs for these objects.
The nice thing about the traversing approach is that (infinitely) nested hierarchies are very easy to create. You can also easily move objects around in model-space without having to remap a lot of URLs, and combine objects from different projects into a single application.
A drawback is that it's a harder to control what URLs get published (though the skin concept helps quite a bit). The nice thing about a routing approach is that you get a limited amount of places to look at to see what URLs the application supports.
My apologies if I said the wrong things about routing though; I'm quite sure there are solutions to all the issues I listed. I am making the case that traversing makes these cases feel very natural.
Content steaming.
Sadly, django doesn't support any form of it.
Pylons has some issues but it's getting there.
The WSGI spec supports streaming, unfortunately a lot of middleware isn't compliant.
I'm not sure if this meets your needs, but your post reminded me of the tgcrud addon for TurboGears (1.0). I think these are docs for it http://docs.turbogears.org/1.0/CRUDTemplate.
TG1 may not be the new hotness, but it is stable and tgcrud does seem to support SQLAlchemy.
Anyway, I have not used tgcrud myself, but since I did not see anyone else mention it, I thought I would.
HTH,
Krys
We're making the same decision. Our plan is to do some prototypes with TG1 right now but build our real system in TG2. A little bleeding-edge, but all the important components look mature, so it's really Pylons + SA + Elixir + Genshi with some glue. And, if the glue ends up being messier than our current impression, we can fix it and contribute back to the project.
The only questionable side is ToscaWidgets which seems poorly-documented. We'll evaluate that a bit later. If so, we'll have to wrap some parts of Dojo ourselves.
Any advise?
Frank
http://spyced.blogspot.com/2008/10/formalchemy-10.html
http://spyced.blogspot.com/2008/10/small-admin-app-for-pylons.html
The formalchemy mailing list is a good place for followup questions.