Thursday, March 29, 2007

One thing I don't hate about Python

Sure, some things about Python bug me. But that's not what this is about. I wanted to react to Jacob Kaplan-Moss's gripes instead of promulgating my own. Specifically, his problem with Python's interfaces, or lack thereof.

I think I can keep this brief: interfaces are a hack that Java uses because Gosling et al thought multiple inheritance was too confusing and/or dangerous. (I believe I've read something recently where Gosling said that this was one decision he might do differently if he were re-designing Java now with the benefit of hindsight, but I can't find the source. Anyone remember seeing that?)

Python has MI. It doesn't need interfaces. I'm a little baffled that someone on the django core team would cite this as a problem with Python.

Jacob's precise objection is,

I shouldn’t need to care care about the difference between something that pretends to be a list and something that really is a list.

That's just it! You don't! But of course what Jacob really means is, "It should be easy to discover what methods a library expects to find on MY object that pretends to be a list." Which seems reasonable. And sure, good documentation is always welcome.

But when you cross the line to an Interface, at least the kind of Interface where Python itself would raise an error if I ignored the recommendation and left a method out (because I knew it wasn't necessary), that's bondage & discipline. That's not Python.


beza1e1 said...

Gosling: There are a bunch of things that I'd do differently. There are a number of things that I'm not entirely happy with and it's not clear what the right answer is. I'm not really happy with the schism between interfaces and classes; in many ways it feels like the right solution wouldn't get in the way.

From an interview.

Marius said...

The Zope 3 project uses interfaces in Python quite successfully. They are introspectable and mostly used for looking up adapters.

Xavier said...

If you want to ensure that you aren't missing any of the methods expected out of a list, simply make your class extend the list class, and overload whatever methods you wish to replace. That works well for me.

Sean Taylor said...

Interfaces are an elegant solution to some types of problems, including some which cannot be solved with multiple inheritance. Adapters can be registered dynamically, whereas multiple inheritance must occur at class declaration (well I'm sure there is a way to do it dynamically, but I can't imagine it looks nicer than the interface/adapters approach). As always, pick the right tool for the job...

Martijn Faassen said...

Zope 3 interfaces (already mentioned) are not bondage and discipline. They are used in a number of ways:

* specification what a class should implement in order to comply with an interface. This is really useful information to have if you have a large system with many plugin points. Of course you can still cheat, it's just information.

* a way to organize API documentation. The docstrings on the interfaces are API documentation.

* the already mentioned adapter lookup. "I have this object, I want a way to render it to HTML. Oh, I do IShowAsHTML(object)". If the object already implements IShowAsHTML, the object is returned. If not, it'll look in a registry to see whether any adapter is registered for it, by the application or an extension to this application. It'll do "that_adapter(obj)" and return an object that has the API specified by IShowAsHTML. Very powerful way to have objects that do only 1 thing well (instead of inheriting a zillion base classes), and very pluggable.

There are some drawbacks, mostly to do with increased indirection in the application (which always can make something harder to understand). It's also more typing. This is bad for smaller applications, but if you have a large framework with many plugin points, it is worthwhile.

Jeff Shell said...

As mentioned, Zope 3 uses Interfaces extensively. They provide specification, extended introspection, adaptation (an honestly useful tool), etc. A small gripe I have with Python's loose concept of interfaces is the differences in 'file-like'. Some classes / tools say "needs a file like object", but only need a read method. Others only need a write method. Others need seek and tell. Sometimes even StringIO might not work. Typically it is just read and write, but sometimes it's more.

Zope 3 makes interfaces even more useful with its registries. You can use the interface spec as a lookup mechanism. Something similar is entry points in setuptools; but with Interfaces you have objects as well as names. Thus the 'buffet' templating plug-ins might be registered and looked up with `buffetspec.ITemplatePlugin`. ITemplatePlugin is an object. That means `buffetspec.interfaces.ITemplatePlugin` could be a valid reference as well. By using objects instead of mere strings as identifiers, you start to get the same rich benefits that you get when you replace string exceptions with Exception classes/objects.

But it's not bondage and discipline, these Zope interfaces. There are tools that can be used to verify an implementation: add two lines to a unit or doc test and you can check to make sure you're meeting the ITemplatePlugin spec. If it changes in the future, you can be warned by running your test suite. But it won't stop your code from compiling and executing.

Many large projects that start to develop APIs and specifications for extending the system can benefit from a formal tool for declaring those specifications and/or testing against them; and it's nice to separate out the specification from implementation, but to still be able to use that spec in code. zope.interface provides a solution that I believe is still Pythonic.

Leo said...

Python doesn't particularly need something interfaces-like to enforce contracts. It might be useful, but it isn't essential to a dynamic language. What Python does very much need is something interfaces-like to do namespace management for the right-hand-side of the method-call dot. Then I can do something like arbitrary_object.Duck:quack() safe in the knowledge that the result will be either a duck-noise or an AttributeError, not a dodgy medic. There's also the incidental but not trivial benefit that typing help(Duck) might produce some useful documentation. These are quite distinct concerns, at least in a language that isn't statically typed by default. The sharing of implementations, rather than contracts, for example through inheritance, is a different matter again.

Bruno said...

Then tell me what methods a file-like object should implement and their arguments.

That's what I think JKM's interface gripe means. Does cgi.FieldStorage require .readline(size)?

Jonathan Ellis said...

bruno, that's the beauty of duck typing -- the answer is, whatever you need! There _is_ no one-size-fits-all! If all you need is read(), that's all you should have to write; forcing you to add seek() -- or even just stubbing it in with NotImplementedError -- is just wasting your time. Other times you _will_ need seek. So a generic Interface that specifies more than you need sucks, but so does proliferating interfaces like FileWithSeek, FileWithSeekAndTell, WriteableFileWithSeekAndTell, etc.

Interfaces are just one of those ideas that seems ok on the surface but is really worse than the problem it's supposed to be solving.

Bruno said...

Jonathan, I don't exactly *need* interfaces. They're just the *only* way I've thought about to get a "TypeError: cgi.FieldStorage() fp must support .read() with 1 arg (it supports 0 args instead)".

Anyway, you've prodded me into smacking some neurons together. I don't think that the interface names are that important -- they could be anonymous and I wouldn't care. I'll try to post some Python code below as an example:

class FieldStorage:
def __init__(self, fp=None, ...):

And @expects() uses introspection to check if the fp= argument if gets supports that interface.

I guess this is where Faassen chimes in and points out I'm reinventing the wheel Zope has already made available; I haven't even looked at zope.interface. Will do.

Bruno said...

Well, zope.interface is a wee bit different than what I had in mind; it does satisfy the cgi.FieldStorage/WSGI use case, but I guess it goes around it the wrong way.

I'd prefer to specify what I, as user of an instance, need; ZopeInterface says what interface provides and let's me check if the instance I got implements that interface.

Some points the Zope guys have dealed with: an "interface" is more than signatures -- it's also invariants (I'll go ahead and call it a contract). And I don't know how to say that.

I guess it's time for me to write up @expects(). Maybe I'll "activate" it only when 'DEBUG' in os.environ (C tradition lives on!).

Bruno said...

I think I can hear the laughter from LtU: "ha, the grasshopper has discovered type inference! such greenish-ness is just... adoooorable!"

Nathen said...

In an abstract data type world interfaces are a useful feature for ensuring proper publishing of methods. In my experience I rarely find enough information on a method by pure introspection. In the end I rely on the developer documentation and if that fails, read the code. Aren't we all programmers ? Isn't one of the main points of Python legibility ? The built-ins in python are a great example of abstraction, however they have been in use for a really long time and have copious documentation and discussion about their characteristics. If one is developing a framework for use by other programmers it might be more useful to spend time on code comments and writing the code legibly then worrying about inserting programmatic conventions into the language.