Skip to main content

A task-scheduling module

Part of the next release of Spyce will be the new scheduler module. This actually has no Spyce dependencies, so it may be useful in any (web or non-web) application that needs to run tasks at given times or intervals. Some things I have used it for:
  • Scan the log for errors and email me a summary
  • vacuum (pre-autovacuum daemon days...)
  • Purge stuff from the global cache
  • Send email to users whose accounts are about to be suspended
Why a python module instead of cron? The primary benefit in my mind is that scheduler runs as a thread of your application (or app server, in Spyce's case), so you can access your database connection pool, cache, or other global state without jumping through nasty hoops. This is a bigger deal the more complex your application is... Putting logic here also makes deploying to testing or release servers (or to a client) a matter of running "svn up" rather than having to mess with crontab.

You can browse the svn source here: http://svn-hosting.com/svn/spyce/trunk/spyce/scheduler.py

>>> import scheduler
>>> help(scheduler)
Help on module scheduler:

DESCRIPTION
    A module for scheduling arbitrary callables to run at given times
    or intervals, modeled on the naviserver API.  Scheduler runs in
    its own thread; callables run in this same thread, so if you have
    an unusually long callable to run you may wish to give it its own
    thread, for instance,
    
    schedule(lambda: threading.Thread(target=longcallable).start())
    
    Public functions are threadsafe.

CLASSES
    Task
    
    class Task
     |  Instantiated by the schedule methods.
     |  
     |  Instance variables:
     |    nextrun: epoch seconds at which to run next
     |    interval: seconds before repeating
     |    callable: function to invoke
     |    last: if True, will be unscheduled after nextrun
     |  
     |  (Note that by manually setting last on a Task instance, you
     |  can cause it to run an arbitrary number of times.)
     |  
     |  Methods defined here:
     |  
     |  __init__(self, firstrun, interval, callable, once)

FUNCTIONS
    pause()
        Temporarily suspend running scheduled tasks
    
    schedule(interval, callable, once=False)
        Schedules callable to be run every interval seconds.
        Returns the scheduled Task object.
    
    schedule_daily(hours, minutes, callable, once=False)
        Schedules callable to be run at hours:minutes every day.
        (Hours is a 24-hour format.)
        Returns the scheduled Task object.
    
    unpause()
        Resume running scheduled tasks.  If a task came due while
        it was paused, it will run immediately after unpausing.
    
    unschedule(task)
        Removes the given task from the scheduling queue.

Comments

Anonymous said…
I recommend using VisualCron instead of the task scheduler.

regards,

Ryan
Tim Lesher said…
<sarcasm>
Great idea, Ryan. That will work great when running a Spyce app on my colocated FreeBSD box.
</sarcasm>

Popular posts from this blog

Why schema definition belongs in the database

Earlier, I wrote about how ORM developers shouldn't try to re-invent SQL . It doesn't need to be done, and you're not likely to end up with an actual improvement. SQL may be designed by committee, but it's also been refined from thousands if not millions of man-years of database experience. The same applies to DDL. (Data Definition Langage -- the part of the SQL standard that deals with CREATE and ALTER.) Unfortunately, a number of Python ORMs are trying to replace DDL with a homegrown Python API. This is a Bad Thing. There are at least four reasons why: Standards compliance Completeness Maintainability Beauty Standards compliance SQL DDL is a standard. That means if you want something more sophisticated than Emacs, you can choose any of half a dozen modeling tools like ERwin or ER/Studio to generate and edit your DDL. The Python data definition APIs, by contrast, aren't even compatibile with other Python tools. You can't take a table definition

Python at Mozy.com

At my day job, I write code for a company called Berkeley Data Systems. (They found me through this blog, actually. It's been a good place to work.) Our first product is free online backup at mozy.com . Our second beta release was yesterday; the obvious problems have been fixed, so I feel reasonably good about blogging about it. Our back end, which is the most algorithmically complex part -- as opposed to fighting-Microsoft-APIs complex, as we have to in our desktop client -- is 90% in python with one C extension for speed. We (well, they, since I wasn't at the company at that point) initially chose Python for speed of development, and it's definitely fulfilled that expectation. (It's also lived up to its reputation for readability, in that the Python code has had 3 different developers -- in serial -- with very quick ramp-ups in each case. Python's succinctness and and one-obvious-way-to-do-it philosophy played a big part in this.) If you try it out, pleas

A review of 6 Python IDEs

(March 2006: you may also be interested the updated review I did for PyCon -- http://spyced.blogspot.com/2006/02/pycon-python-ide-review.html .) For September's meeting, the Utah Python User Group hosted an IDE shootout. 5 presenters reviewed 6 IDEs: PyDev 0.9.8.1 Eric3 3.7.1 Boa Constructor 0.4.4 BlackAdder 1.1 Komodo 3.1 Wing IDE 2.0.3 (The windows version was tested for all but Eric3, which was tested on Linux. Eric3 is based on Qt, which basically means you can't run it on Windows unless you've shelled out $$$ for a commerical Qt license, since there is no GPL version of Qt for Windows. Yes, there's Qt Free , but that's not exactly production-ready software.) Perhaps the most notable IDEs not included are SPE and DrPython. Alas, nobody had time to review these, but if you're looking for a free IDE perhaps you should include these in your search, because PyDev was the only one of the 3 free ones that we'd consider using. And if you aren