Skip to main content

A task-scheduling module

Part of the next release of Spyce will be the new scheduler module. This actually has no Spyce dependencies, so it may be useful in any (web or non-web) application that needs to run tasks at given times or intervals. Some things I have used it for:
  • Scan the log for errors and email me a summary
  • vacuum (pre-autovacuum daemon days...)
  • Purge stuff from the global cache
  • Send email to users whose accounts are about to be suspended
Why a python module instead of cron? The primary benefit in my mind is that scheduler runs as a thread of your application (or app server, in Spyce's case), so you can access your database connection pool, cache, or other global state without jumping through nasty hoops. This is a bigger deal the more complex your application is... Putting logic here also makes deploying to testing or release servers (or to a client) a matter of running "svn up" rather than having to mess with crontab.

You can browse the svn source here: http://svn-hosting.com/svn/spyce/trunk/spyce/scheduler.py

>>> import scheduler
>>> help(scheduler)
Help on module scheduler:

DESCRIPTION
    A module for scheduling arbitrary callables to run at given times
    or intervals, modeled on the naviserver API.  Scheduler runs in
    its own thread; callables run in this same thread, so if you have
    an unusually long callable to run you may wish to give it its own
    thread, for instance,
    
    schedule(lambda: threading.Thread(target=longcallable).start())
    
    Public functions are threadsafe.

CLASSES
    Task
    
    class Task
     |  Instantiated by the schedule methods.
     |  
     |  Instance variables:
     |    nextrun: epoch seconds at which to run next
     |    interval: seconds before repeating
     |    callable: function to invoke
     |    last: if True, will be unscheduled after nextrun
     |  
     |  (Note that by manually setting last on a Task instance, you
     |  can cause it to run an arbitrary number of times.)
     |  
     |  Methods defined here:
     |  
     |  __init__(self, firstrun, interval, callable, once)

FUNCTIONS
    pause()
        Temporarily suspend running scheduled tasks
    
    schedule(interval, callable, once=False)
        Schedules callable to be run every interval seconds.
        Returns the scheduled Task object.
    
    schedule_daily(hours, minutes, callable, once=False)
        Schedules callable to be run at hours:minutes every day.
        (Hours is a 24-hour format.)
        Returns the scheduled Task object.
    
    unpause()
        Resume running scheduled tasks.  If a task came due while
        it was paused, it will run immediately after unpausing.
    
    unschedule(task)
        Removes the given task from the scheduling queue.

Comments

Anonymous said…
I recommend using VisualCron instead of the task scheduler.

regards,

Ryan
Tim Lesher said…
<sarcasm>
Great idea, Ryan. That will work great when running a Spyce app on my colocated FreeBSD box.
</sarcasm>

Popular posts from this blog

The Missing Piece in AI Coding: Automated Context Discovery

I recently switched tasks from writing the ColBERT Live! library and related benchmarking tools to authoring BM25 search for Cassandra . I was able to implement the former almost entirely with "coding in English" via Aider . That is: I gave the LLM tasks, in English, and it generated diffs for me that Aider applied to my source files. This made me easily 5x more productive vs writing code by hand, even with AI autocomplete like Copilot. It felt amazing! (Take a minute to check out this short thread on a real-life session with Aider , if you've never tried it.) Coming back to Cassandra, by contrast, felt like swimming through molasses. Doing everything by hand is tedious when you know that an LLM could do it faster if you could just structure the problem correctly for it. It felt like writing assembly without a compiler -- a useful skill in narrow situations, but mostly not a good use of human intelligence today. The key difference in these two sce...

Why PHP sucks

(July 8 2005) Apparently I got linked by some PHP sites, and while there were a few well-reasoned comments here I mostly just got people who only knew PHP reacting like I told them their firstborn was ugly. These people tended to give variants on one or more themes: All environments have warts, so PHP is no worse than anything else in this respect I can work around PHP's problems, ergo they are not really problems You aren't experienced enough in PHP to judge it yet As to the first, it is true that PHP is not alone in having warts. However, the lack of qualitative difference does not mean that the quantitative difference is insignificant. Similarly, problems can be worked around, but languages/environments designed by people with more foresight and, to put it bluntly, clue, simply don't make the kind of really boneheaded architecture mistakes that you can't help but run into on a daily baisis in PHP. Finally, as I noted in my original introduction, with PHP, ...

A week of Windows Subsystem for Linux

I first experimented with WSL2 as a daily development environment two years ago. Things were still pretty rough around the edges, especially with JetBrains' IDEs, and I ended up buying a dedicated Linux workstation so I wouldn't have to deal with the pain.  Unfortunately, the Linux box developed a heat management problem, and simultaneously I found myself needing a beefier GPU than it had for working on multi-vector encoding , so I decided to give WSL2 another try. Here's some of the highlights and lowlights. TLDR, it's working well enough that I'm probably going to continue using it as my primary development machine going forward. The Good NVIDIA CUDA drivers just work. I was blown away that I ran conda install cuda -c nvidia and it worked the first try. No farting around with Linux kernel header versions or arcane errors from nvidia-smi. It just worked, including with PyTorch. JetBrains products work a lot better now in remote development mod...