Thursday, October 18, 2007

Utah Data Recovery

About three years ago (so pre-Mozy and definitely pre-Mac Mozy) my brother had his powerbook hard disk die. As in, not just mostly dead -- it would not power up. It had a lot of stuff on it that he didn't want to lose, but he felt like the usual suspects who charge $1k to $2k for data recovery were a rip off. So he hung onto the disk in case a cheaper option came along.

Then just recently when I saw some people on a local linux group mailing list recommend utah data rescue I suggested to my brother that he give it a try. UTDR starts at "only" $300.

UTDR did indeed recover the data, although they charged $100 extra for this one. Mac fee? Tricky hw problem? I don't know. But it was still a lot cheaper than the other companies I googled for fixing a physically dead drive. (As opposed to a corrupt partition table or something where the hardware itself was okay.) At least, the ones that actually give you a price up front rather than hiding behind "request a quote!"

Tuesday, October 09, 2007

Semi-automatic software installation on HP-UX, with dependencies

I had to install subversion on a couple HP-UX boxes. Fortunately, there's an HP-UX software archive out there with precompiled versions of lots of software. Unfortunately, dependency resolution is like the bad old days of 1997: entirely manual. And there's fifteen or so dependencies for subversion.

So, I wrote a script to parse the dependencies and download the packages automatically. It requires Python -- which you can install from the archive with just the Python package and the Db package -- and BeautifulSoup, which you can google for. Usage is

hpuxinstaller <archive package url> <package name>
[e.g., hpuxinstaller http://hpux.cs.utah.edu/hppd/hpux/Development/Tools/subversion-1.4.4/ subversion]
[wait for packages to download]
gunzip *.gz
[paste in conveniently given swinstall commands]

Here is the script:

#!/usr/local/bin/python

import urlparse, urllib2, sys, os
from subprocess import Popen, PIPE
from BeautifulSoup import BeautifulSoup

required = {}
if not os.path.exists('cache'):
    os.mkdir('cache')

def getcachedpage(url):
    fname = 'cache/' + url.replace('/', '-')
    try:
        page = file(fname).read()
    except IOError:
        print 'fetching ' + url
        page = urllib2.urlopen(url).read()
        file(fname, 'wb').write(page)
    return page

def dependencies(url):
    scheme, netloc, _, _, _, _ = urlparse.urlparse(url)
    soup = BeautifulSoup(getcachedpage(url))
    text = soup.find('td', text='Run-time dependencies:')
    if not text:
        return
    tr = text.parent.parent
    td = tr.findAll('td')[1]
    for a in td.findAll('a'):
        yield (a.contents[0], '%s://%s%s' % (scheme, netloc, a['href']))

def add(name, url):
    required[name] = url
    for depname, depurl in dependencies(url):
        if depname in required:
            continue
        print "%s requires %s" % (name, depname)
        required[depname] = depurl
        add(depname, depurl)
        
def download(full_url):
    print 'downloading ' + full_url
    _, _, path, _, _, _ = urlparse.urlparse(full_url)
    fname = os.path.basename(path)
    f = file(fname, 'wb')
    def chunkify_to_eof(stream, chunksize=64*1024):
        while True:
            data = stream.read(chunksize)
            if not data:
                break
            yield data
    for chunk in chunkify_to_eof(urllib2.urlopen(full_url)):
        f.write(chunk)


# Compute dependencies before checking for installed files, since swinstall
# can let a package be installed w/o its dependencies. If there are such
# packages installed we don't want to skip their [missing] dependencies.
add(sys.argv[2], sys.argv[1])

try:
    p = Popen(['swlist'], stdout=PIPE)
except:
    print 'Warning: unable to list installed packages'
    installed = {}
else:
    installed = set(line.strip().split()[0] for line in p.stdout if line.strip())

to_install = []
for name, url in required.iteritems():
    if name in installed:
        print name + ' is already installed'
        continue
    full_url = '%s%s-ia64-11.23.depot.gz' % (url.replace('/hppd/', '/ftp/'), url.split('/')[-2])
    to_install.append(os.path.basename(full_url))
    download(full_url)

if to_install:
    print "\nAfter gunzip, run:"
    for fname in to_install:
        print "swinstall -s %s/%s %s" % (os.getcwd(), fname[:-3], fname.split('-')[0])
else:
    print 'Everything is already installed'

Friday, October 05, 2007

Congratulations, Mozy

I left backup service provider Mozy about three months ago, and yesterday they were acquired by EMC as rumored by techcrunch earlier.

The cool thing about startups is they pretty much have to hire people who are totally not qualified to do awesome things and let them try. There's no way Amazon would have hired me to write S3, but that's what I did for Mozy.

Mozy was the third startup I've been a part of, and the first to amount to anything. I was employee number #3 and saw it grow from sharing a single rented office to 50 employees in two years. With people who didn't think it was strange to wear a tie to work. Trippy.

Unfortunately I'm not there to witness the final stage of being assimilated by the Borg firsthand, but I hear that's not really any more fun than it sounds so perhaps it's just as well.

Nice work, guys.

Tuesday, October 02, 2007

Wing IDE 3, Wing IDE 101 released

Wing IDE version 3 has been released.

The list of new features is a little underwhelming. Multi-threaded debugging and the unit testing tool (only supporting unittest -- does anyone still use that old module anymore?) are nice but I don't see myself paying to upgrade from 2.1 yet. Now if they could get the GUI to keep up with my typing in Windows, I'd pay for that... I guess this is a sign that Python IDEs are nearing maturity; Komodo 4 didn't have any earth-shaking new features either, at least as far as Python was concerned.

(Personally I think someone should start supporting django/genshi/mako templates already. Maybe in 3.1, guys?)

Following ActiveState's lead, Wingware has also released a completely free version, Wing IDE 101. The main difference is that where the most essential feature Komodo Edit leaves out as an incentive to upgrade is debugging, Wing IDE 101 includes the debugger but omits code completion. Wingware also continues to offer the low-cost Personal edition.

But the really big difference between Wing IDE 101 and Komodo Edit is that you can freely use Komodo Edit for paying work. Wing IDE 101, like Wing IDE Personal has a no-commercial-use clause. (Komodo versions compared; Wing versions compared.) I'm still of the opinion that at $180, Wing Professional will pay for itself in short order, but for the hobbyist, Komodo Edit is very compelling. I've been using it myself for TCL and XML editing for several months now and it's a nice little IDE.

Too bad Komodo's emacs bindings continue to suck balls -- I mean, it's one thing to not implement fancy things like a minibuffer or kill ring, but if you can't even get C-W (cut) right, there's not much hope. Users contributed much-improved Emacs bindings to the ActiveState bug tracker way back in the version 3 timeframe. I guess ActiveState just doesn't care.