from __future__ import * 2003/12

I'm back, Darwin bumped to 7.2.0, music management software notes

I just got back yesterday morning from my ~2.5 week stint in Tokyo. I didn't really do any coding (open source or otherwise) while I was away, but I'm pretty sure I still remember how ;)

The Mac OS X 10.3.2 updater upgrades the Darwin kernel to 7.2.0 from 7.0.0. This has some effect on my package repository (mostly an effect on the tools I use to build it), so I'll probably be sweeping through and doing package updates this week. It did have a short lived effect on the official database, but Jack fixed the issue on Dec 26th.

Here's my blog-relevant TODO for the next two weeks or so:

• Finish the next version of aeve (huge changes!)

• Update my package repository

• Start writing a to-be-named (mac specific) music management software program that makes managing large collections of music possible. It will be written in PyObjC and will be dependent on iTunes via aeve as a backend (at least at first, because it's a pretty good database that abstracts ID3 really well). Kevin pointed me at musicbrainz, which in theory could make this stuff a lot more useful and efficient so I hope to integrate that, possibly in the first version. The big idea is to make it easy to write regexes to generate ID3 tag info from existing ID3 tag info and filenames, as well as bringing in metadata from other sources (like cddb, gracenote, musicbrainz, etc.) when the existing information is not complete.

Pure python "structseq" factory

This is a pattern I've been playing with lately (to be used in the next version of aeve, in the aeut/aete decoder). Basically it's a factory for "tuples with a little metadata that can be used a little bit like dicts too". Yeah I know that dict-like methods will stomp on names if there's a clash, but I don't have any use cases where that happens so my implementation doesn't consider that.

def newNamedTupleType(typename, names):
    """Return a subtype of tuple with named accessors"""
    dct = dict([(_name, property(lambda self,_i=_i:self[_i], doc=_name)) for (_i, _name) in enumerate(names)])
    dct['__names__']    = names
    dct['keys']         = lambda self: self.__names__
    dct['items']        = lambda self: zip(self.__names__, self)
    dct['iteritems']    = lambda self: itertools.izip(self.__names__, self)
    dct['values']       = lambda self: self
    return type(typename, (tuple,), dct)

It's primarily useful in conjunction with the struct module, for example, to unpack a version record, which is a pair of 16bit words, you could use something like the following:

import struct
VersionRecord = newNamedTupleType('VersionRecord', ('major', 'minor'))

def decodeVersionRecordFromPackedString(s):
    return VersionRecord(struct.unpack('HH', s))

def prettyPrintVersionRecordFromPackedString(s):
    rec = decodeVersionRecordFromPackedString(s)
    print "%(major)s.%(minor)s" % dict(rec.iteritems())

Yes, the example is contrived. But it really does make things easier when you're passing non-trivial versions of these around. Instead of having to remember the index of a field, you can just remember the name. In my typical use case, these names come from C structures, so it makes the Python code closer to what a C implementation would be like (in a good way).

I have a more generic (but mutable and not nearly as lightweight) solution to this problem (making structures easier to work with), currently called "ptypes" since it is so much like ctypes Structures API-wise. It's currently being used in a pure python package (macholib, formerly potool) for reading/writing Mach-O headers (the OS X analog of Linux's ELF, or Win32's PE). macholib is useful for dependency walking, and automation of install_name_tool like functionality. However, since it also understands symbol tables, it would be useful for determining if a particular source file contains Objective C class definitions.. which is good to know because Objective C classes can not be unloaded at runtime. It could even be used to write a symbolic debugger if one were so inclined. I have written a largely untested symbolic PowerPC disassembler in Python, so it would be neat to hook the two together eventually.

Tokyo has such cool toys

Well I got into Tokyo yesterday.. did some eating and sleeping. Today I did some shopping, bought some clothes, and got a sweet deal on the Sony DSC T1 ¹. This camera isn't available in the US yet, but it's really dope, and I figured out how to get it into English (without help, mind you). It's really small, and 5 megapixel, here's a quick test snapshot of buju yawning (warning: full res, not scaled).

Why having multiple installed Pythons is a problem on OS X

Jack nailed down the reason that extension modules fail to link properly on 10.3 if you have a separate Python installed. The -framework search path on OS X for linking (compile and runtime) hits /System last, and the Makefile for the system Python does not override the search path with -F.

If you want to use the /System version of Python at all, you really shouldn't have another Python installed, because the extensions you build won't work. Proposed fixes include a patch to distutils, or the Makefile, but the easiest solution is just not installing another Python.

You really don't need one unless you're staring a 2.3.0 interpreter bug in the face, or you need to build 10.2 compatible extensions and app bundles. This may not even work reliably, it has not been thoroughly tested. Doing it properly would at least require setting the MACOSX_DEPLOYMENT_TARGET environment variable to 10.2, but it's probably not that easy, especially for non-trivial extensions. Personally, I just don't have enough time in my day to deal with building software for 10.2 users, especially because 10.3 has new frameworks and libraries that make development easier.

Updates:

Brian Lenihan posted a "gcc_select" flavor script ¹ for swapping Python around, which may be of some use (but is of course not a real solution to this problem). I can't really recommend moving Apple Python out of the way for very long, because it is used by the faxing (I believe Python is used to generate cover pages) and printing (optionally, as far as I can tell) subsystems. Another Python will not do, because these scripts depend on the proprietary CoreGraphics wrapper that comes with OS X 10.3.

Make sure to check the trackbacks, some discussion about this issue is happening elsewhere.

meta/instance method descriptor

This is odd, but I ran into a situation where I wanted to have a method be callable from both the class (as a metamethod) and the instance (as a normal instance method). The only decent way I could think of was to use a descriptor, here's an example:

class metainstancemethod(object):
    def __init__(self, instancemethod, metamethod):
        self.metamethod = metamethod
        self.instancemethod = instancemethod

    def __get__(self, frominstance, fromclass):
        if frominstance is None:
            return self.metamethod.__get__(fromclass, fromclass.__metaclass__)
        return self.instancemethod.__get__(frominstance, fromclass)

class Meta(type):
    def magic(self):
        print self, "metamagic"

class Minor(object):
    __metaclass__ = Meta
    def magic(self):
        print self, "magic"
    magic = metainstancemethod(magic, Meta.magic)

Updates:

Phillip J. Eby came up with a better way to do this for PyProtocols. Basically, the descriptor goes on the metaclass rather than the class, which is less work and requires less magic. Check the comments (or PyProtocols) for his implementation. Thanks!