from __future__ import * 2004/8

Using the compiler module to help internationalize your software

While working on I18N for the Talking Panda iLingo installers I needed a reliable way to extract strings from Python code. I decided to use the _(u'string') pattern, even though I'm not using gettext. This simple class uses the AST facilities in Python to extract such strings, as long as they are constants in the code.

from compiler import parseFile
from compiler.visitor import ASTVisitor
from compiler.ast import Name, Const
from sets import Set

class StringVisitor(object):
    def __init__(self):
        self.strings = Set()
        self.visitor = ASTVisitor()

    def findStrings(self, fn):
        self.visitor.preorder(parseFile(fn), self)
    
    def visitCallFunc(self, node):
        fn = node.node
        if not (isinstance(fn, Name) and
                fn.name == '_' and
                len(node.args) == 1 and
                isinstance(node.args[0], Const)):
            for child in node.getChildNodes():
                self.visit(child)
            return
        self.strings.add(node.args[0].value)

if __name__ == '__main__':
    import sys
    sv = StringVisitor()
    for fn in sys.argv[1:]:
        sv.findStrings(fn)
    lst = list(sv.strings)
    lst.sort()
    for s in lst:
        print s.encode('unicode_escape')

Forget Spreadsheet::ParseExcel!

I've been working on some automation scripts to take data out of excel and do useful things with it, and I hit a big stumbling block with Spreadsheet::ParseExcel. Unicode SUCKS in Perl, and Spreadsheet::ParseExcel does nothing at all to help you with that. Each cell gets its own encoding ('ucs2', '_native_' which I haven't seen, or it's simply undef.. which seems to be latin-1). Anyway, it's completely bogus, so I started shopping around for another implementation.

Andy Khan's JExcelApi does the trick and is light-years more correct and faster than the alternatives I have tried (other than the time it takes a JVM to start). Not only that, but by default the jar does exactly what I want it to do. It gets the unicode right, and everything worked perfectly the first time. My dealings with Excel files have been reduced to the following:

java -jar -Djxl.encoding=latin1 jxl.jar -xml EXCELFILE.xls

And the Python code to parse the workbook xml document from jxl looks roughly like this:

from xml.dom import minidom

def parseDocRows(doc):
    for row in doc.getElementsByTagName(u'row'):
        rowdata = [
            u''.join([x.nodeValue for x in col.childNodes])
            for col in row.getElementsByTagName(u'col')]
        if rowdata:
            yield rowdata

if __name__ == '__main__':
    import sys
    for row in parseDocRows(minidom.parse(file(sys.argv[1]))):
        print row

Thanks!

py2app begins: find_modules.py

I started ripping apart py2exe to see what tasty little chunks could be used in py2app (its evil Mac OS X clone-to-be). It seems like a lot of py2exe is almost cross-platform-ish, but most of the logic is just in one huge Python script.

So far the only working code I have is the modulefinder based code, plus a whole bunch of hacks to ignore modules that should be ignored. The result is find_modules.py, which is almost 400 lines long and should be totally cross-platform-ish (far more so than py2exe, anyway). Basically it knows how to ignore a non-definitive list of platform specific modules based upon which platform you're currently using.

Perhaps at some point this little ugly monster could be shared between several packaging projects (py2exe, py2app, cx_Freeze, etc.)? I'm not sure if it can go into py2exe as-is, because I used a bunch of Python 2.3+ features.

macholib does symbols now

I wrote some code last night that adds some rudimentary Mach-O symbol table support to macholib. I did this primarily because py2exe introspects dll and pyd files to see if they reference PyImport_ImportModule so that it can potentially display a warning message since the extension will be able to dynamically add dependencies to the application in a way that is undetectable without actually running the code.

>>> import macholib
>>> s = macholib.SymbolTable(macholib.MachO('pygame/display.so'))
>>> for (nlist, name) in s.undefsyms:
...   if name == '_PyImport_ImportModule':
...     print name
...     break
... 
_PyImport_ImportModule

PIL plugins for obscure image formats

I've written two read-only PIL plugins for obscure image formats over the past few years. I offered them up for PIL inclusion on image-sig, but received no response whatsoever.

Anyway, here they are:

icns:

Decoder for the Mac OS X 'icns' resource format

SoftimageImage:

Decoder for lossless Softimage PICT (.pic) files

Changing win32 icons from a Mac with Perl

The current build procedure for Talking Panda involves taking two stub applications (Mac OS X, Win2K/XP) and replacing their resources. This is extremely easy for the Mac installer, because it's just a bunch of folders. It's actually so convenient that I use the Mac application bundle to house the majority of resources that the Windows installer uses. Unfortunately, I do have to change one in-exe resource for the Win32 installer: the application's icon.

The Win32 application icons are still created "by hand" using Axialis IconWorkshop, which I highly recommend. It understands the Mac OS X icns format and does everything in a few shortcut keystrokes. Eventually I hope to write a script to generate the Windows icons on the fly with PIL, but I had a bit of trouble trying to find updated specs for the Windows XP additions to the ICO format.

I did some research into the Microsoft Portable Executable File Format, but was unable to find any portable C or Python libraries that could create a new executable using an existing one as a template. py2exe does this, but it uses a non-portable C library. Whenever this happens, I generally turn to Perl. CPAN is pretty good about having modules to read/write various file formats (the only other Perl I've used in the past few years is for reading/writing Excel spreadsheets).

Unsurprisingly, CPAN did indeed have what I needed in a module called Win32::Exe, which has a pretty painless API. And here it is, my most recent Perl monstrosity. It takes 3 arguments, the source executable, the new icon, and the destination executable. If someone wrote an equivalent Python module, I'd love to switch, but this does the job quite nicely for now.

#!/usr/bin/perl

use Win32::Exe;
use strict;
my $exe = Win32::Exe->new(shift());
$exe->update(icon => shift());
$exe->write(shift()); 

(I hate Cheetah. Please, somebody write a simple pycs-compatible blog tool that uses ReST and isn't PyDS.)

Windows/Python developer needed, urgent!

The in-development Windows installer for Talking Panda needs some professional help. Currently it's a mess of Tkinter and win32all COM code. It works, on some computers anyway, and I need a domain expert to help me clean it up and make it right. This is a paid gig, and I need someone ASAP. Ideally the person would have the following environment available to them (preferably multiple computers):

• Windows 2000/XP

• A 3rd generation or later (iPod Mini, Click-Wheel) iPod

And they would be proficient with:

• Python 2.3

• Tkinter (unless you can and want to rewrite in some other toolkit, as long as you can do it quickly)

• COM (communication with iTunes is done via COM)

• Win32 API

Please send any inquiries to bob@redivi.com.

NOTE: If you don't have an iPod an really want to help out, I will send you one. However, someone who is moderately experienced with the iPod is preferable, of course.