from __future__ import * 2004/10

dirtSimple.org

PJE, the author of PEAK and PyProtocols, finally got around to starting a blog! I definitely look forward to see what he has to say, I think that the blog will be a much better venue for his insights than article-sized emails to relatively obscure mailing lists :)

dirtSimple.org: Making simple things hard, and complex things impossible

ANN: py2app 0.1.4

py2app is the bundlebuilder replacement we've all been waiting for. It is implemented as a distutils command, similar to py2exe, that builds Mac OS X applications from Python scripts, extensions, and related data files. It tries very hard to include all dependencies it can find so that your application can be distributed standalone, as Mac OS X applications should be.

py2app 0.1.4 is a refactoring and new features release:

• altgraph, my fork of Istvan Albert's graphlib, is now part of the distribution

• py2app.modulegraph has been refactored to use altgraph

• py2app can now create GraphViz DOT graphs with the -g option (TinyTinyEdit example)

• Moved the filter stack into py2app.modulegraph

• Fixed a bug that may have been in 0.1.2 where explicitly included packages would not be scanned by macholib

• py2app.apptemplate now contains a stripped down site module as opposed to a sitecustomize

• Alias builds are now the only ones that contain the system and user site-packages directory in sys.path

• The pydoc recipe has been beefed up to also exclude BaseHTTPServer, etc.

• The altgraph from 0.1.3 had a pretty nasty bug in it that prevented filtering from working properly, so I fixed it and bumped to 0.1.4.

Known issues:

• Commands marked with XXX in the help are not implemented

• Includes all files from packages, it should be smart enough to strip unused .py/.pyc/.pyo files (to save space, depending on which optimization flag is used)

• macholib should be refactored to use altgraph

• py2app.build_app and py2app.modulegraph should be refactored to search for dependencies on a per-application basis

Download and related links are here: http://undefined.org/python/#py2app

ANN: py2app 0.1.2

py2app is the bundlebuilder replacement we've all been waiting for. It is implemented as a distutils command, similar to py2exe, that builds Mac OS X applications from Python scripts, extensions, and related data files. It tries very hard to include all dependencies it can find so that your application can be distributed standalone, as Mac OS X applications should be.

py2app 0.1.2 is primarily a bugfix release:

• The encodings package now gets included in the zip file (saves space)

• A copy of the Python interpreter is not included anymore in standalone builds (saves space)

• The executable bootstrap is now stripped by default (saves a little space)

• sys.argv is set correctly now, it used to point to the executable, now it points to the boot script. This should enhance compatibility with some applications.

• Adds an "Alias" feature to modulegraph, so that sys.modules craziness such as wxPython.wx -> wx can be accomodated (this particular craziness is also now handled by default)

• A sys.path alternative may be passed to find_modules now, though this is not used yet

• The Command instance is now passed to recipes instead of the Distribution instance (though no recipes currently use either)

• The post-filtering of modules and extensions is now generalized into a stack and can be modified by recipes

• A wxPython example demonstrating how to package wxGlade has been added (this is a good example of how to write your own recipe, and how to deal with complex applications that mix code and data files)

• PyRuntimeLocations is now set to (only) the location of the current interpreter's Python.framework for alias and semi-standalone build modes (enhances compatibility with extensions built with an unpatched Makefile with Mac OS X 10.3's Python 2.3.0)

Known issues:

• Includes all files from packages, it should be smart enough to strip unused .py/.pyc/.pyo files (to save space, depending on which optimization flag is used).

Download and related links are here: http://undefined.org/python/#py2app

py2app: Packaging wxGlade

(Note: this information is relevant to py2app 0.1.2 - which is currently just on the svn trunk)

In order to see how hard it would be to package a complicated application such as wxGlade, I decided to give it a try. wxGlade is quite a dynamic beast with regard to code generation and widget plugins, so it was a bit of a pain to deal with. I actually had to add several features to py2app in the process:

• A feature where one module can be "aliased" to another module. I needed this because wxPython.wx points to wx, but is not actually on the filesystem.

• An extensible module filtering system, so a recipe can say "yes" or "no" to any extension or python module it comes across. Before this, there was only one static filter that would say "no" to standard library modules when using a vendor Python.

wxGlade causes problems because it needs to list the "codegen" and "widgets" directories, and load code from them. This would be easy if:

• The path were configurable in some way (using sys.path, a separate variable, etc.). Packages go into .../Python/site-packages/, where it expected to see them at .../

• The list of modules were configurable in some way (right now it expects to perform an os.listdir)

The naive way would be to just include them as data files, however this prevents dependency analysis from occurring. It turns out that wxGlade did indeed cause new dependencies in these modules so a more complicated solution was required. The solution to this particular problem is in three parts:

1. Add "codegen" and "widgets" as data files

2. Add a wxglade recipe that scans all of these plugins for dependencies

3. Have the wxglade recipe install a filter that makes sure these dependencies aren't copied into the normal place

The other customization I used was a custom boot script, wxGlade.py. This boot script does two things:

1. Starts up wxGlade as per wxglade.py, but in a simpler way (less is needed, the environment is already sane)

2. Monkeypatches a bug in wxGlade so that it can open the help files

The example setup.py and bootstrap wxGlade.py are located at: http://undefined.org/python/wxGlade-0.3.4-py2app.tgz

Note that setup.py expects a wxGlade 0.3.4 distribution to be unpacked in ./wxGlade-0.3.4

ANN: py2app 0.1.1

py2app is the bundlebuilder replacement we've all been waiting for. It is implemented as a distutils command, similar to py2exe, that builds Mac OS X applications from Python scripts, extensions, and related data files. It tries very hard to include all dependencies it can find so that your application can be distributed standalone, as Mac OS X applications should be.

py2app 0.1.1 is primarily a bugfix release:

• Several problems related to Mac OS X 10.2 compatibility and standalone building have been resolved

• Scripts that are not in the same directory as setup.py now work

• A new recipe has been added that removes the pydoc -> Tkinter dependency

• A recipe has been added for py2app itself

• a wxPython example (superdoodle) has been added. Demonstrates not only how easy it is (finally!) to bundle wxPython applications, but also how one setup.py can deal with both py2exe and py2app.

• A new experimental tool, py2applet, has been added. Once you've built it (python setup.py py2app, of course), you should be able to build simple applications simply by dragging your main script and optionally any packages, data files, Info.plist and icon it needs.

Known issues:

• Includes all files from packages, it should be smart enough to strip unused .py/.pyc/.pyo files (to save space, depending on which optimization flag is used).

• The default PyRuntimeLocations can cause problems on machines that have a /Library/Frameworks/Python.framework installed. Workaround is to set a plist that has the following key: PyRuntimeLocations=['/System/Library/Frameworks/Python.framework/Versions/2.3/Python'] (this will be resolved soon)

Download and related links are here: http://undefined.org/python/#py2app

SVN::Mirror

Mirroring of Subversion repositories isn't very well supported at this point. It seems that the only way to do it without filesystem access to the server is with a perl package, SVN::Mirror.

Installing SVN::Mirror was a bit troublesome. Basically you need to:

• Install Subversion >= 1.0.4

• Install the Subversion Perl bindings (make install-swig-py, I think)

• Install SVN::Simple from CPAN (an unlisted dependency)

• Install SVN::Mirror from CPAN (say no if it asks you about VCP)

After that, you should have a svm utility, which you can mirror your repository with. There is a documentation bug in the current version where it says that that init takes a skip_to argument. It does not. sync does, however.

In order to get the mirror going, you need to do something like this

$ setenv SVMREPOS ~/svm
$ svm init mirror/bob http://svn.red-bean.com/bob
$ svm sync mirror/bob 3

The 3 is important, because it's the skip_to argument that allows your mirror's revisions to match your repository. Don't ask.

To check it out, you do:

$ svn co file://$SVMREPOS/mirror/bob bob

And to update the mirror:

$ svm sync mirror/bob

ANN: py2app 0.1

py2app is the bundlebuilder replacement we've all been waiting for. It is implemented as a distutils command, similar to py2exe.

Even at this early stage, it's probably the most advanced python packaging utility out there with regard to dependency analysis. It's at a "works for me and the two other people that have tried it" stage. It's not particularly documented, and I don't have time to document it today, but you should be able to get along given the examples. Since it is very similar to py2exe in design, you can also borrow ideas from there.

If you have any issues with py2app, please let me know. Provide source if possible. I'm particularly interested in problematic packages which use import tricks or in-package data files such as PIL or pygame. These two are explicitly supported by the extensible "recipe" mechanism of py2app, but there are surely others. I believe PEAK, ZConfig, Twisted, etc. may do one or both of these things. Until a recipe is created for a particular problematic package, you may always just include it wholesale with the -p argument.

Download and related links are here: http://undefined.org/python/#py2app

py2app kicks it

I've spent some more time with py2app this week, and I'm pretty sure it's way beyond bundlebuilder's capabilities at this point (except for half-assed plugin building). It actually has a significantly smarter scheme for dealing with dependencies than py2exe now. Feel free to borrow it :)

I rewrote the standard library's modulefinder to use a dependency graph (py2app.modulegraph), and I dropped all of that "missing module" craziness. I have never seen a case where the missing module reports were useful at all, so it's gone.

Using a graph allows you to selectively prune dependencies, and it makes adding implicit dependencies much more sane. For example, you definitely don't want a Tkinter dependency just because PIL.Image does "import FixTk" for initialization purposes. However, it's entirely likely that the user might want Tkinter later, so in the PIL recipe I simply remove the PIL.Image -> FixTk dependency. Importing ImageTk, or anything else that depends on Tkinter, will of course cause Tkinter get included.

I have added full package include support, but have not done a no-zip distribution mode. This is of little utility with the full package include mode and alias build mode for development, so I may drop it altogether. If you have a strong opinion about this, state it now, or else it will almost definitely go away.

I have also added an extensible recipe system, and included recipes for PIL and pygame. The PIL recipe is quite complex, because PIL's initialization is. The pygame recipe simply forces it to be included as a full package. The cool thing about py2app's full package support is that it still does mach header rewriting of the extensions it can find (via macholib).