he previous articles in this "Developer's Guide to Python 3.0" series covered the major changes to the language and the standard library. This article covers some of the more esoteric changes and enhancements, along with a few lower-impact changes.
PEP-370: The New Per-User Site-Packages Directory
Until version 2.5, Python had a single site-packages directory into which it installed packages. Except for the Mac Framework build, this directory wasn't controlled by the user—so users without admin privileges weren't able to install packages to the site-packages directory; they had to request IT department help (and you how well that works).
Workarounds included using alternative installation schemes (such as --home or --prefix installation modes) and you could use PYTHONPATH to add a directory to the package lookup process, but these weren't exactly the same as relying on site-packages because they ignore .pth files.
More flexible solutions are available, such as virtualenv, which solves a much broader problem and allows you to have multiple isolated Python environments, albeit at the expense of a little administration (you have to activate and deactivate the virtual environment), but PEP-370 takes a different approach. In addition to the standard site-packages it also supports a per-user site-packages directory where packages can be installed.
Here's how all the options play out if you install the same package (anysql-0.0.5) in various ways on a Mac OS X laptop, removing any previously installed copy of the package before each fresh installation. It's worth noting that the official documentation on installing Python modules doesn't mention PEP-370 and per-user site-packages.
Here's the standard installation for anysql:
~/Contributors/Gigi/anysql-0.0.5 > python3.0 setup.py install
copying build/lib/anysql.py ->
Versions/3.0/lib/python3.0/site-packages/anysql.py to anysql.pyc
At this point, the anysql package has been installed successfully to the standard site-packages directory: /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/site-packages, and I was able to import it just fine.
|Author's Note: The egg-info in the last line is additional metadata installed by setuptools-based packages, and is not important for this discussion.|
Using the --home scheme, the installation looks like this:
~/Contributors/Gigi/anysql-0.0.5 > python3.0 setup.py install --home=~
copying build/lib/anysql.py -> /Users/gsayfan/lib/python
byte-compiling /Users/gsayfan/lib/python/anysql.py to anysql.pyc
The preceding output shows that the anysql package has been installed to ~/lib/python. To import it I had to add this directory to my sys.path:
>>> import anysql
Now, here's the --prefix scheme installation:
~/Contributors/Gigi/anysql-0.0.5 > python3.0 setup.py install --prefix=~
copying build/lib/anysql.py -> /Users/gsayfan/lib/python3.0/site-packages
byte-compiling /Users/gsayfan/lib/python3.0/site-packages/anysql.py to anysql.pyc
As you can see, it's very similar to the home scheme, except that the package gets installed to ~/lib/python3.0/site-packages. To import it I had to add this directory to my sys.path:
>>> import anysql
In general, the --prefix scheme lets you control more aspects of the installation.
Finally, here's the new per-user scheme. On the Mac OS X, packages get installed to ~/.local/lib/python3.0/site-packages. Unfortunately the ~/.local directory and all its subdirectories are owned by root, which means you need sudo access to actually install into this directory.
sudo python3.0 setup.py install --user
copying build/lib/anysql.py -> /Users/gsayfan/.local/lib/python3.0/site-packages
byte-compiling /Users/gsayfan/.local/lib/python3.0/site-packages/anysql.py to anysql.pyc
One nice thing about the --user scheme is that the user-specific site-packages directory is already in sys.path.
Overall, the user-specific site-packages scheme seems to target a very small population. There are many other existing solutions (virtual environments, portable python, PYTHONPATH, site.py, --home/--prefix installation) that seem to address the same problem area. The added value of the per-user site packages solution is that you don't have to alter sys.path, and it handles .pth files correctly.