Python implementation details
March 26, 2009
I’m sitting here at PyCon at the Python Language summit discussing differences between Python versions, and the topic of implementation details came up. The main part of getting Django working on alternate Python VMs was fixing the various assumptions we made about implementation details of Python.
I went back and dug through Django’s ticket tracker for the issues alternate VM issues (most from the Jython developers, some from the PyPy developers) that we had to fix; it’s a pretty instructive list of things you shouldn’t rely on if you’d like your Python code to run on alternate VMs:
- Using the __builtins__ super-global instead of import __builtin__.
- Assumptions about the terminal that’s available. Not all terminals will support curses, so anything involving term colors or even isatty is a bit suspect.
- Assumptions about what’s available in the os module. os.access, os.chmod, os.getpid, may not be available.
- Relying on features of “old-style” classes, or dynamically creating classes at runtime based on types.ClassType. In general, avoid old-style classes completely if at all possible.
- Assumptions about function internals, especially im_func.func_code.
- Assumptions about dict ordering. Remember that dict really is unordered; just because you “usually” get the same order doesn’t mean that you’ve gotten a magic ordered dict.
- Making code that requires gc.collect.
- Relying on built-in exception message strings; some (IndexError) differ between VMs.
- Not paying close attention to type conversion at boundaries (sockets, databases); you might get a boxed type (i.e. a java.lang.String) from the underlying VM where you’re not expecting it.
- Not thinking carefully about PYTHONPATH, or ignoring things like JYTHONPATH.
- Not using __import__ properly. Use importlib instead.
- Assuming the existence or format of .pyc or .pyo objects, or assuming that Python modules must be *.py, or really any assumptions about the names of files (thanks, Leo).