RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Developer's Guide to Python 3.0: Python 2.6 and Migrating From 2 to 3 : Page 5

Python 3.0 has been released. Are you ready to migrate your code? Find out what you need to know to make the switch.


Implement Custom Fixers

Armed with all the knowledge about 2to3 and this simple fixer, here's a new 2to3 fixer implementation and instructions to integrate it into 2to3. The fix in this case is for the file() --> open() issue, which 2to3 stumbled on earlier in this article.

The first step is to implement a fix_file.py module that contains a FixFile class. The only part that needs explaining is the PATTERN. The effect of this pattern is to cause 2to3 to look for any invocation of a function named file with an arbitrary number of arguments.

Fixer that changes file() to open().
# Author: Gigi Sayfan
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixFile(fixer_base.BaseFix):
    PATTERN = """
                 (name='file') trailer< '(' args=any ')' >
              rest=any* >
    def transform(self, node, results):
        name = results["name"]
        name.replace(Name("open", prefix=name.get_prefix()))

Save the fix_file.py file in the fixes subdirectory. 2to3 should automatically discover the new fixer. To check, see if the new fixer shows up in the fixer list:

> 2to3 -l | grep file

As you can see, it did. The next step is to see if the file fixer actually works. Here's a small test program that uses the file() function:

file('test.txt', 'w').write('Yeah, it works!')
s = file('test.txt', 'r').read()
assert s == 'Yeah, it works!'
file = 8

The program also uses "file" as a variable (this is OK in Python, but not recommended because it hides the original file() function). Finally it prints the string "file()" for good measure. The test program runs successfully under Python 2, but fails with the following error under Python 3.

> py3 fix_file_test.py 
Traceback (most recent call last):
  File "fix_file_test.py", line 1, in <module>
    file('test.txt', 'w').write('Yeah, it works!')
NameError: name 'file' is not defined

The original version of 2to3 silently ignores the file function as demonstrated earlier. But if you run the version of 2to3 with the new file fixer in place, you get:

> 2to3 fix_file_test.py 
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
--- fix_file_test.py (original)
+++ fix_file_test.py (refactored)
@@ -1,4 +1,4 @@
-file('test.txt', 'w').write('Yeah, it works!')
-s = file('test.txt', 'r').read()
+open('test.txt', 'w').write('Yeah, it works!')
+s = open('test.txt', 'r').read()
 assert s == 'Yeah, it works!'
RefactoringTool: Files that need to be modified:
RefactoringTool: fix_file_test.py

Yes, it works! By adding the additional fixer, 2to3 can now detect and replace file() calls with open() calls. Note that the file variable and the "file()" string were not replaced because 2to3 is not doing simple string substitution. This is exactly the reason behind actually parsing the Python grammar and building a Python AST tree.

Preparing for Python 3 in a Python 2 World

Many people may find that they can't migrate right away, either because of Python 3's current state, or for backward compatibility reasons—but they may still want to be prepared. Migration occurs in two steps: First, migrate to Python 2.6 (should be a snap) and then start refactoring toward Python 3. Python 2.6 comes with a module called future_builtins. This module lets you use some Python 3 built-ins in your Python 2 code. The 2to3 tool recognizes the built-ins, and will leave them alone (because they already follow the Python 3 behavior). The defined built-ins are: ascii, filter, hex, map, oct, and zip.

Python 2.6 also contains the __future__ module, which lets you import even more Python 3 syntax and semantics: absolute_import, division, generators, nested scopes, Unicode literals, and the with statement.

The following short code snippet shows division behavior with and without the __future__ division under Python 2.6:

>>> 5 / 2
>>> 5 // 2
>>> from __future__ import division
>>> 5 / 2
>>> 5 // 2

Writing Code that Runs Under Python 2.x and Python 3

Before you try, I'll warn you: It is possible, but not worth the effort in most cases. You will have to resort to strange idioms (especially for exception handling) and limit yourself to a subset of features and third-party packages that are available to both versions. You will also need to test constantly under both Python 2 and Python 3 to make sure you didn't use a "forbidden" feature or dependency.

One Google "summer of code" project called 3to2 has the goal of allowing you to write Python 3 code and then transform it to Python 2.5 (or 2.6) code. If it takes off you may have yet another option.

In this article, you saw an overview of Python 2.6 and the process of migrating from Python 2.x to Python 3 using various scenarios, taking the current state of Python 3 third-party packages and various migration paths into account. Finally, you've seen a best-practice migration process, including a full example that migrated a sample program through all the phases—and even included an example of how to build a custom fixer for the 2to3 tool.

This article also completes the Developer's Guide to Python 3.0 series. In the earlier articles (see the Related Resources in the left column of this article) you can find a thorough introduction to Python 3, both the language and the standard library, an explanation of the rationale and the process that arrived at Python 3, and a deep exploration of every major PEP (and many minor ones), with working code.

Python 3 is the future of Python, so I hope you begin exploring and using it sooner rather than later.

Gigi Sayfan specializes in cross-platform object-oriented programming in C/C++/C#/Python/Java with an emphasis on large-scale distributed systems. He is currently trying to build brain-inspired intelligent machines at Numenta.
Email AuthorEmail Author
Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date