Browse DevX
Sign up for e-mail newsletters from DevX


A Brief Rundown of Changes and Additions in Python 3.1 : Page 4

Changes to the core language, the standard library, and some welcome performance improvements make Python 3.1 a balanced and worthwhile release.




Building the Right Environment to Support AI, Machine Learning and Deep Learning

Version Info

Python 3.1 uses the named tuple construct introduced in Python 3.0 to make the version info more readable. In Python 2.5:

>>> import sys >>> sys.version_info (2, 5, 4, 'final', 0)

In Python 3.0:

>>> import sys >>> sys.version_info (3, 0, 1, 'final', 0)

In Python 3.1:

>>> import sys >>> sys.version_info sys.version_info(major=3, minor=1, micro=0, releaselevel='final', serial=0)

Pickling of Partial Functions

Partial functions are one of my favorite functional features. They allow you take a function that accepts X arguments, make some of the arguments fixed (static), and get a new function that accepts only the arguments you didn't specify. A trivial example is an add() function that accepts two arguments, adds them, and returns the result. Now, if you fix one of the arguments to be 5, you end up with a new function that accepts only one argument, adds 5 to it and returns the result:

from functools import partial def add(a, b): return a + b add5 = partial(add, 5) assert add5(8) == 13

Partial functions are very useful when working with APIs that require arguments that are always the same in your use case. Consider a web API that requires a username and a password in each method signature. If you create partial functions that fix the username and password your life will be much easier, because you don't have to pass the arguments around. Arguably, your code will be safer too, because the user name and password will not show up in every call site.

However, partial functions had an annoying limitation up to Python 3.1. They could not be pickled. Python 3.1 addressed this issue. Here's an example:

import pickle from functools import partial def add(a, b): return a + b s = pickle.dumps(partial(add, 10)) add10 = pickle.loads(s) assert add10(8) == 18

This code snippet passes under Python 3.1, but fails under Python 3.0 and earlier with the following error:

Traceback (most recent call last): File "test_partial_pickle.py", line 12, in <module> s = pickle.dumps(partial(add, 10)) File "/Library/Frameworks/Python.framework/Versions/ 2.5/lib/python2.5/pickle.py", line 1366, in dumps Pickler(file, protocol).dump(obj) File "/Library/Frameworks/Python.framework/Versions/ 2.5/lib/python2.5/pickle.py", line 224, in dump self.save(obj) File "/Library/Frameworks/Python.framework/Versions/ 2.5/lib/python2.5/pickle.py", line 306, in save rv = reduce(self.proto) File "/Library/Frameworks/Python.framework/Versions/ 2.5/lib/python2.5/copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle partial objects

Pickled functions (and partial functions) are all the rage if you use the processing module for parallel programming. This module, which has been part of the standard library since Python 2.6, is the best Python solution for taking advantage of the power of today's multi-core machines. Under the covers, the processing module pickles everything it sends between processes, so picklable partial functions increase the expressive power and the tools available to you.

Unit Test Improvements

Python has a standard unittest module that you use to write xUnit-style tests. You can reuse setup/teardown code, organize your tests in suites (collections of tests), and even run your tests. Here's a unit test for the add5() partial function. The TestAdd5 class is derived from unittest.TestCase and defines a setUp() method called before executing each test method. It ensures that some consistent state is available to every test method. The test methods call unittest's assertEqual() and assert_() methods. If any call fails, the hosting test method considers that a failure, and moves on to the next test.

import unittest from functools import partial def add(a, b): return a + b add5 = partial(add, 5) class TestAdd5(unittest.TestCase): def setUp(self): self.values = range(1, 10) def test_positive(self): for v in self.values: self.assertEquals(add5(v), v + 5) def test_negative(self): for v in self.values: self.assertEquals(add5(-v), 5 - v) def test_zero(self): self.assert_(add5(0) == 5) if __name__ == '__main__': unittest.main()

In this case, unittest.main() runs when the module is run, locates all the test classes (just one in this case), runs their test methods and reports the results:

... ------------------------------------------------- Ran 3 tests in 0.000s OK

Author's Note: You can see that each test method is really a separate test case that can pass or fail independently, so it is somewhat of a misnomer to call the test class the unittest.TestCase.

Python 3.1 added the capability to skip tests and to mark tests as "expected to fail." Skipping tests is useful in many scenarios. For example, you may want to save some time and run only the tests you are actively working on at the moment, or you may want to skip some platform specific tests if you run on a different platform (yes, you can conditionally skip tests). Expected failures are useful when you're working in test-first mode, but need to check-in your code before the new functionality works. In this case, the test for the new functionality is expected to fail until you fix the code or implement the new feature. Here's a smarter version of add5 that lets it to operate on strings that contain numbers such as 3. The new version contains an added test_string method, which takes each value from the sequence, turns it into a string, and then feeds it to add5:

def test_string(self): for v in self.values: self.assertEquals(add5(str(v)), v + 5)

Running the test now results in an error as expected:

> python3.1 unittest_test.py ..E. ============================================= ERROR: test_string (__main__.TestAdd5) --------------------------------------------- Traceback (most recent call last): File "unittest_test.py", line 26, in test_string self.assertEquals(add5(str(v)), v + 5) File "unittest_test.py", line 4, in add return a + b TypeError: unsupported operand type(s) for +: 'int' and 'str' ---------------------------------------------------------------- Ran 4 tests in 0.001s

Now, let's skip the zero test and allow the test_string method to fail:

@unittest.skip("skipping this guy") def test_zero(self): self.assert_(add5(0) == 5) @unittest.expectedFailure() def test_string(self): for v in self.values: self.assertEquals(add5(str(v)), v + 5)

Now, the tests run successfully, reporting the skipped and expected failures:

> python3.1 unittest_test.py ..xs ----------------------------------- Ran 4 tests in 0.001s OK (skipped=1, expected failures=1)

Another new unittest feature is that you can use assertRaises as a context manager:

with self.assertRaises(ImportError): import no_such_module

In Python 3.0 and earlier you had to wrap the code in yet another function and pass it to assertRaises:

def import_no_such_module(): import no_such_module self.assertRaises(ImportError, import_no_such_module)

In addition many new assertXXX() functions have been added. Check the documentation for all the details.

Thanks for your registration, follow us on our social networks to keep up-to-date