RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX


Drill-down on Three Major New Modules in Python 2.5 Standard Library : Page 2

The freshly minted 2.5 version of Python has lots of goodies, but the three in this article are the cream of the crop. Find out how ctypes, pysqlite, and ElementTree can save you time and aggravation in this extensive article with a ton of great sample code.

Look Who's Tokenning!
Let's take it up a notch and call a relatively complicated function such as strtok. strtok is unusual because it keeps state between consecutive calls. It takes a character buffer (char *) as its first argument and a separator string (const char *) as its second argument. Here is the C signature of strtok:

char *strtok(char *s1, const char *s2);
The strtok() function gets the next token from string s1, where tokens are strings separated by characters from s2. To get the first token from s1, strtok() is called with s1 as its first parameter. Remaining tokens from s1 are obtained by calling strtok() with a null pointer for the first parameter.

Great, a very complicated way to do s1.split(). It has its merits though if you need to process a huge buffer of text that you don't mind destroying and you don't want to pay for all the copies that are done implicitly by split(). Anyway, the main point here is how to make ctypes call this function from Python. The process with any function is similar:

  1. Observe the return type and argument types
  2. Wrap argument types that are not supported automatically
  3. Call the function
  4. Get the results
In case of strtok s2 can be a regular Python string. The return type however is not an int so it needs wrapping and s1 needs to be a writable character array. A Python string won't do for s1. ctypes provides the create_string_buffer() function for creating character arrays from Python strings (or just empty ones with a given size).

The following sample code creates a character array from the Python string '123 456 789', sets the return type to be ctypes.c_char_p and then calls strtok() repeatedly and prints each token until it returns NULL/None:

p = ctypes.create_string_buffer('123 456 789')
libc.strtok.restype = ctypes.c_char_p
x = libc.strtok(p, ' ')
while x:
            print x
            x = libc.strtok(None, ' ')

Arrays, Pointers and Callbacks
ctypes supports even more constructs that, together, cover everything you may want to do with C function parameters and results. It allows you to define pointers to any data type, arrays of arbitrary data types, and custom structs and unions. It even allows you to define callback functions in Python. That means that a C function that accepts a C function pointer as an argument and calls it during its execution will be able to call your Python function. I will use the venerable qsort function to demonstrate arrays, pointers, and a callback function. You will have to take my word about structs and unions or try it yourself.

Pointers are really easy with ctypes. You can create a pointer for any ctypes type using the ctypes.POINTER factory function. To create a pointer from an existing variable use the pointer function. To access the value of a pointer px you can use px.content.value or simply px[0]. Note that ctypes variables are always mutable.

    x  = ctypes.c_int(888)
    px = ctypes.pointer(x)
    print 'x.value=', x.value
    print 'px[0]=', px[0]
    px.contents.value = 444
    print 'x.value=', x.value
    print 'px[0]=', px[0]

x.value= 888
px[0]= 888
x.value= 444
px[0]= 444
Arrays are even easier. You need to create an array type for each element type and size. An array of three integers is a different type from an array of five integers. You create an array type by multiplying a ctypes type by an integer n. Once you have an appropriate array type you create an instance by using the array type as a factory function and passing in the elements of the array. You can access the array elements using standard indexing or standard iteration (for loop).

garbled_song = ('mo', 'mini', 'ini', 'miny')

# Create an array type of char pointers the size of the garbled song
StringArrayType = ctypes.c_char_p * len(garbled_song)

# Create an instance of this array and assign it the garbled song
strings = StringArrayType(*garbled_song)
print ' '.join(strings)

# Modify an element of the array
strings[1] = 'used_to_be_mini'
print ' '.join(strings)

mo mini ini miny
mo used_to_be_mini ini miny
Now, I'll put everything together and show you how to implement callback functions in Python (called from C). The children's song/expression "ini mini miny mo" has the nice property of being an alphabetically sorted sequence of words. I'll use the indispensable qsort function to sort the garbled sentence: "mo mini ini miny."

qsort is very convenient since it's available in libc and it operates on arrays using pointers. Here is the C signature of qsort:

void qsort(void *base, size_t length, size_t width, 
       int (*compare)(const void *, const void *));
This means that qsort is a function that returns nothing and accepts an array of arbitrary items as its first argument, the number of elements in the array as the second argument, the size (in bytes) of each element as its third argument, and finally a comparison function pointer as its fourth argument. The comparison function should accept two (const) pointers to array elements and return a negative number if the first element is smaller than the second element, 0 if they are equal, and a positive number if the second element is smaller than the first element.

This is pretty complicated, but ctypes comes through. You already know how to define pointers and arrays, so the only unknown is how to define a callback function. ctypes provides the CFUNCTYPE factory function. You pass in as first argument the result type and then the types of the arguments in order. Here is the definition of the comparison function type:

CmpFuncType = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.POINTER(ctypes.c_char_p), ctypes.POINTER(ctypes.c_char_p))
Note that I specified as arguments pointers to ctypes.c_char_p because I intend to compare strings. The signature would be different if I wanted to compare other data types. The next step is to define a Python comparison function that corresponds to this signature. This is very easy. The only semi-tricky part is that the s1 and s2 are passed in as pointers so I have to dereference them using the s1[0] and s2[0] indexing. Then I just use Python's built-in cmp function that follows the same convention of returning a negative, zero, or positive integer based on the result of the comparison.

def string_compare(s1, s2):
    return cmp(s1[0],  s2[0])
All the pieces are in place and I can finally sort the sentence. The initial setup code is identical to the arrays sample code from earlier. I set the result type of qsort to None (because qsort is a void C function) and then invoke it using the strings array and the string_compare comparison function. Note that although garbled_song is an immutable Python tuple, the array created from it is very mutable (otherwise qsort wouldn't work).

garbled_song = ('mo', 'mini', 'ini', 'miny')

StringArrayType = ctypes.c_char_p * len(garbled_song)
strings = StringArrayType(*garbled_song)

print ' '.join(strings)

libc.qsort.restype = None

print ' '.join(strings)

mo mini ini miny
ini mini miny mo

Close Icon
Thanks for your registration, follow us on our social networks to keep up-to-date