Login | Register   
RSS Feed
Download our iPhone app
Browse DevX
Sign up for e-mail newsletters from DevX

By submitting your information, you agree that devx.com may send you DevX offers via email, phone and text message, as well as email offers about other products and services that DevX believes may be of interest to you. DevX will process your information in accordance with the Quinstreet Privacy Policy.


Dig Deep into Python Internals

Python, the open source scripting language, has grown tremendously popular in the last five years—and with good reason. Python boasts a sophisticated object model that wise developers can exploit in ways that Java, C++, and C# developers can only dream of.




Application Security Testing: An Integral Part of DevOps

his article is the first in a two-part series that will dig deep to explore the fascinating new-style Python object model, which was introduced in Python 2.2 and improved in 2.3 and 2.4. The object model and type system are very dynamic and allow quite a few interesting tricks. In this article I will describe the object, model, and type system; explore various entities; explain the life cycle of an object; and introduce some of the countless ways to modify and customize almost everything you thought immutable at runtime.

The Python Object Model
Python's objects are basically a bunch of attributes. These attributes include the type of the object, fields, methods, and base classes. Attributes are also objects, accessible through their containing objects.

The built-in dir() function is your best friend when it comes to exploring python objects. It is designed for interactive use and, thereby, returns a list of attributes that the implementers of the dir function thought would be relevant for interactive exploration. This output, however, is just a subset of all the attributes of the object. The code sample below shows the dir function in action. It turns out that the integer 5 has many attributes that seem like mathematical operations on integers.

dir(5) ['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__', '__float__', '__floordiv__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__', '__init__', '__int__', '__invert__', '__long__', '__lshift__', '__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', ' __radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__str__', '__sub__', '__truediv__', '__xor__']

The function foo has many attributes too. The most important one is __call__ which means it is a callable type. You do want to call your functions, don't you?

def foo() pass ... dir(foo) ['__call__', '__class__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

Next I'll define a class called 'A' with two methods, __init__ and dump, and an instance field 'x' and also an instance 'a' of this class. The dir function shows that the class's attributes include the methods and the instance has all the class attributes as well as the instance field.

>>> class A(object): ... def __init__(self): ... self.x = 3 ... def dump(self): ... print self.x ... >>> dir(A) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'dump'] >>> a = A() >>> dir(a) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'dump', 'x']

The Python Type System
Python has many types. Much more than you find in most languages (at least explicitly). This means that the interpreter has a lot of information at runtime and the programmer can take advantage of it by manipulating types at runtime. Most types are defined in the types module, which is shown in the code immediately below. Types come in various flavors: There are built-in types, new-style classes (derived from object), and old-style classes (pre Python 2.2). I will not discuss old-style classes since they are frowned upon by everybody and exist only for backward compatibility.

>>> import types >>> dir(types) ['BooleanType', 'BufferType', 'BuiltinFunctionType', 'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType', 'DictProxyType', 'DictType', 'DictionaryType', 'EllipsisType', 'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType', 'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType', 'MethodType', 'ModuleType', 'NoneType', 'NotImplementedType', 'ObjectType', 'SliceType', 'StringType', 'StringTypes', 'TracebackType', 'TupleType', 'TypeType', 'UnboundMethodType', 'UnicodeType', 'XRangeType', '__builtins__', '__doc__', '__file__', '__name__']

Python's type system is object-oriented. Every type (including built-in types) is derived (directly or indirectly) from object. Another interesting fact is that types, classes and functions are all first-class citizens and have a type themselves. Before I delve down into some juicy demonstrations let me introduce the built-in function 'type'. This function returns the type of any object (and also serves as a type factory). Most of these types are listed in the types module, and some of them have a short name. Below I've unleashed the 'type' function on several objects: None, integer, list, the object type, type itself, and even the 'types' module. As you can see the type of all types (list type, object, and type itself) is 'type' or in its full name types.TypeType (no kidding, that's the name of the type).

>>> type(None) <type 'NoneType'> >>> type(5) <type 'int'> >>> x = [1,2,3] >>> type(x) <type 'list'> >>> type(list) <type 'type'> >>> type(type) <type 'type'> >>> type(object) <type 'type'> >>> >>> import types >>> type(types) <type 'module'> >>> type==types.TypeType True

What is the type of classes and instances? Well, classes are types of course, so their type is always 'type' (regardless of inheritance). The type of class instances is their class.

>>> class A(object): ... pass >>> a = A() >>> type(A) <type 'type'> >>> type(a) <class '__main__.A'> >>> a.__class__ <class '__main__.A'>

It's time for the scary part—a vicious cycle: 'type' is the type of object, but object is the base class of type. Come again? 'type' is the type of object, but object is the base class of type. That's right—circular dependency. 'object' is a 'type' and 'type' is an 'object'.

>>> type(object) <type 'type'> >>> type.__bases__ (<type 'object'>,) >>> object.__bases__ ()

How can it be? Well, since the core entities in Python are not implemented themselves in Python (there is PyPy but that's another story) this is not really an issue. The 'object' and 'type' are not really implemented in terms of each other.

The one important thing to take home from this is that types are objects and are therefore subject to all the ramifications thereof. I'll discuss those ramifications very shortly.

Comment and Contribute






(Maximum characters: 1200). You have 1200 characters left.



We have made updates to our Privacy Policy to reflect the implementation of the General Data Protection Regulation.
Thanks for your registration, follow us on our social networks to keep up-to-date